In discussing the Fed's recent to decision to reinvest the money it earns from mortgage backed securities back into long-term government debt the New York Times presented at length the views of Carl Walsh, an economics professor at the University of California, Santa Cruz. He warned that if banks suddenly withdrew the $1 trillion in reserves that they held at the Fed it could generate inflation.
While this is in principle possible, it would have been worth noting the mechanism through which inflation would be generated. The banks would have to lend out the money to firms who invest it, thereby increasing employment. This would lead to more jobs, higher wages, and then higher demand, which would allow firms to be able to raise prices.
This process takes time. The Fed would have ample opportunity to raise interest rates and slow growth before inflation got too high. Most people would probably be willing to take the risk that the economy might jump back to full employment too quickly.