After a vote in the Senate, the Federal Reserve will proceed with cutting debit card "swipe fees" by billions in a win for merchants—especially small retailers.

In a vote of 54 to 45, the amendment was not agreed to, as it needed 60 votes to pass.

According to the Fed, banks charged retailers more than $16 billion in swipe fees in 2009 on 38 billion consumer transactions.  Critics said that's too much.

"Can we stand up representing the people of this country and say that is fundamentally unfair?" Sen. Richard Durbin (D-Ill.) said. "You've got to treat our consumers and merchants fairly.  If we can't stand up and do that, why are we here?"

But some senators wanted to delay the cuts saying the loss of revenue for small banks could shut some down.

"I could care less about the Wall Street banks.  They're going to do fine," Sen. Jon Testor (D-MT) said. "But I'll tell you what... we lose the banks in our small towns in Montana or Wyoming or Tennessee and you can put another nail in the coffin of rural America."

With the Senate vote the Fed can proceed in July with mandating a proposed 12 cent limit on swipe fees, down from the current average of 44 cents—a 70% reduction.

Financial firms argue many retailers, especially big chains, will not pass the savings along to customers through lower retail prices and will just pocket them for profit.

Peter Barnes joined FOX Business Network (FBN) in September 2007. He serves as FBN's senior Washington correspondent.