A masked officer leads a terror suspect (in blue) to a helicopter
Federal prosecutor Monika Harms said the three had trained at camps in Pakistan and procured some 700kg (1,500lbs) of chemicals for explosives.
She said the accused had sought to target facilities visited by Americans, such as nightclubs, pubs or airports.
Defence Minister Franz Josef Jung said the men had posed "an imminent threat".
Media reports said the men were planning attacks against a US military base in Ramstein and Frankfurt airport.
The US praised the actions of the German government and said the incident showed everyone needed to be vigilant in finding "terrorists".
German government sources have told the BBC they believe at least seven members of the cell are still at large.
Surveillance effort
Ms Harms said the men planned to use vehicles loaded with the explosives to kill or injure large numbers of people.
They obviously planned these attacks on the orders of an international network,
Wolfgang Schaeuble
German Interior Minister
German fears over plotters
The arrests had prevented "massive bomb attacks", she said.
The suspects, aged 22, 28 and 29, were alleged to be members of the German cell of a group she named as Islamic Jihad Union.
Two of the men were German nationals who had converted to Islam, while the third was a Turkish man.
Ms Harms said the al-Qaeda-linked group had set up the cell last year.
Joerg Ziercke, the head of Germany's federal crime office, said the men had a "profound hatred of US citizens".
They had been under surveillance since December, but the authorities decided to act when it became clear the men were planning to move their huge stores of hydrogen peroxide.
They were arrested on Tuesday afternoon in a raid on an apartment in North Rhine-Westphalia.
Another 40 raids were been carried out on properties across the country.
Mr Ziercke told Agence France-Presse news agency one of the three arrested had been detained for allegedly spying on a US base at Hanau in December, but was freed shortly afterwards.
Interior Minister Wolfgang Schaeuble called the men "very dangerous terrorists".
"They obviously planned these attacks on the orders of an international network," he said.
Mr Schaeuble said one man was linked to a mosque in the southern city of Neu-Ulm that authorities had suspected for years was an extremist base.
White House spokesman Tony Fratto said the arrests were a "reminder that we need to be vigilant in all of our efforts to find terrorists, deny them resources and hiding places, and prevent attacks".
Growing concerns
Frankfurt airport is continental Europe's busiest, and the base at Ramstein in western Germany is a major transport hub for US military operations.
Germany, which has soldiers in Afghanistan but did not send troops to Iraq, has been largely spared terrorist attacks.
However, there have been growing concerns that Islamist militants are operating in the country.
Six years ago, the northern city of Hamburg was thrust into the spotlight after it emerged a cell had used it as a base for planning the 11 September 2001 attacks in the US.
Last summer, two identical suitcase bombs were planted on two commuter trains in the German cities of Dortmund and Koblenz but they failed to explode.
On Monday, two people with suspected links to al-Qaeda were arrested in Denmark on suspicion of planning a bomb attack.
Terminal 2 de l'aéroport de Francfort
Trois militants islamistes ont été arrêtés par la police allemande à Oberschloedorn, un village retiré situé au nord de Francfort. Les suspects, qui se sont entrainés au Pakistan appartiennent à l'. Ils sont également convaincus d'avoir des liens étroits avec Al Qaida.
Deux des supsects sont citoyens allemands et le troisième est un résident turc d'Allemagne. Ils ont été découverts en possession d'environ 750 kg de peroxyde d'hydrogène, qui est le composé chimique principal utilisé dans les attaques à la bombe suicides au Royaume-Uni en juillet 2005. Un des suspects est supposé avoir voulu commettre ce genre d'attaque.
, a indiqué le Procureur fédéral d'Allemagne, Monika Harms dans une conférence de presse, soulignant l'ampleur de l'enquête qui a duré six mois. Les cibles des attaques devaient sans doute comprendre les lieux utilisés par les Américains, comme l'aéroport international de Francfort, et la base militaire américaine de Ramstein.
Le plan était prévu pour être exécuté la semaine prochaine, 6 ans après les attentats du 11 septembre 2001.