Mr Prodi, who had led his centre-left coalition for 20 months, was defeated by five votes, despite the support of several unelected lifetime senators.
President Giorgio Napolitano must now choose whether to call a snap election or appoint an interim government.
Mr Prodi has been asked to continue in a caretaker capacity until then.
Correspondents say the centre-right opposition alliance led by the former Prime Minister, Silvio Berlusconi, is regarded as well-placed to win an early election.
Mr Berlusconi announced a surprise party at his house in Rome.
"We will say what we want to do in the first 100 days of our government," he said.
'Political vacuum'
Mr Prodi won a confidence vote in the lower house of parliament on Wednesday, but had been widely expected to resign rather than risk a humiliating defeat in the Senate, with many commentators saying he had no chance of winning.
The loss of the small, centrist Udeur party's three seats in the upper house left him without a majority and requiring the support of several unelected life senators.
Stopping the government's work is a luxury Italy cannot afford
Romano Prodi
Analysis: Chaos and betrayal Profile: Romano Prodi
Udeur pulled out, citing a lack of support for its leader, the former Justice Minister Clemente Mastella, who resigned after he and his wife were named in a corruption probe. They maintain they are innocent.
President Napolitano is reported to have advised Mr Prodi to consider resigning so the vote could be avoided.
In an impassioned speech before the vote, Mr Prodi warned that a defeat for the government would mean a paralysing "political vacuum" for weeks until a new coalition could be formed, or new elections held.
"Stopping the government's work is a luxury Italy cannot afford," he said.
Nuccio Cusumano (centre) fainted after reports of abuse and spitting
He said that the country needed continuity and backing the government would allow it to deal "urgently" with electoral reform, economic renewal and its role in international affairs.
At one stage it seemed he had won a much-needed defection when Udeur senator Nuccio Cusumano announced he was breaking with his party to back the government.
The move forced a brief suspension of the session after shouts that the senator was a "traitor" and a "clown" and one senator made a hand gesture as if to shoot Mr Cusumano.
Mr Cusumano was reportedly spat on, and then fainted, before being carried out on a stretcher.
At the end of the session, however, Mr Prodi's government fell four votes short of the 160 it needed to survive, with 161 senators voting against and one abstention.
Electoral system
The BBC's Christian Fraser in Rome says that Italy's president must now decide what happens now.
Mr Napolitano will begin consultations with the speakers of the Senate and the Chamber of Deputies on Friday afternoon, his office said in a statement after Mr Prodi resigned.
HAVE YOUR SAY If Berlusconi is the other alternative, then I certainly hope Prodi can survive. Jim Bo, Sweden
There is widespread agreement among several senior senators - even among the prime minister's opponents - that the country needs new electoral laws, our correspondent says.
Under the current system rushed in by Mr Berlusconi, smaller parties with only a handful of seats hold the balance of power in parliament.
Mr Napolitano is thought to favour appointing an interim government of technocrats to force through such reforms.
But other politicians, particularly those belonging to Mr Berlusconi's Forza Italia party, which is leading the polls, insist an election is the only way forward, he adds.
Prodi in 2006. 
Romano Prodi, the Prime Minister of Italy, resigned Thursday evening after losing a vote of confidence in the Senate by a vote of 161-156.

The vote comes in the wake of a growing crisis regarding corruption charges against Prodi's former Minister of Justice, Clemente Mastella.  Prodi's fate was sealed when the UEDUR Populars, a smaller component of Prodi's ruling Olive Tree coalition, broke ranks and voted against him.

Prodi did win the confidence vote in the Chamber of Deputies, but was not expected to win the vote in the Senate.  Some believed he would resign before the Senate vote and avoid the embarrassment of losing the majority approval of Parliament.

The President of Italy, Giorgio Napolitano, must now decide whether to appoint a new Prime Minister or to dissolve Parliament and schedule new elections. Some want a reform of election rules before a new election is called. Recent opinion polls indicate that Silvio Berlusconi would become Prime Minister once more if elections were held.

It was the current election rules, introduced by Berlusconi during his most recent stint as Prime Minister, that allowed Prodi to assume power after the 2006 parliamentary elections.

"Now we must go to vote. We will say what we want to do in the first 100 days of our government," said former prime minister and opposition leader Berlusconi, demanding elections.

In a speech prior to the vote, Prodi said, "Stopping the government's work is a luxury Italy cannot afford," and warned that Italy "cannot afford a power vacuum."