Mr Karami denies any involvement in the attack on Mr Hariri
Tens of thousands of people waved Lebanese flags and demanded that Syria remove its troops from the country.
Prime Minister Omar Karami announced the resignation two weeks after the murder of his predecessor Rafik Hariri.
The US hailed it as an "opportunity" for Lebanon, calling for fair elections free of Syrian influence.
Delight
Mr Karami said in his announcement: "I am keen the government will not be a hurdle in front of those who want the good for this country.
"I declare the resignation of the government that I had the honour to head. May God preserve Lebanon."
His announcement came as an opposition-sponsored motion of no-confidence in the government was being debated in parliament.
The statement prompted delight from at least 25,000 protesters estimated to have gathered in Beirut's Martyrs Square despite a ban on demonstrations.
"Karami has fallen, your turn will come, Lahoud, and yours, Bashar," the demonstrators chanted, referring to Lebanese President Emile Lahoud and his Syrian counterpart Bashar al-Assad.
Mr Lahoud accepted the resignation of the government and asked it to continue in a caretaker capacity, a statement said.
The immediate reaction from Syria, which backs the Lebanese government, was non-committal, saying only that it was "an internal affair" for Lebanon.
However, a BBC correspondent in Damascus says the Syrian authorities must be worried the situation in Lebanon is spiralling out of their control and might result in a new government demanding the immediate withdrawal of Syria's estimated 15,000 troops in the country.
Government accused
Both Mr Karami's government and the Syrian government have been accused of involvement in the 14 February assassination of Mr Hariri - charges they deny.
Earlier, Mr Karami - who took office after Mr Hariri resigned last year - said those who accused his government of involvement in the killing "committed a grave injustice".
Before the parliamentary debate opened, MPs observed a minute's silence in memory of Mr Hariri.
A former minister said the government bore partial responsibility for the killing.
"I accuse this government of incitement, negligence and shortcomings at the least, and of covering up its planning at the most... if not executing," the attack, former minister Marwan Hamadeh said.
We want to see free and fair elections take place [in Lebanon] this spring
David Satterfield
US Deputy Assistant Secretary of State
Protesters watched events from giant TV screens in the square. Many had spent the night there, wrapped in blankets or under tents, before the ban came into force at 0500 (0300 GMT).
Despite army checkpoints, people managed to get to the square and there were no serious clashes.
Many schools and businesses remained shut across the country, following a call by the opposition for a general strike.
Earlier, US Deputy Assistant Secretary of State David Satterfield, on a visit to Lebanon, called on Syria to withdraw its troops in compliance with UN resolution 1559, passed in September.
Syria says not even the Lebanese want a full Syrian withdrawal, but last week it said it would draw troops back from western Lebanon to areas nearer the Syrian border.
right 

O governo do Líbano renunciou 28 de fevereiro em meio a um protesto de 10 mil pessoas na Praça dos Mártires. O chefe do governo apoiado pela Síria, o   Primeiro Ministro Omar Karami foi obrigado  a renunciar devido a pressão em semanas recentes, principalmente  depois da morte do ex-Primeiro Ministro Rafik al-Hariri, no início de fevereiro.

Os protestos são contra a grande presença militar  da Síria no Líbano.  A Síria tem mais de 14 mil soldados em território libanês.

Após anunciar  a sua renúncia Karami disse:  "Está fora de cogitação  o fato de este  governo tornar-se um empecilho ao bem do país,  assim, eu anuncio a renúncia  a ele,  o qual tive a honra de conduzir".  Em resposta à decisão de renúncia do  governo, o líder de oposição Walid Jumblat disse: "O povo venceu."

O governo sírio,  que apoiou o governo do sr. Karami,  disse que o caso  é  "um assunto interno" do Líbano.