Mr Yusuf (l) and his prime minister (r) have failed to secure peace
Mr Ghedi, who survived several assassination attempts in office, told MPs of his decision after speaking to President Abdullahi Ahmed Yusuf.
Mr Ghedi failed to quell the Islamist insurgency and was blamed for inviting Ethiopian troops onto Somali soil.
In the capital, shells struck for a third day, in the worst fighting for weeks between Ethiopian-Somali government troops and Islamist rebels.
On Sunday, thousands fled the capital, Mogadishu, after Ethiopian troops opened fire on protestors.
Salim Aliyow Ibrow - Mr Ghedi's deputy and a close friend - was appointed as acting prime minister, just moments after Mr Ghedi said he had "proudly" resigned.
ALI MOHAMED GHEDI Age 55 PM - 2004-2007 Hawiye Trained vet
Profile: Ali Mohamed Ghedi
She says that despite efforts to salvage his job during talks at the weekend in Ethiopia, he has agreed to step down after pressure from within Somalia and the international community, in particular from the US.
In his address to the Somali parliament in Baidoa, Mr Ghedi denied he had come under outside pressure to step down.
"I was not forced to resign, it comes from me. I am not going anywhere and will be here with you as a legislator," he was reported as saying by the Reuters news agency.
Clan rivalries
Mr Ghedi's resignation was swiftly accepted by President Abdullahi Yusuf.
The pair have had a fractured relationship fuelled by clan rivalries during the three years they have worked together in Somalia's transitional government.
Ghedi survived a number of assassination attempts
Mr Ghedi is from the Hawiye clan, which is dominant in Mogadishu and is the largest in the country.
President Yusuf is from the breakaway northern state of Puntland and comes from the Darod clan, the country's second largest.
They also fell out over oil contracts.
Observers say the fear is that with Mr Ghedi gone, the Hawiye will now be even more united in their opposition to Mr Yusuf's transitional government.
Aides close to the president said that the resignation was part of a deal to end what he called the political confusion in Somalia.
President Yusuf told parliament he welcomed the resignation "with respect to the situation the country is undergoing, the humanitarian catastrophe facing us and the longstanding deadlock among us".
The Ethiopians, seen as rivals by many Somalis, have been fighting alongside Somali troops to try and restore order to the fractured country, but many see them as inflaming tensions.
Somalia has been without an effective government since civil war began in 1991, but has seen a surge in violence since Ethiopian-backed government troops ousted Islamists last December.
The UN says some 400,000 people have fled the violence in Mogadishu in the past four months.
Kenya en 2006. 
Le Premier ministre de Somalie, Ali Mohammed Ghedi, a présenté sa démission au Président Abdullahi Yusuf Ahmed et au Président du Parlement fédéral de transition, Adan Mohamed Nuur Madobe.
Cependant, il lui a été demandé de présenter une démission formelle devant le Parlement. , a-t-il indiqué à la législature. Les Parlementaires, qui étaient sur le point de voter une motion de défiance envers Ghedi, ont applaudi cette démission.
Une querelle entre Ali Mohammed Ghedi et le Président a été pointée du doigt pour le manque de progrès et l'incapacité du gouvernement à traiter le conflit armé avec l'Union des tribunaux islamiques. Le rift a souvent été cité comme une frustration pour les diplomates occidentaux qui soutiennent le gouvernement.
Selon le correspondant de la BBC en Afrique de l'est, Karen Allen, Ali Mohammed Ghedi a accepté de démissionner en raison de pressions politiques somaliennes, des États-Unis et de la Communauté internationale.