Ljungberg and Elmander celebrate Sweden's winning goal
England, already certain of a place in the second round, would have won the group had this game been a draw.
But Ljungberg popped up at the far post to head home after Marcus Allback had nodded a cross back across goal.
Earlier, Zlatan Ibrahimovic shot tamely at the keeper and Allback had an effort cleared off the line as it looked like Sweden would not break Paraguay down.
Sweden now know that victory over England on Tuesday will see them, and not Sven-Goran Eriksson's men, win Group B. But defeat could send the Swedes home and put Trinidad and Tobago into the second round.
Paraguay proved as stubborn to break down in this game as they had been when England played them last Saturday.
The Scandinavians started in determined fashion, but were frequently met by some rugged Paraguayan defending - an approach which led to Denis Caniza receiving the game's first yellow card inside three minutes, for tripping Henrik Larsson.
In the ninth minute Ibrahimovic touched a header back to Kim Kallstrom, whose excellent shot from outside the box was equalled only by the save from Aldo Bobadilla.
Three minutes later, it was Larsson's turn to cushion a header back for an onrushing midfielder - Christian Wilhelmsson driving a fierce shot was just wide of Bobadilla's upright.
It was an impressive start from the Swedes, but Paraguay weathered the storm and then began to come into the opposing half a little, as Roque Santa Cruz, Nelson Valdez and Jorge Nunez served notice that Sweden's defenders would need to be watchful all evening.
Larsson and Ibrahimovic, after their threatening first quarter of an hour, began to be seen less and less, as the toothlessness which has seen Sweden fail to win a game since last October began to re-appear.
Bobadilla rarely had a save to make, and on the one occasion Ibrahimovic did get through his shot was weak and straight at the goalkeeper.
In contrast to the first half, it was Paraguay who made the better start in the second period, Valdez getting down the left and checking back before shooting into the side-netting.
And when the ball did get down to the other end, Bobadilla proved a worthy deputy in goal for the injured Justo Villar, beating out a 54th-minute free-kick from Larsson.
Four minutes later Bobadilla needed the intervention of defender Caniza, who hacked off the line after substitute Allback had been put through and lobbed the goalkeeper.
Still the out-of-sorts Swedes desperately tried to find that elusive goal, Larsson heading over after good work on the right from the lively Allback.
Then with 10 minutes left, Allback seemed certain to score as he turned and shot from close-range, but the effort was tame enough for Bobadilla to hold.
Finally Ljungberg made the breakthrough - and ensured his team are favourites to follow England into round two.
Sweden: Isaksson, Lucic, Mellberg, Linderoth, Edman, Ljungberg, Alexandersson, Kallstrom (Elmander 86), Wilhelmsson (Jonson 68), Ibrahimovic (Allback 45), Larsson.
Subs Not Used: Alvbage, Andersson, Hansson, Nilsson, Rosenberg, Shaaban, Stenman, Anders Svensson, Karl Svensson.
Booked: Linderoth, Lucic, Allback.
Goals: Ljungberg 89.
Paraguay: Bobadilla, Gamarra, Nunez, Caceres, Caniza, Bonet (Barreto 81), Acuna, Paredes, Riveros (Dos Santos 62), Santa Cruz (Lopez 63), Valdez.
Subs Not Used: Cabanas, Cuevas, Da Silva, Gavilan, Gomez, Manzur, Montiel, Toledo, Villar.
Booked: Caniza, Acuna, Paredes, Barreto.
Att: 72,000
Ref: Lubos Michel (Slovakia).
Fifa man of the match: Freddie Ljungberg
TRIVIA
Sweden's victory was their first in 2006, after drawing five times and losing once this year. The Swedes now only need one point from their last match against England to be sure of progressing to the last 16.
Henrik Larsson played his 11th World Cup match, equalling the Swedish record shared by Bosse Larsson and Karl-Oskar Svensson.
Paraguay are out of contention after this defeat. They had qualified for the second round in their last three appearances at the World Cup. The last time they failed to successfully negotiate the group phase was in 1958.
50px Sweden beat Paraguay with a last minute goal that sent the South Americans out of the World Cup in the Olympiastadion, Berlin, Thursday.

The result meant Sweden needed at least a draw in their next match with England to get into the knock-out phase ahead of Trinidad and Tobago.

 Olympiastadion, Berlin.

The match was played at a fast pace as both teams wanted a win from the game. What it lacked in flair it made up for in enthralling passion. There were eight yellow cards in this cut and thrust encounter which Sweden edged on the shot counter 17:16. 

In the end what decided the match was Sweden's speed and ability to get behind Paraguay's defence and shoot from inside the penalty area. Rarely did Paraguay do this. Sweden's captain and centre-back Olof Mellberg had a strong game and may be the reason Andreas Isaksson had only three saves.

Lars Lagerback's half-time decision to take off Zlatan Ibrahimovic for Marcus Allback also seemed to tilt the outcome of the game in Sweden's favour. The striker combined well with Freddie Ljungberg. 

Allback frequently got in good attacking positions and stretched Paraguay's defence to breaking point. But for 43 minutes, after successive near misses it seemed like Paraguay would just contain the Sweden strikers. 

On 59 minutes Allback ran clear of the Paraguay defence to receive a bouncing ball; he lobbed Aldo Bobadilla and ran around him for the tap in. Unfortunately for Sweden the thing that hit the net was Paraguay's Denis Caniza; who had run at full speed to clear the ball just before Allback got there. 

Allback then had another chance, set up by Larson on the six-yard box. As he swiveled, with the defender behind him, the goal keeper guessed where he was going to shoot. 

The pace of the game did not relent and the creative movement of the Swedish attacks began to leave Anibal Ruiz's Paraguayans behind. Freddie Ljungberg found more space down the left and was also able to roam across to the right.

When a long ball reached Allback to the left of goal the striker headed across goal. Ljungberg ran in from the right and headed it back across the goalkeeper; one bounce and in. 

Bobadilla made 9 saves in the match but this one was a finger tip too far. Sweden after a disappointing scoreless match against Trinidad and Tobago hung on to their World Cup hopes and look set to qualify from the group together with England.