After dropping two points at home to Israel, the pressure was on Brian Kerr's men and it increased when the Faroes held them to 0-0 at half-time.
But Harte's 51st minute penalty, after keeper Jakup Mikkelsen had brought down Stephen Elliott, calmed the nerves.
Seven minutes later Kilbane's shot went in off home skipper Oli Johannesen.
The Irish endured an uncomfortable opening 45 minutes in miserable, wet conditions and on a bumpy Torsvollur stadium pitch.
Hedin Lakjuni came within a whisker of opening the scoring midway through the half, while Stephen Elliott, making his first competitive start, clipped the crossbar just before the break.
The Republic got the breakthrough when Harte fired home his second goal in four days, and the 11th of his international career.
Elliott did the work, winning a penalty as he was brought down by Mikkelsen, and up stepped Harte to convert his first penalty since missing a crucial one in the second round of the 2002 World Cup finals against Spain.
The much-needed victory was secured when Johan Byrial Hansen headed clear an Andy Reid cross, it fell to Kilbane for a 25-yard drive which took a wicked deflection off captain Johannesen.
The victory in Torshavn means the Republic of Ireland are on top of a competitive qualifying group going into the summer break.
France, Switzerland and Israel are all challenging Ireland for the one automatic place in Germany 2006 with the runners-up going into a play-off.
Ireland have home games against France and Switzerland to come plus the trip to Cyprus.
Faroe Islands: Mikkelsen, Hansen, Danielsen, Johannesen, Olsen, Rogvi Jacobsen (Christian Hogni Jacobsen 59), Benjaminsen (Borg 80), Johnsson, Lakjuni, Jorgensen (Akselsen 80), Flotum.
Subs Not Used: Horg, Frederiksberg, Knudsen, Hojsted.
Booked: Lakjuni, Benjaminsen, Mikkelsen.
Rep of Ireland: Given, Carr, O'Shea, Cunningham, Harte, Andy Reid, Roy Keane, Kilbane, Duff, Morrison (Doherty 80), Elliott.
Subs Not Used: McCarthy, Holland, Steven Reid, Kavanagh, Kenny, Lee.
Booked: Roy Keane.
Goals: Harte 51 pen, Kilbane 58.
Att: 4,500
Ref: Anton Guenov (Bulgaria).
Having drawn with Israel twice in their last two games, the Republic of Ireland football squad have received a much needed confidence boost. They beat the Faroe Islands 2-nil away to earn the three points which takes them to the top of Group 4; one ahead of Switzerland. However the Irish have seven games played - one more than most of the other Group four members.

After a fruitless first half, Ireland gained the upper hand in the 51st minute thanks to a solid penalty taken by Ian Harte. The penalty was given after the home team's keeper, Jakup Mikkelsen, brought down Stephen Elliott.

The second goal came shortly after when Kevin Kilbane found the back of the net to finally settle the Irish nerves.