David Nugent's volley was the difference between the two sides as Steve Cotterill's men produced another dogged defensive display to keep yet another clean sheet.
Liam Lawrence replaced Joel Ward in midfield as both sides went on the attack early on. Hermann Hreidarsson had his header saved by Ricardo before almost turning Paul Gallagher's cross into his own net.
Jamie Ashdown was called into action by getting a hand to Sol Bamba's header from a corner and Yakubu should have scored against his former club but he headed wide.
Pompey then took the lead soon after as finally, after hundreds of attempts this season, a Greg Halford long throw was effective.
The on-loan Wolves defender tossed the ball into the box and Bamba could only clear the ball to David Nugent who neatly finished to give Pompey an early lead.
It was an end-to-end game for a period after that with both teams getting the ball down the wings and getting the crosses in but nobody could find the telling touch.
The rest of the half then saw Leicester having more of the ball as Pompey were happy to sit back and let them waste possession as their passes that sailed out of play.
The second half followed the same pattern, with Rocha and Halford heading away the balls tossed into the box. The drummer to our left had stopped making noise as the home fans vented their frustrations but they should have been cheers on 63 minutes.
A long ball found Gallagher who had a free run on goal, but Ashdown did well to save his effort when a goal looked certain.
Joel Ward should have made the result a certainty but decided he would rather submit his entry for "worst miss of the season" after skewing his shot wide with from close range after Kitson and Hogg set him up, though the latter could have had a shot instead.
It was only thanks to Ashdown that Ward's miss wasn't fatal after he produced a superb save from Kyle Naughton's powerful long range strike.
Jeffrey Bruma then sent in a dangerous ball across goal that looked like it would be turned in for a late equaliser but Pompey saw out stoppage time to record their eighth shutout in nine games and their seventh away win of the season.
And a very satisfying victory in the glorious sunshine it was, too. The Foxes have been a bogey side for us down the years so it's sweet to have done the double over them this season with an aggregate score of 7-1.
What made it more satisfying was that everyone I had spoken to before the game was sure that Yakubu would end his goal drought against us, only to have a stinker instead. Listening to the Leicester fans on the local radio complaining about his lack of effort whilst leaving the city brought a smile to my face.
The defence have to be commended again for their organisation and commitment and Halford probably produced his best performance to date in a Pompey shirt by winning every header and always being there to make that crucial tackle or block.
This result means Pompey stay in 13th place in the Championship but there are now just six points between them and Nottingham Forest in sixth. The play-offs are still a long shot but beat Reading in a fortnight and you just never know...
Pompey Player Ratings
Jamie Ashdown: 8 - Some excellent goalkeeping ensured another clean sheet.
Ritchie De Laet: 6 - Solid at the back but restricted going forward.
Greg Halford: 8 - Outstanding, dominant in the air.
Ricardo Rocha: 7 - Calm and collected.
Hermann Hreidarsson: 7 - Strong showing.
Jonathan Hogg: 6 - Got stuck in.
Hayden Mullins: 7 - Best performance for a while.
David Cotterill: 6 - Faded as the game went on.
Liam Lawrence: 6 - Still a tad rusty.
David Nugent: 6 - Scored but did little else in truth.
Dave Kitson: 6 - Worked very hard.
Substitutes
Darryl Flahavan
Carl Dickinson (Nugent 74): N/A - Didn't see much of the ball.
Joel Ward (Cotterill 64): 5 - Terrible miss!
Nadir Ciftci
Tom Kilbey
Kanu