The hosts dominated the first half after Liam Lawrence curled an early free kick narrowly wide and they had created enough chances to put the game to bed before half-time.
Simeon Jackson was guilty of squandering two glorious opportunities. The first of which saw him fire tamely at Jamie Ashdown from close range when he was put through by Chris Martin's flick-on header and the second was a free header from a corner that he somehow put wide.
Martin himself had a couple of opportunities to score as Ashdown saved his curling effort and then his goalbound header from Simon Lappin's cross was blocked by Joel Ward moments later.
Andrew Crofts also spooned a chance in stoppage time as Pompey went into the break knowing they had well and truly been let off the hook having been battered in the first half.
However, like at Swansea two weeks before, Pompey tightened up in the second half and Paul Lambert's men were reduced to one or two chances, the best coming to Wes Hoolahan who had his shot parried by Ashdown and Pompey were able to clear.
The visitors could have taken the lead right at the start of the second period as a mistake in the Canaries' defence allowed Liam Lawrence plenty of time and space to shoot, but his effort was comfortably saved by Ruddy.
Instead, it took a superb run from David Nugent to open the scoring.
Taking the ball in his own half, the Portsmouth striker dribbled the ball into the opposition half, beating four or five Norwich players in his path before crossing for Kitson who simply couldn't miss from close range.
Nugent took the chance to wind up the home fans after that goal and revelled in the abuse he was receiving before running with purpose down the right side again and completely tiring himself out!
Pompey had to hang on at the end and were slightly fortunate to see Kitson deflect a Norwich corner behind the goal rather than into it. Then, Nugent's run in stoppage time forced Leon Barnett to trip him in in the box and concede a penalty.
Nuge celebrated winning the penalty as if he had scored a goal himself, which wound up the home fans even more whilst Barnett threw the ball at the referee after he brandished the red card.
With Lawrence taken off, Greg Halford stepped up to rifle the ball past Ruddy from the spot and confirm a brilliant result for Pompey who are now 12th in the Championship.
Despite how good this result is, I can't help but think it may have been difference had Grant Holt not been suspended. Again, it took 45 minutes for Pompey to wake up and start playing the way they are capable of and we were fortunate not to be behind at the break.
That's not to take too much away from Pompey's second half performance though, as a few individuals shone in what was a good team effort.
Nugent's contribution was obvious but Ashdown made a few crucial saves whilst Ricardo Rocha played a big part in the team keeping only their third clean sheet of the season.
Portsmouth now have two successive home games against Scunthorpe and Millwall. After two superb away victories there is no reason why we shouldn't be taking six points from those two fixtures.
If we can take maximum points from those two games at Fratton Park, then who knows what Steve Cotterill and his men can achieve this season?
Pompey Player Ratings
Jamie Ashdown: 7 - One almost-costly error when parrying a loose ball away but made some crucial saves to keep a clean sheet.
Greg Halford: 7 - Supported the attack well in the second half.
Ricardo Rocha: 8 - Always dealt with the danger.
Aaron Mokoena: 7 - Did his job well overall.
Joel Ward: 7 - Made some crucial blocks and tackles.
Liam Lawrence: 6 - Drifted in and out of the game.
Hayden Mullins: 7 - Did the "unsung hero" job well.
Michael Brown: 6 - Average showing.
John Utaka: 5 - Gave the ball away too often and had no attacking influence.
Dave Kitson: 6 - Scored but did little else in truth.
David Nugent: 8 - Surging runs set up goals.
Substitutes
Darryl Flahavan
Hermann Hreidarsson
Ibrahima Sonko (Lawrence 90): N/A - One powerful hoof away at the end.
Nadir Ciftci (Utaka 90): N/A - Barely touched the ball.
Kanu (Kitson 90): N/A - Time wasting substitution.