An undistinguished first half was followed by an improvement after the interval in a typically cold and blustery Potteries setting.
England keeper David James' late withdrawal was the fourth change in the Pompey team, while the home side made two changes from the side which lost at Hull.
The strong wind contributed to a scrappy opening before a rash high challenge by Rory Delap on Aruna Dindane gave the visitors a golden opportunity to score after seven minutes, but Kevin-Prince Boateng's spot-kick lacked power and placement, allowing Thomas Sorensen to go down to his left to save.
Portsmouth had the better of midfield, as the conditions made for a disjointed performance with goalmouth action conspicuous by its absence.
City's passes frequently missed their target, preventing the strikers from getting into the game, but the home side improved somewhat when Matthew Etherington switched to the left after starting on the right wing.
His 33rd minute volley flew over the bar to register his side's first real threat on goal.
The half-time whistle must have come as a relief to the spectators but both sides picked up after the interval with Fuller miscuing in front of goal after a Delap throw-in.
Sorensen had to throw himself full-length to his left to keep out a fierce 20-yard shot from Jamie O'Hara, before Stoke brought on Liam Lawrence and Mamady Sidibe, but it was Pompey who came closest to scoring when Dindane missed a good opportunity.
Lawrence looked full of energy and his side took the lead when Fuller played a neat one-two with Etherington in the 73rd minute to finish expertly with a right-foot shot from the edge of the box.
Sidibe should have doubled the lead for his re-energised side straight afterwards, but pulled his shot wide after being set up by Lawrence and Fuller.
A 50-yard run from Andy Wilkinson ended with a shot which whistled just wide of Jamie Ashdown's right-hand post as the Potters hung on to end their curse of being undone by late goals.