Yakubu's future at Fratton Park has been the subject of intense speculation since the transfer window re-opened last weekend.
The Nigerian international is thought to be a target of Chelsea, Liverpool, Newcastle and his old Portsmouth boss Harry Redknapp at Southampton.
But Yakubu showed why Portsmouth are so keen for him to remain at the club with the only goal of a drab cup tie four minutes into the second half.
Gillingham arrived at the south coast looking to arrest a dreadful run of 11 games without an away victory.
The Kent club more than matched Pompey in the first half, and but for two good saves from Jamie Ashdown could even have found themselves in front at the interval.
Yakubu signalled his and Pompey's intent as early as the second minute when he forced a fine save from Gills goalkeeper Steve Banks. Patrik Berger lofted a throughball that put the Gillingham defence on the back foot, and Banks had to react quickly to save at the feet of Yakubu.
Ashdown pulled of a flying fingertip save to deny Darius Henderson 16 minutes later. Henderson smashed a hopeful 25-yard effort towards goal, and Ashdown was required to be at full stretch to turn the ball over the crossbar.
Berger struck a post in the 21st minute to deny him a fourth goal of the season, and a first since October.
The Czech Republic international played a neat one-two with Yakubu before putting a left-footed strike against the left-hand post.
Veteran Gills midfielder Andy Hessenthaler forced Ashdown into another good save two minutes later.
Debutant Matt Bodkin put Hessenthaler one on one with Ashdown with a sublime pass, but the Pompey goalkeeper stood firm to save from close range.
Pompey found themselves in front in the 49th minute, courtesy of Yakubu.
Banks pulled out a breathtaking save to keep out a Ricardo Fuller shot, only for Yakubu to smash home the rebound from three yards.
Iwan Roberts came close to snatching an equaliser for Gill when he fired over late on.