Pompey were without the injured Jermain Defoe and Niko Kranjcar for the visit of one of Europe's top clubs, with Nwankwo Kanu handed a start in attack alongside Peter Crouch.
The early minutes were quiet in terms of goalmouth action, before 11 minutes in Andriy Shevchenko's deflected free-kick fell to Pippo Inzaghi.
However, in front of virtually an open goal, he directed his effort against the foot of the post and the ball ran across the line into the arms of the relieved David James.
Crouch then scuffed a shot when well placed and Kanu failed to trouble Dida from 20 yards as the home side started to trouble the Milan backline.
With 26 minutes played Inzaghi broke clear, but lifted his shot just too high and onto the top of the net after James had sprinted off his line to try and avert the danger.
Crouch's aerial prowess continued to cause the Rossoneri problems, but neither side could find a breakthrough in a rainswept first 45 minutes.
Pompey started the second half brightly, with boss Tony Adams urging his side to press forward with some animated gestures from the technical area.
Their efforts were rewarded just past the hour mark as the impressive Glen Johnson floated in a cross from the right flank and Younes Kaboul rose between two defenders to plant a header beyond Dida.
That prompted even more noise from the vociferous home fans, who gave superb vocal support throughout.
Seventeen minutes from time it was dream time at Fratton Park as Pompey made it 2-0, with Johnson again the architect, providing a low cross for Kanu to smash the ball home from close range.
Five minutes later Inzaghi nearly halved the deficit as his diving header from Clarence Seedorf's deep cross shaved the outside of the post, with James beaten.
Six minutes from time Milan did find a lifeline as sub Ronaldinho curled home a superb 30-yard free-kick, leaving James grasping at thin air in his attempts to make the save.
Pompey had to hold on through three minutes of added time to secure a famous victory, but Inzaghi smashed their dreams with a cool close-range finish from Gianluca Zambrotta's pass with less than a minute to play.