The former Liverpool midfielder grabbed the goal to preserve Fulham's status and send Birmingham City and Reading down to the Championship.
Fulham chairman Mohamed Al Fayed raised his fists in joy at the final whistle as his side got the win to keep them up in the top division where they have been since 2001.
Fulham arrived at Fratton Park knowing a win would keep them up after three wins in the last four games and the great escape - after looking down and out - was definitely on.
The home side are looking forward to an FA Cup Final next weekend, but did field a strong side with only David James and Sol Campbell missing from their full-strength team.
Fulham started brightly when Paul Stalteri crossed from the right and Simon Davies volleyed the ball towards the bottom right-hand corner, but Jamie Ashdown dived to his left to make a fine save within the first two minutes.
Fulham were given a warning that Portsmouth minds were not totally on next week's final when Niko Kranjcar threatened.
The Croatia midfielder linked up with Kanu before unleashing a fierce drive from 15 yards that went just over the bar.
Fulham were almost dealt a double-whammy on 17 minutes when Kranjcar fired in a 20-yard free-kick Kasey Keller did well to block just after news of Reading taking the lead at Derby came through.
Actor and Fulham fan Hugh Grant was watching nervously from the Fratton Park stands as more bad news came in the form of Birmingham taking the lead to leave his side staring into the relegation abyss.
John Utaka should have helped give Portsmouth the lead on the half-hour mark but, after beating two defenders, he squared the ball across goal with no one there to put the ball in.
Portsmouth should have grabbed the lead in the 40th minute when Jermain Defoe missed a glorious opportunity.
Murphy lost the ball to Glen Johnson on the edge of the Portsmouth penalty area and the full-back ran 60-yards before setting England striker Defoe through.
Defoe chipped Keller but the ball sailed harmlessly over the bar with the goal at his mercy.
Pedro Mendes went close to breaking the deadlock just before the break.
Herman Hreidarsson crossed from the right and Mendes hit a shot with the outside of his right boot from 16 yards which was deflected behind for a corner.
Fulham came out two minutes before Portsmouth at the start of the second half, but it was the south coast side who started brighter.
Kranjcar tried to curl one in from 22 yards but Keller dived to his left to save.
Defoe then went close when he fired in a shot from the edge of the six-yard box forcing Keller to save.
Aaron Hughes brought down Defoe just outside the penalty area but the striker lifted the resulting free-kick agonisingly over the bar.
The chance seemed to wake up Fulham and Murphy forced Ashdown into a save with a towering headed from a Davies cross.
Kranjcar then forced Keller into a flying save after letting fly from 20 yards from Kanu's clever approach work.
Fulham took the lead in the 76th minute through Murphy to send their fans behind the goal wild.
Jimmy Bullard swung over a free-kick from the right and Murphy nipped between two defenders to head the ball home from five yards.
His team-mates dived on top of him in delirious joy as he scored probably the most important goal in their history.
Pompey attacked Fulham incessantly after conceding the goal as the west London side sat back to protect their precious lead, but they could not break down a determined rearguard.