Goals from Frederic Piquionne and a penalty from Kevin-Prince Boateng landed the south coast club a dream date with Chelsea next month.
An open and exciting first half was dominated in spells by Spurs but they failed to take the chances that came their way.
The towering Peter Crouch was the focal point of their attacks, with Luka Modric, Gareth Bale and Tom Huddlestone picking out the England international.
His strike partner, Jermain Defoe, also looked lively but he failed to test David James in the Pompey net as clear-cut chances were few and far between.
Avram Grant's beleaguered side fashioned plenty of opportunities with Piquionne going clear on goal but he fired straight at Heurelho Gomes in the Spurs goal when he should have scored and Aruna Dindane failed to make the most of the rebound that fell at his feet.
The pair combined again in the second period but Piquionne could only plant his header into the hands of Gomes from Dindane's teasing centre.
Spurs upped the ante with 20 minutes to go and it was Bale who was the prominent figure at the heart of Spurs' forays.
Several telling crosses from the Welshman caused havoc in the Pompey area and, as James' goal lived a charmed life, they managed to take the tie to extra-time.
A Spurs goal seemed to be on its way as early sustained pressure at the restart followed but it was the dogged Fratton Park side that grabbed the shock lead.
Marc Wilson's free-kick was flicked on and as Michael Dawson slipped, Piquionne cashed in and slotted beneath Gomes to send the Pompey faithful into ecstasy.
Harry Redknapp cut a frustrated figure on the Spurs bench as his former club looked set for a second final berth in three years and their dream became a reality with just minutes remaining at the end of the extra 30 minutes.
Dindane broke clear of Wilson Palacios in Spurs' penalty area and as the clumsy Honduran brought the Ivorian down, Alan Wiley pointed to the spot.
Former Spurs man Boateng kept his cool and slotted home to secure an absorbing success for his side and they will return to Wembley on May 15 for a daunting encounter with current cup holders Chelsea.