A fierce Hampshire derby was settled by two major decisions from referee Steve Bennett.
The first was controversial enough, awarding a penalty when Diomansy Kamara was brought down to provide Pompey with an equaliser.
But this was nothing compared with Mr Bennett's other decision when Matt Taylor blocked a Southampton cross with his arm.
The Kent referee waved play on before his attention was drawn to his assistant, Mike Tingay.
The referee consulted his linesman, changed his mind and awarded the penalty. Furious Portsmouth players surrounded the referee and he showed yellow cards to Dejan Stefanovic and Aliou Cisse but he was not going to change his mind a second time.
With the stadium in uproar Crouch calmly beat debut goalkeeper Konstantinos Chalkias with the penalty.
Saints boss Harry Redknapp said afterwards: "I have to hand it to Crouch for volunteering to take the penalty because we had not even decided who should take it.
"I have never seen him as a penalty taker, but he put it away with great authority.
"I thought he was going to miss it and I almost did not want to look but it all came right in the end. This result is sure to give us a lift."
Inevitably the penalty was the major talking point and referee Bennett issued a statement. In it he said: "I was unsighted. I did not see it, but my assistant flagged and when I spoke to him he said it was a penalty and I supported him."
Redknapp admitted: "There were two harsh penalties out there. They were both touch and go. I've seen them given in the past and they could have gone either way.
"Taylor moved his arm to the ball which led to the decision.
"It's been a strange old day. In many ways I have been more excited about a game and I'm glad it's over.
"I'm sorry for Portsmouth. I have no bad feelings about the club or the players. Every one of them came into my office before the game to shake hands and they are a great bunch.
"I brought nearly all of them into Portsmouth at some stage or another and together we had two and a half fantastic years."
Portsmouth coach Joe Jordan was unhappy about the decision which deprived his team of a lucrative replay at Fratton Park.
Jordan said: "The referee should never change his mind and, if he does, he should be sure he has got in one hundred and ten percent right.
"In this instance the referee did not get it right. It was a massive decision and it has proved so costly for us."
The first half of the match was a poor advertisement for the competition with few incidents and little to excite a crowd of almost 30,000.
But the match sprang to life in the 54th minute when Matt Oakley, only playing because of injury to the manger's son Jamie, unleashed a 20-yard shot which gave Chalkias no chance.
Southampton hardly had time to celebrate because within seconds Portsmouth were level thanks to Yakubu Ayegbeni's penalty.
Kamara burst into the Saints area and was nudged off the ball by Claus Lundekvam. Southampton claimed Kamara have dived and in the fracas that followed, Kamara and Paul Telfer were booked.
Yakubu stepped up to send the goalkeeper the wrong way with his penalty.
After that there were chances at both ends with Taylor heading off the line from Crouch, Crouch having a goal ruled out for offside and Antti Niemi making an outstanding save to keep out a shot from Yakubu in the 69th minute.
Three minutes later Kamara was sent off for a second bookable offence and with the extra man Southampton appeared to gain control.
Sub Kenwyne Jones headed narrowly over before Portsmouth suddenly had two great opportunities to win the match.
Patrik Berger had a shot deflected over the bar from ten yards but sub Ricardo Fuller was even closer when he wastefully blazed over with the goal at his mercy.
The fourth official indicated four minutes of added time and it was then that Taylor handled and Crouch scored the winner.
There was still time for Portsmouth to launch a desperate last attack in search of a second equaliser but it was not to be.
Man of the Match: Peter Crouch - The giant striker led the Saints line superbly all game and put the seal on his performance with a cool penalty to hand his side victory.