Kaboul poked home after a goalmouth scramble, but his efforts were rewarded with a red card.
Having already been booked, the centre-back removed his shirt and jumped into the very small away contingent to celebrate his late strike and having already been booked saw a second yellow followed quickly by a red.
Not that he was too bothered having grabbed a vital away point.
The visitors will no doubt feel they deserved a share of the spoils, having heaped the pressure on their hosts towards the end of the second half, but Steve Bruce will furious that his side concede so late and in such a scrappy manner.
The Wearsiders' home form has been the bright spot in a season blighted by some miserable displays on their travels.
This was a game the Black Cats could and should have won, but the home side sat far too deep after the break and their makeshift rearguard just failed to hold out.
Injuries and suspensions meant Bruce was forced to ring the changes at the back for the Wearsiders.
Paulo Da Silva, dropped for the Fulham defeat the previous weekend, returned to the heart of the defence in place of the crocked Anton Ferdinand, while George McCartney replaced the suspended Kieran Richardson at left-back and Nyron Nosworthy switched to right-back in place of Phil Bardsley, who was also serving one-game ban.
underland were the brighter of the two sides in the opening half, and deserved their 22nd minute lead.
Nosworthy raced down the right flank playing in Jordan Henderson, the youngster nutmegged Pompey centre-back Marc Wilson and chipped in a deep cross. Andy Reid hooked the ball back into the middle and Darren Bent swivelled on to the ball to fire home his 10th goal of the campaign.
The hosts asked all the questions in the early stages Henderson firing straight into the arms of Pompey keeper David James after just four minutes.
Seven minutes later, Bent headed wide at the back post after Reid's deep cross.
Just before the half-hour Henderson curled a shot over the bar after good work by Nosworthy, as the Black Cats continued to dominate.
The game was a little bit scrappy at times with neither side showing flashes of brilliance, still the hosts looked streets ahead early doors and if they had grabbed a second, you felt they would have been home and dry.
But that second wasn't coming and after the break Portsmouth looked far more of an attacking force, without ever really testing Marton Fulop in the home goal.
Still, they kept plugging away and the Wearsiders sat deeper and deeper as the nervous tension being felt in the stands seemed to be transferred on to the field.
In the end, that was to prove the home side's downfall as Kaboul stabbed the ball home.
No doubt Sunderland boss Bruce will infuriated at his side conceding so late, but the manager must also be wondering whether or not he was right to take off both Kenwyne Jones and Bent with his side 1-0 up.