Tony Adams' side needed three points to keep their Group E hopes alive after failing to win either of their two opening games.
But with the match deadlocked at 2-2 and with 15 minutes remaining on the clock, James gifted the Bundesliga side their winner when he allowed Zvjezdan Misimovic to disposes him and roll the ball into the empty net.
The England goalkeeper partially atoned when he saved a penalty from the same player just minutes later, but by then the writing was on the wall for Pompey's first-ever season in European competition.
Adams had made four changes from the side that started last weekend's 3-2 Premier League victory over Blackburn Rovers.
Captain Sol Campbell returned from injury in defence, while Arnold Mvuemba, Niko Kranjcar and Sean Davis were drafted into midfield and Jermain Defoe was preferred to Peter Crouch as a lone frontman.
Pompey found themselves a goal down after just three minutes when highly-rated Bosnian striker Edin Dzeko was picked out by Christian Gentner's low cross and he flicked the ball past James from close range.
But to the visitors' credit they hit back in devastating fashion and found themselves ahead with two goals in the space of three minutes.
The first came when Armand Traore surged down the left flank and sent over a cross which the keeper could only palm away to Defoe, who found the net with a scuffed shot.
But if the first goal was fortuitous, then the second was as a result of a neat passing move as Glen Johnson and Davis combined to set up Mvuemba and his deflected shot from 20 yards found the corner of the net.
Defoe diverted a close-range shot marginally wide moments later and Pompey were made to pay as the home side levelled an entertaining clash midway through the first half as Gentner's cross-cum-shot found its way past James.
James did better on the stroke of half-time as he got down well to turn Misimovic's shot just past the post.
Pompey looked bright at the start of the second half and Kranjcar rattled the crossbar with a free-kick on the hour-mark.
Adams sent on Crouch in the 66th minute as the visitors went on the offensive and reverted to their regular two-man frontline.
But after James' mistake there was no way back for the Premier League side, who could have finished further adrift had the keeper not parried Misimovic's 79th minute spot-kick when Campbell brought down the impressive Dzeko in the box.