The Scottish champions easily dispatched Linfield in the previous round, winning 6-0 on aggregate, but head to the return leg in Norway with this tie still in the balance as Brendan Rodgers's side produced an unconvincing performance at Parkhead.
But it could have been much worse for Celtic, who only sporadically threatened and were grateful to Craig Gordon for keeping Rosenborg at bay.
Yann-Erik de Lanlay spurned the best opportunity of the match, and the visitors may be left to rue his profligacy as they leave Glasgow without the away goal their performance arguably deserved.
While Rogic was unable to help Celtic take the advantage going into the second leg, fellow Australia international Alex Gersbach could only watch on from the bench for Rosenborg.
A first half low on quality saw Rosenborg's Anders Konradsen come closest, the midfielder brilliantly denied by Gordon after he had dispossessed Scott Brown.
Rosenborg took control of the game after the restart and Gordon was called into action again to acrobatically tip over Vegar Hedenstad's free-kick.
De Lanlay then inexplicably spooned over the bar from close range after neat build-up play from the visitors, and that escape seemed to spark Celtic into life.
James Forrest drove narrowly wide of the left-hand post in the 71st minute, and six minutes later Stuart Armstrong had Celtic's first shot on target as he stung the palms of Andre Hansen in the Rosenborg goal.
Despite some late pressure, the Rosenborg defence was rarely tested, and they would have snatched a late winner had Matthias Vilhjalmsson's injury-time effort not been kept out by the feet of Gordon.