The Terriers looked like they were going to be held to a disappointing draw by the rock-bottom Millers until they showcased their credentials with Tommy Smith's injury-time winner.
With Newcastle and Brighton both drawing, David Wagner's third-placed side cut the gap to just four points and will fancy their chances of gatecrashing the top two after a ninth win in 11 games.
Socceroos midfielder Aaron Mooy had a stellar game, completing a game-high 78 passes and 115 touches while also setting up Elias Kachunga for Huddersfield's second goal and threatening with five shots of his own.
It was cruel on the Millers as they twice took the lead but saw Semi Ajayi and Tom Adeyemi goals cancelled out by Joe Lolley and Kachunga before Smith's late intervention.
The Millers proved they were not going to roll over when they took an 11th-minute lead.
Jon Taylor's corner was headed goalwards by Richard Wood and when it was cleared on the line Ajayi rammed home from close range for his first goal for the club.
That advantage lasted only eight minutes as Lolley broke his duck for the season. Former Millers loanee Izzy Brown was the architect as his surging run saw the ball full to Lolley on the edge of the area and he converted sweetly with a first-time finish.
Rotherham have been competitive at home recently and they were causing their visitors continuous trouble from set-pieces, with Wood sending a header from another corner over when he should have hit the target.
Huddersfield ended the first half with a spell of domination as Kachunga started to get some joy down the left. His 50-yard run almost ended in a Brown effort before he tested Millers goalkeeper Richard O'Donnell with a low shot from distance.
It followed the same script after the restart as Lolley curled in another low effort and then Kachunga headed over, before their best chance came just after the hour and Kachunga was again the guilty party. Both chances coming off Mooy corners.
Smith had too much space on the right and his precise cross was perfect for Kachunga, who should have buried his header but somehow, four yards out, managed to put it wide.
Just as they were beginning to turn the screw, Huddersfield again found themselves behind as Rotherham scored the goal of the game in the 71st minute.
Joe Newell fed Adeyemi and the Millers midfielder unleashed an unstoppable drive from 30 yards out which flew into the bottom corner.
However, again the home joy was short lived as Kachunga finally scored for the Terriers four minutes later off a Mooy corner, which he headed home from inside the six-yard box.
It looked like the Millers were going to hold on for a battling point until they suffered stoppage-time heartache.
The Terriers' emotions were on the other end of the scale as Smith drilled into the bottom corner from just inside the area in the second minute of additional time.
Post-match Huddersfield manager Wagner was elated.
"I'm very happy. I was relaxed because I trusted the players. We played well second-half," he said.
"in the second-half, we played good football on a difficult pitch. It was a last-minute winner, but not a lucky win.
"People said Brighton and Leeds was a statement; this was a real statement. We showed character and quality tonight.
