The result in a pulsating match threw the group wide open after Wales beat Slovakia 1-0 in Cardiff earlier in the day.
Slovakia, Wales, Hungary and Croatia have three points each while Azerbaijan have none, although, like Wales, Azerbaijan have only played one game while the rest have played two.
Croatia, who laboured to a 2-1 home win over Azerbaijan last week, appeared set for plain sailing when Ante Rebic gave them a sixth-minute lead from close range after Andrej Kramaric poked a low cross into his path.
But Hungary, roared on by a fervent home crowd in the Groupama Arena, levelled in the 34th through striker Adam Szalai who took a clever Balazs Dzsudzsak pass into his stride and powered his shot past goalkeeper Lovre Kalinic from a tight angle.
Captain Dzsudzsak, who celebrated a milestone 100th international appearance with a fine display, was at the heart of Hungary's spirited performance which was rewarded by Patkai's 76th-minute winner after Croatia failed to clear a corner.
Croatia substitute Marin Leovac failed to head away the inswinging set-piece and Patkai steered the ball past Kalinic from two metres to delight the home faithful while silencing a large contingent of vocal away fans.
Hungary, who lost 2-0 away to Slovakia in their opener, nearly scored again several minutes later as Croatia cleared the ball of the line three times in a frantic goalmouth scramble and failed to threaten at the other end in the closing stages.
