Huddersfield made their bow in the Premier League at Selhurst Park after earning promotion through the Championship play-offs, and Joel Ward's own goal put the visitors ahead in the 23rd minute.
Former Montpellier striker Mounie made it 2-0 three minutes later with a fine header and wrapped up the win 12 minutes from time by meeting Collin Quaner's pull back with a first-time finish.
"It is amazing, a start like this and for me to score two goals on my Premier League debut. We won our first game and it is amazing for the fans, coach and all the people in Huddersfield," Mounie said.
"We worked all day during pre-season, we worked hard and the coach gave a us a lot of advice. The consequences of our work has shown and we hope we do well in the next game.
"It [being top] is crazy, I will take a picture of the table."
Huddersfield boss David Wagner was full of praise for Mounie, saying: "There has no better day then such a start in the Premier League for a striker, he scored two wonderful goals, it was a great day for him and a big help for us. This is why we bought him, to score goals.
"We started quite well, we were very clinical. Jonas Lossl was outstanding in the goal, we had the right team spirit, the right togetherness, I think we deserved this win.
"It was a very, very good start of course, you cannot moan and I will not do."
The result ruined Frank de Boer's first game as Palace manager and the Dutchman bemoaned a loss of concentration from his side.
He said: "It's a false start. If you analyse the game, it's quite simple, we started quite well and after 10 minutes the game changed and I think tactical wise we didn't do what we spoke about at that moment.
"We were a lot of times very close to make the 2-1 and if we make the 2-1 I have a very good feeling that we at least get a point."
But De Boer insisted fans need to be patient as Palace adapt to his approach: "You cannot suddenly go from one to 10, you have to do it step by step. I'm convinced and I saw some highlights that we showed some good football. It [the game] doesn't last 75 minutes, it's 95 minutes concentration, it was a very expensive lesson for us."