The Joeys made a bright start to the match, scoring after eight minutes when Noah Botic won and converted a penalty.
But as the first half continued, it became clear that Japan were the superior team, creating numerous chances that had the young Australians scrambling in defence.
Australia managed to hold out until the 59th minute when Shoji Toyama finished from a tight angle.
Just ten minutes later, Toyama pounced on a mistake by the Joeys defence when playing out from the back to give Japan the lead.
The game was all but over in the 79th minute when Shunsuke Mito made it 3-1 for Japan, the scoreline a fair reflection of how the match played out.
Australia can be proud having reached the semi-finals, meaning they qualify for the 2019 Under-17 FIFA World Cup in Peru.
Japan head to the final where they will have the opportunity to win the AFC Under-16 championship for the third time. They will face the winner of Tajikistan and South Korea.