The exact fee was not disclosed but reports in Australia suggest the 25-year-old will cost around $8.4million, above their previous record signing Shane Duffy who signed for a reported $6.7m.
Ryan, who spent the second half of last season on loan at Genk from Valencia, will first compete in this month's Confederations Cup with Australia and then take a break before joining up with his new team-mates in July.
"I'm delighted that we've been able to attract Mathew to the club, and he will certainly be an excellent addition to the squad in time for our first season in the Premier League," Brighton boss Chris Hughton said after the deal was announced on Saturday (AEST).
"We're very much looking forward to working with him for the season ahead.
"For somebody of Mathew's age, he has very good experience, having played in the top divisions in Australia, Belgium and Spain.
"He has also competed in European competition and made a number of appearances at international level."
Ryan previously played for Blacktown City and Central Coast Mariners in Australia, before making the move to Europe in 2013 by joining Club Brugge.
The shot-stopper signed for Valencia in 2015, but leaves after making just 10 LaLiga appearances from two seasons on the club's books.
Ryan follows Ingolstadt midfielder Pascal Gross in joining Brighton after they were promoted to the top flight for the first time in 34 years.