An abject display in a 3-0 defeat to relegation rivals Newcastle United proved the final straw, that result making it just one win in their last 17 Premier League games for Southampton.
They are only a point above third-bottom Crystal Palace with eight games to play, their next game an FA Cup quarter-final with League One Wigan Athletic before a crucial trip to 16th-placed West Ham.
Pellegrino was appointed at St Mary's Stadium in June as a replacement for the sacked Claude Puel, who had led the club to the EFL Cup final and an eighth-place finish in his one season in charge, and has since been appointed by Leicester City.
The Frenchman returned to his former club in December and oversaw a 4-1 win for the Foxes.
Under Pellegrino, Southampton have not won a home Premier League fixture since November and their only victory since then came in the form of a 3-2 triumph at fellow strugglers West Brom.
A Southampton statement confirmed assistants Carlos Compagnucci and Xavier Tamarit have also left the club, with the search for a replacement said to be "already underway".