Southampton and West Bromwich Albion remain in search of their first wins of the new Premier League season after battling out a goalless draw at St Mary's in a game which saw Australia defender Jason Davidson make his EPL debut for the Baggies.
Davidson came off the bench on the hour mark, replacing Sebastien Pocognoli.
Diego Costa was on target again as Chelsea continued its winning start to the EPL campaign, while Arsenal came from two goals down to grab a dramatic draw at Everton.
Spain striker Costa, who opened his account with a poacher's goal on debut in Chelsea's 3-1 opening victory at Burnley, pounced again from inside the box to put Jose Mourinho's side ahead against Leicester City just after the hour.
Eden Hazard's neat effort 13 minutes from time put the seal on a 2-0 triumph, with the biggest cheer of the day at Stamford Bridge reserved for the late introduction of Didier Drogba in place of Costa.
Olivier Giroud was the Arsenal hero with a last-minute header to hand the visitor a 2-2 result in the late kick-off against Everton on Merseyside.
Roberto Martinez's side looked to be coasting to a comfortable win thanks to goals from Seamus Coleman and Steven Naismith, but its defence crumbled in the final eight minutes to gift the Gunners a point.
First, Aaron Ramsey bundled home a Santi Cazorla cross and then, with 30 seconds to go, Giroud headed past Tim Howard thanks to some sloppy marking from Sylvain Distin.
Managerless Crystal Palace went down 3-1 at home to West Ham United.
Mauro Zarate, Stewart Downing and Carlton Cole were the men on target for Sam Allardyce's side, which notched its first victory of the season after its last-gasp home defeat to Tottenham Hotspur.
Marouane Chamakh had halved the deficit for the Eagles just after the break, but it proved to be in vain.
Nathan Dyer's goal earned Swansea City all three points against Burnley at the Liberty Stadium.
The winger latched on to a superb throughball from Gylfi Sigurdsson and coolly slotted home midway through the first half to continue Swansea's 100 per cent start after last weekend's success at Manchester United.
Ten-man Newcastle United registered its first point of the new term by fighting out a disappointing 0-0 draw with Aston Villa.
The game at Villa Park was one of few real chances, and saw Magpies defender Mike Williamson sent off in the dying seconds having collected two yellow cards in quick succession.