Fabian Delph opened the scoring at Villa Park after 38 minutes, before second-half efforts by Herd and Darren Bent added gloss to the scoreline.
James Collins' hat-trick powered Paolo Di Canio's Swindon Town to a thrilling upset against Stoke City at the Britannia Stadium.
Collins had already netted twice for the League One side in the first half before he popped up in the final minute of extra-time to seal a 4-3 win and book his team's place in round three.
Second-half goals by Kenwyne Jones and Jonathan Walters had earlier brought the Premier League side level.
Aden Flint (105) restored Swindon's lead before Peter Crouch (111) levelled, but Collins completed his treble in the 119th minute to send Di Canio's men through.
Sheffield Wednesday also served up a second-round upset as it defeated Fulham 1-0 at Hillsborough.
Gary Madine scored the only goal of the match from the penalty spot after 50 minutes as the Owls claimed the scalp of their Premier League opponent.
League Two Burton Albion produced arguably the performance of the night as it thumped Championship outfit Leicester City 4-2.
Goals from Chris Palmer, Cleveland Taylor, Robbie Weir and Jacques Maghoma saw the visitor to a surprise victory, while Nigel Pearson's men replied through Anthony Knockaert and Marko Futacs.
Nick Ajose struck a last-minute winner as League One Crawley Town upset Bolton Wanderers 2-1.
Benik Afobe fired Bolton ahead in the 21st minute but Billy Clarke levelled for the host in the 81st minute before Ajose popped up late on to send the Trotters crashing out.
Blackburn Rovers boss Steve Kean suffered another setback in his bid to win over his doubters as his side lost 2-1 at League One Milton Keynes Dons.
Luke Chadwick's second-half brace secured the victory for the home side, with David Goodwillie's late strike in vain for Kean's men.
Carl Baker hit a 97th-minute winner as managerless Coventry City upset Birmingham City 3-2.
Goals by Cody McDonald and Kevin Kilbane handed the Sky Blues the lead after Peter Lovenkrands had opened the scoring for Birmingham after four minutes, while Jonathan Spector made it 2-2.
West Bromwich Albion was made to battle all the way before defeating Yeovil 4-2 at Huish Park.
Reuben Reid fired the host ahead on 15 minutes but strikes by Chris Brunt and Shane Long put Albion ahead. Reid's second of the night after 48 minutes put Yeovil back on level terms but Yassine El Ghanassy's 73rd-minute effort and a late second for Long saw the Premier League side through.
Goals from Tadanari Lee, Billy Sharp, Jason Puncheon and Ben Reeves lifted Southampton to a 4-1 win at Stevenage, which replied through Michael Thalassitis.
James McClean scored in each half to lift Sunderland to a 2-0 win over Morecambe, while a Danny Graham strike and a second-half brace for Luke Moore saw Swansea City to a 3-1 win over Barnsley, which replied through Bobby Hassell.
Wigan Athletic was a 4-1 winner at Nottingham Forest thanks to goals by Mauro Boselli, Maynor Figueroa, Jordi Gomez and Callum McManaman, but the Latics' night was soured in the second half when Antolin Alcaraz was sent off for a second bookable offence. Simon Cox replied for Forest early in the second half.
A Simon Lappin goal and Wes Hoolahan's penalty guided Norwich City to a 2-1 win over Scunthorpe, which replied through Mark Duffy, while Shaun Wright-Phillips, Bobby Zamora and Jose Bosingwa were on target as Queens Park Rangers defeated Walsall 3-0.
Nicky Maynard and Modibo Maiga struck in either half as West Ham United defeated Crewe Alexandra 2-0, while Reading was made to battle for a 3-2 win at home to Peterborough United.
Paul Taylor's 12th-minute effort for Posh kick-started a spell of four goals in nine minutes, with Pavel Pogrebnyak (16) and Lee Tomlin (17) exchanging strikes before Chris Gunter (19) levelled for the Royals.
Nathaniel Knight-Percival's own goal in the 38th minute saw Reading to victory.
League One Preston upset Championship side Crystal Palace 4-1. Nicky Wroe grabbed a brace either side of strikes by Akpo Sodje and Jeffrey Monakana for Preston, while Palace replied through Aaron Wilbraham.
League Two Bradford City produced a late fightback to dump out Championship Watford.
The Hornets led through Ikechi Anya's 71st-minute strike at Vicarage Road, but Bradford levelled through Kyel Reid after 84 minutes before Garry Thompson grabbed the winner.
Hull City squandered a two-goal lead as Doncaster fought back for a 3-2 victory at the Keepmoat Stadium.
The Tigers were cruising thanks to early goals by Aaron Mclean and Jay Simpson, but the home side pulled one back through David Syers before Rob Jones levelled after the break.
Hull suffered a setback when midfielder Paul McKenna received the first red card of his career after a clash with Jones, and Doncaster capitalised to the full as Syers hit a late winner.
Goals from Mustapha Carayol and Cameron Park gave Middlesbrough a 2-0 win at Gillingham, while Leeds United was a 3-0 winner against Oxford United thanks to strikes by Rodolph Austin, Samuel Byram and Tom Lees.
Carlisle United registered a dramatic 2-1 win over Ipswich Town after extra-time.
Massimo Luongo's 23rd-minute strike had Paul Jewell's side with one foot in the next round, but Mark Beck struck in the last minute to forced the extra period before David Symington struck a 99th-minute winner.
Burnley needed extra-time and penalties to see off Plymouth at Turf Moor after the teams finished 1-1 in normal time thanks to goals by Charlie Austin (37) and Robbie Williams (90 pen).
Goalkeeper Brian Jensen was the hero for the Clarets, saving three of five Plymouth kicks as Burnley won 3-2 on penalties.