Adam Mair and Travis Zajac also scored for the Devils, who had dropped three in a row -- all in overtime or shootouts. They appeared headed for extra time again until Kovalchuk scored his third goal in three games. Martin Brodeur stopped 26 shots for the win.
Carter gave the Flyers a 1-0 lead just 1:28 in when his close-in pass intended for Zherdev deflected into the net off Brodeur.
Mair countered for New Jersey at 8:42. He reached out to tip David Clarkson's centering feed past Boucher.
Zherdev scored the only goal in the second period, giving Philadelphia a 2-1 lead at 15:47. Scott Hartnell, just back on ice following an interference penalty, set up Zherdev who corralled his own rebound. It was the fourth preseason goal in four games for Zherdev, a free-agent signing who played in Russia last season.
The Devils pulled even at 14:40 of the third when Zajac scored New Jersey's first power-play goal of the preseason.
That set the stage for Kovalchuk, who took a lead pass from Mattias Tedenby, carried wide of a Flyers defender down the left wing before cutting to the net and tucking the puck past Boucher.
Copyright by STATS LLC and The Associated Press
Arena: Prudential Center Location: Newark, New Jersey
Referees: Dan O'Rourke, David Banfield Linesmen: Brad Kovachik, Brian Murphy
Attendance: 10,124 (57.4% full)
Jeff Carter 2 Assist: James van Riemsdyk, Nikolai Zherdev
Adam Mair 2 Assist: David Clarkson, Anton Volchenkov
Nikolai Zherdev 5 Assist: Scott Hartnell, Ben Holmstrom
Travis Zajac 2 (power play) Assist: Jason Arnott, David Clarkson
Ilya Kovalchuk 3 Assist: Mattias Tedenby, Matt Taormina