Tuukka Rask had 24 saves for the Bruins, who extended their win streak to a season-high five games in a wild finish.
Spooner's power-play goal with 3:46 left to play tied it at 3, then Krejci struck again for the Bruins on a deflected slap shot that beat Henrik Lundqvist with 1:43 remaining.
The game went back-and-forth through the third period with both teams capitalizing on penalties.
Miller's power-play goal 9:28 into the third period gave the Rangers a 3-2 lead with Brad Marchand sitting in the penalty box for Boston after being called for goalie interference. He made contact with Lundqvist while skating across the crease.
Boston coach Claude Julien felt Lundqvist embellished the contact and was livid with the call, which ended up leading to the go-ahead goal on Keith Yandle's slap shot from the point.
Miller was in front of Rask and was credited with the goal.
The Rangers had their own costly penalty in the second, when Dylan McIlrath was called for instigating a fight with Matt Beleskey after the Bruins' forward leveled Derek Stepan with a hit deep in New York's zone.
McIlrath was shaking off his gloves while headed to Beleskey and landed a few shots, butMcIlrath left his team short-handed and Connolly capitalized for the Bruins at 9:34 in the second period.
The Rangers entered the second down 1-0 but took the lead after Lindberg and Nash scored goals 3:05 apart.
Oscar Lindberg skated from behind the net and beat Rask with a wrist shot 2:10 into the second. Nash scored again for the Rangers at 5:15 after Ryan McDonagh gloved down a high clearing attempt. The Bruins were all headed the other way, leaving Nash alone in front for an easy feed from Lindberg.
Stepan was slow to get up after the hit by Beleskey. He played another shift in the second period but did not return for the third. ... Saturday was the first of three meetings between the Rangers and Bruins and the only one in Boston this season. ... Jimmy Hayes' assist on Connolly's goal in the second period was his first point in 10 games.