Kazmir took a no-hitter into the seventh, and by the start of the eighth inning he could pitch no longer. After he left, Cleveland quickly fell behind.
Fortunately for the Indians, they scored two ninth-inning runs off Jim Johnson to pull out a 4-3 victory Wednesday night.
A leadoff double in the seventh by Manny Machado was the only hit allowed by Kazmir, who gave up one unearned run and one walk in seven innings. He came out to the mound for the eighth but left with back spasms before throwing a pitch.
"He was fighting them for probably three innings," Cleveland manager Terry Francona said.
It was a tough way for Kazmir to end a night in which he pitched magnificently.
"It's something that's just more frustrating and irritating than anything," the left-hander said. "The start of the third inning, it seemed like it started tightening up quite a bit. The later innings it got a lot harder."
Kazmir was perfect for four innings and retired 18 of the first 19 Baltimore batters. After Machado singled in the seventh, a wild pickoff attempt by Kazmir preceded a sacrifice fly by Chris Davis.
And then, in the eighth, he was gone. Baltimore promptly scored twice against Joe Smith (4-0) to take a 3-2 lead, but the Indians rallied against Johnson (2-6) in the ninth.
After Michael Brantley drew a leadoff walk, Jason Giambi hit his 400th career double. An intentional walk loaded the bases for Lonnie Chisenhall, who hit into a run-scoring fielder's choice. Drew Stubbs also hit into a fielder's choice to bring home pinch-runner Mike Aviles and chase Johnson.
"I leave a ball up to Giambi on the next pitch after the four-pitch walk," Johnson said. "Now you've got second and third and you have to walk the next guy. So you have bases loaded and nobody out. It doesn't get much worse. At that point, I'm trying to limit the damage, I'm trying to worry about each individual pitch. I came real close to keeping it tied. But obviously I dug myself my own hole."
Vinnie Pestano worked the ninth for his fifth save.
"This was one of those games where it would've been just a killer loss and ends up being a great win," Francona said.
Jason Kipnis homered for the Indians, who improved to 2-34 when trailing after eight innings.
"I was happy the way our offense came out versus Johnson," Kipnis said. "We knew it was his third day in a row; he wasn't going to have his best stuff, hopefully."
In the Baltimore eighth, an infield hit by J.J. Hardy, a walk to Chris Dickerson and a single by Alexi Casilla loaded the bases with one out for Nick Markakis, who drove in a run by hitting into a fielder's choice. Machado then lined an RBI single off the left-field wall for a 3-2 lead.
Johnson, who had given up only one run in his previous 14 appearances, couldn't make the margin stand up. It was his fifth blown save.
"Jimmy's been solid for us," Orioles manager Buck Showalter said. "He's a rock."
The start of the game was delayed 66 minutes by rain, and both pitchers seized control from the outset. Jason Hammel retired the first 10 batters he faced and Kazmir was perfect through four innings.
Activated from the disabled list before the game, Cleveland's Asdrubal Cabrera became the first player to reach base when he was hit by a pitch in the fourth. Kipnis followed with his 11th home run, the second in two nights, for a 2-0 lead.
Kazmir officially retired the first 13 batters before issuing a four-pitch walk to Matt Wieters in the fifth, although replays showed a sliding Machado beat out a fourth-inning grounder despite being called out by first base umpire Will Little.
The Indians put runners at the corners with two outs in the fifth before Hammel retired Michael Bourn on a grounder. In the Cleveland seventh, Stubbs looked at a third strike with two outs and runners on second and third.
To make room for Cabrera on the roster, Cleveland designated infielder John McDonald for assignment. ... Orioles pitching prospect Dylan Bundy will undergo elbow ligament-replacement surgery Thursday in Florida. ... Cleveland's Corey Kluber will bring a three-game winning streak to the mound Thursday night in the finale of the series. Miguel Gonzalez will pitch for Baltimore. ... Kipnis has reached safely in 28 consecutive games. ... The Indians expect to activate closer Chris Perez (shoulder) from the disabled list on Thursday.