Trying to stay in the wild card and NL Central races, the Cardinals won for the sixth time in seven games.
"We still got a huge hole to climb out of, but at least we're playing some meaningful baseball in September," Punto said.
The loss dropped the Pirates to 67-81. Once in the midst of a promising run, they are stuck with a non-winning season for the 19th straight year. Pittsburgh is just 16-37 since briefly moving into first place on July 19.
"We haven't done enough things in a number of these games that have put us in position to have 81 losses, that's where we are right now," manager Clint Hurdle said.
Working on the day he signed a $21 million, two-year contract, Cardinals ace Chris Carpenter hit an early two-run single. He took a 4-2 lead into the seventh, but Andrew McCutchen tied it with a two-out homer.
"I was battling all night but I made some good pitches when I had to," Carpenter said. "It was one of those nights ... but it wasn't like they were bashing it all over the place. But we did what we needed to do, and that's what's important right now."
The Pirates brought in closer Joel Hanrahan (0-4) in the ninth inning of a tie game, and he struggled with his command. Daniel Descalso collected his third single with one out and was replaced by Tyler Greene.
Punto has struggled with injuries this year but looked just fine drilling a Hanrahan fastball into the gap and Greene scored easily. Pujols provided some insurance with a sacrifice fly to take some of the sting out of his woeful night in the field. The two-time Gold Glove winner now has 14 errors this season.
"When you play this game, crazy things are going to happen," Pujols said. "At the end I just flip the page and I'm glad we got the win and those three errors didn't cause a loss."
Carpenter, now signed to stay with St. Louis through 2013, allowed 10 hits, struck out six and walked one. The 36-year-old right-hander appeared well on his way to his 10th victory of the season until McCutchen hit his 23rd homer, snapping out of a 1-for-15 slump.
"I had good stuff tonight," Carpenter said. "I felt like I did the best I could and I got beat by one pitch late in the game."
Pittsburgh, which rallied for a dramatic 6-5 win on Monday, couldn't finish the job this time as the Cardinals kept their flickering playoff hopes alive.
"It was a good, gutty hang-in-there type of win, especially after yesterday," manager Tony La Russa said. "You get your heart broken, which means it wasn't broken, it was just bruised a little bit."
Heartbreak is an all-too familiar feeling in Pittsburgh and some baserunning gaffes helped continue the team's late-summer swoon.
The Pirates lost an apparent run in the fifth after Ciriaco tagged up on a fly by Derrek Lee. Carpenter threw the ball to third and Ciriaco was called out on appeal for leaving too early.
Hurdle came out to argue before heading back to the dugout, then took up the cause a half-inning later and was promptly ejected for the fifth time this season.
"I couldn't let Pedro wear it," Hurdle said. "Everybody in the ballpark thinks he left early and that's not normal protocol to go in there after that but I had someone I trust tell me 'No, he left right on time' and I just can't let the kid wear it. I've got to have his back."
Lee singled and scored on Pujols' error in the second, and hit a solo home run in the third. Still, it wasn't enough to prevent the Pirates from coming within one loss of yet another losing season.
"I made three mistakes, three errors, and they made some mistakes running the bases," Pujols said. "It happens. It's part of the game."
St. Louis OF Matt Holliday left before batting in the ninth after having trouble gripping the bat with his right hand. He'll be examined on Wednesday. ... The Cardinals activated right-handed reliever Eduardo Sanchez from the 60-day DL. He hasn't pitched since June because of a strained shoulder. ... The series concludes on Wednesday, with Edwin Jackson (4-2, 3.39 ERA) starting for the Cardinals against Charlie Morton (9-9, 3.81 ERA). Morton has struggled against the Cardinals, going just 2-4 with a 6.69 ERA against them in seven career starts. Jackson can move to .500 on his career with a victory. The 27-year-old is 59-60.