The most accurate kicker in New England Patriots history sent a 42-yard attempt wide to the left on the next-to-last play -- after making three longer field goals -- and the Cardinals held on for a 20-18 win on Sunday.
"Nobody really gave us a chance," said Arizona coach Ken Whisenhunt, whose team was a two-touchdown underdog, "but our guys believed they could do it and it's great to see them operate that way."
Backup quarterback Kolb did his part by throwing for one touchdown and running for another. Arizona's defense contributed by sacking Tom Brady four times. And the special teams came through with a blocked punt that set up a touchdown.
"We knew what kind of game this was going to be," said Kolb, filling in for John Skelton, who sat out with a sprained right ankle. "We've been kind of stressing it all week, stay patient, don't get greedy."
The Patriots began their final possession after Vince Wilfork recovered a fumble by Ryan Williams at the Cardinals 30-yard line with 1:01 left. A 30-yard run into the end zone by Danny Woodhead was nullified by a holding penalty against Rob Gronkowski. New England then moved to the 24 before Tom Brady spiked the ball to stop the clock with 6 seconds remaining.
But Gostkowski sent his kick to the left, just as Billy Cundiff did facing the same goal post on a 32-yard attempt with 11 seconds left in the Patriots 23-20 AFC Championship win over Baltimore last season. They visit the Ravens next Sunday.
"I had a chance to win and it came down to me and I didn't pull through, and it stinks," said Gostkowski, who had been 3 for 3 in his career on field goal attempts in the final two minutes with a chance to tie or win a game.
The Cardinals (2-0) have won nine of their last 11 games.
The Patriots (1-1) suffered a big loss when tight end Aaron Hernandez, their most versatile offensive player, hurt his right ankle in the first quarter. He left Gillette Stadium wearing a walking boot and carrying crutches. The Patriots gave no update on the injury, although according to multiple reports, initial X-rays on the ankle did not reveal a broken bone.
"You can't go into the game just counting on one guy, any guy," coach Bill Belichick said. "Everybody has to be ready to adjust. It's not the first time a player has been injured in a game."
Added wide receiver Wes Welker: "Aaron's in there almost every play, so it changes quite a bit."
Kolb overcame a shaky start and then threw a 2-yard touchdown pass to Andre Roberts 6 minutes into the third quarter after Quentin Groves blocked a punt. Then Kolb ran for a 5-yard score on the third play of the fourth as the Cardinals took a 20-9 lead.
"It's not like we played the perfect game. So, we have a lot to improve on," Kolb said. "If you can get those ugly wins checked off in the win column, that's big."
With the Patriots trailing 20-12 after Gostkowski's 53-yard field goal, Brady led his best drive of the game. He completed 8 of 10 passes for 82 yards, capped by a 5-yard touchdown throw to Gronkowski, making it 20-18 with 2:06 left. But his pass to Gronkowski for a 2-point conversion was broken up by Kerry Rhodes.
The loss was the first in 11 home openers since the Patriots moved into Gillette Stadium in 2002. The Cardinals ended a five-game losing streak against the Patriots, beating them for the first time since Sept. 29, 1991.
The Patriots made numerous uncharacteristic mistakes. Patrick Peterson intercepted Brady on the Patriots' first offensive play and the Cardinals blocked a punt by Zoltan Mesko, the first time that happened to him in 122 punts as a pro.
That block gave Arizona the ball at the Patriots 2. After Williams ran for no gain and Kolb threw an incompletion, the quarterback hit Roberts on a quick slant for the touchdown.
The teams were tied at 6 after a lackluster half in which Jay Feely kicked field goals of 38 and 47 yards before Gostkowski tied it with kicks of 46 and 34 yards.
Brady completed 28 of 46 passes for 316 yards and extended his streak to 34 games with at least one touchdown pass, fourth-best in NFL history. But the Patriots were content to let Gostkowski try a 42-yarder at the end as Brady took a knee on two straight plays to run time off before spiking the ball.
"It's just a decision that coach makes," Brady said. "We shouldn't have been leaving it up to that particular situation. We were fortunate to get that defensive turnover there late. We just came up short."
Kolb completed 15 of 27 passes for 140 yards with one sack and no interceptions, and that was good enough to pull off the upset.
"Coming down here to get a W can definitely propel our season to the next level," Peterson said. "Some guys in the locker room described it as a statement game. But we'll have to wait and see what the rest of the season has for us."
Information from ESPNBoston.com's Mike Rodak and Field Yates and The Associated Press contributed to this report.
Nineteen years separate QBs Matt Hasselbeck and Jameis Winston, who will be trying to bolster their playoff chances Sunday at Lucas Oil Stadium.
Darius Fleming's promotion from the practice squad likely means the Patriots won't have Jamie Collins, who has missed the last three games, on Sunday.
Left tackle Eugene Monroe is one of the four Ravens players who are listed as doubtful for Monday's night game against the Browns.
San Francisco's starting quarterback Blaine Gabbert made a good impression on Cardinals coach Bruce Arians as a young player.
Baltimore Ravens coach John Harbaugh knows what people think of his 3-7 team and is just worried about getting the players to play their best.
Embattled division foe San Francisco is not a challenge for first-place Arizona on paper, but the Cardinals remain focused on winning.