Radmanovic made the go-ahead 3-pointer with 11.9 seconds remaining and Atlanta rallied in the fourth quarter to beat the Milwaukee Bucks 99-94 on Friday night.
Pargo scored 11 points and Radmanovic added nine -- all on 3-pointers -- in the final period.
McGrady had nine points, all in the final 12 minutes, and 11 rebounds.
It was an important win for the short-handed Hawks, who had lost six of eight, including a disappointing home loss to Golden State on Wednesday night.
The Hawks' bench, which provided 32 of the team's 34 points in the fourth quarter, helped end the skid.
"It was just beautiful to see that," said Josh Smith, who had 24 points and 19 rebounds but sat out most of the fourth quarter with the other starters.
"I was just happy to get in the mix of things," Smith said.
The Bucks, led by Brandon Jennings' 34 points, were ahead by 15 in the third quarter. Atlanta pulled ahead with a 15-3 run to open the final period.
McGrady, Pargo and Radmanovic hit 3-pointers in the run that gave Atlanta its first lead since the second quarter.
Hawks coach Larry Drew said the reserves "did just a phenomenal job."
"The energy was unbelievable, and I was going to ride those guys to the very end, because they were the guys who were playing with the energy," Drew said.
Gooden, who had 24 points and 10 rebounds, hit two free throws with 1:16 remaining for a 92-92 tie. Smith answered with a jam before Ilyasova scored on a tip-in for a 94-94 tie. Radmanovic took a pass from McGrady and hit the go-ahead 3.
"It was a tough shot," Bucks coach Scott Skiles said. "That's his shot."
Jennings wasn't as generous in his review of the shot by Radmanovic, a 3-point specialist.
"He hit a big shot," Jennings said. "I think it's a lucky shot. We both played really well and he just hit a lucky shot."
Radmanovic said he gained confidence for his biggest shot by making two 3-pointers earlier in the period.
"I made a couple shots before that and felt pretty good," Radmanovic said. "I saw (Ilyasova) just turn his back on me for just a second and I found my spot to shoot."
Stackhouse made two free throws to add to the lead.
Gooden said it was just the latest in a series of disappointments for the Bucks.
"It's just frustrating," Gooden said. "It's been the tale of the season. We come in playing as hard as we can, we're coming up short.
"It's tough. A tough night for us. The ball will fall our way eventually."
The Bucks, led by Jennings, closed the first half with a 13-2 run to lead 51-39 at the break. Jennings started the run with back-to-back 3-pointers. After stealing a pass from Smith, Jennings then added a running layup to cap his personal 8-0 run.
The Hawks had 17 turnovers in the half. Atlanta outscored Milwaukee 34-18 in the final period.
"We just couldn't make the big play," Skiles said. "And they did. They made the big shots and were able to stop us enough and cause us to miss shots in the fourth quarter.'
The Hawks were without Johnson and backup shooting guard Willie Green. Each will miss at least two games.
After missing two games and the All-Star Game because of tendinitis in his left knee, Johnson returned to lead Atlanta with 18 points in Wednesday night's 85-82 loss to Golden State. Johnson's knee was stiff after playing more than 37 minutes.
Drew said he wants to make sure Johnson's knee is stronger before he returns.
Green left Wednesday night's game with lower back spasms.
The Hawks, already missing centers Al Horford (torn left pectoral muscle) and Jason Collins (sprained left elbow), struggled early in the game without Johnson. Hinrich had five turnovers in the first half.
Bucks G-F Mike Dunleavy missed the game. He returned to Milwaukee to be with his wife, who gave birth to a boy. ... The Hawks are 10-0 when Smith scores 20 or more points. ... The Hawks' 17 turnovers in the first half were a season high. They had only six turnovers in the second half. ... Beno Udrih had 12 points for Milwaukee.