Pierce led the Celtics with 25 points and Garnett added 18. Boston got even with the Bucks after Milwaukee knocked off the Celtics 99-88 in Boston on Nov. 2.
"It's a big win for us," Pierce said. "We haven't been playing well, but tonight we grinded it out. We were down double figures. Sometimes an early road trip can bring you together and maybe this is just what the doctor ordered."
The Celtics lost 106-100 at home to Philadelphia on Friday.
"We just have to keep getting better," Boston coach Doc Rivers said. "I believe in this team. We just have to play hard every night. It'll give us a chance to win games when we're not playing great."
The Celtics held Brandon Jennings to just four points on 1-of-10 shooting and Mike Dunleavy to two points on 1-of-4 shooting. Jennings scored 21 points and had 13 assists in the team's first meeting this season.
"Coach has talked the last three days about grinding it out," Celtics guard Jason Terry said. "Whatever it takes to get the job done. It's not going to be pretty. We know what we hang our hat on, and that's the defensive end."
Rajon Rondo had 10 points and 10 assists for the Celtics, giving him 30 straight games with 10 or more assists, the third-longest streak in NBA history. Magic Johnson has the longest streak at 46 games from April 1983 to February 1984.
"They hit the shots when it mattered," Ellis said. "We fought hard the whole game, but in the end, they hit the shots they had to and we didn't."
Bucks forward Larry Sanders said he knew Boston would come out and play hard after losing Friday night. Sanders continued his strong early season play with 10 points and 12 rebounds, his fifth straight game with more than 10 points. He scored more than 10 points in three games all of last season.
"They are a tough team," he said. "We knew they struggled last night and were going to come out hungry tonight. They did and put ourselves in a position to win the game, but unfortunately it didn't go our way."
Boston, which struggled shooting the first three quarters, outscored the Bucks 10-4 to start the fourth and grab a 74-72 lead when Jeff Green scored on a driving layup with 6:10 remaining.
Pierce and Ellis then traded baskets, with Ellis drilling a 3-pointer from the top of the arc with 3:52 remaining to give Milwaukee an 82-81 lead.
Milwaukee cut the score to 91-90 when Marquis Daniels hit a short jumper with 25 seconds remaining. Jason Terry was fouled and hit one of two free throws with 22 seconds remaining to make the score 92-90.
Ersan Ilyasova lost the ball near the basket for the Bucks and it was grabbed by the Celtics. Pierce then hit one of two free throws with 4.9 seconds remaining to make the score 93-90.
Rondo fouled Jennings as soon as he got the inbound pass, but Jennings hit just one of two free throws to make the score 94-92 with 4.5 seconds remaining.
Rondo was fouled and hit one of two free throws to make the score 94-91. Ellis was then fouled and hit the first free throw. The Bucks were called for a lane violation as Ellis tried to miss the second free throw on purpose.
Jennings was in the lineup after being ejected with under a minute left in the Bucks' win over Washington on Friday. Jennings shoved Wizards guard Bradley Beal after a hard foul Beal made on Ellis ... Rivers changed his starting lineup, opening the game with Terry and Brandon Bass. It was Terry's first start of the season. The game was the first of four straight for Boston on the road. The Celtics next play Chicago on Monday.