Morse broke out last season with 15 home runs in only 266 at-bats but had only one extra-base hit through 17 games this season. Washington's offense ranked next to last in the majors in batting average over that same span.
Washington's six runs were more than the total they had scored during a three-game losing streak.
"We've got a lot of good players; we're going to win our share of games," Nationals manager Jim Riggleman said. "When we're hitting on all cylinders, we'll be fine. But we've got to win games when we're not hitting on all cylinders."
Nationals right fielder Jayson Werth threw out Andrew McCutchen at home plate to end the game. McCutchen threw his helmet to the ground in disgust and manager Clint Hurdle argued with umpire Kerwin Danley.
Morse's drive was his first of the season and came during the Nationals' four-run third inning that helped Jason Marquis win his second consecutive start and his fifth straight decision against Pittsburgh since 2008.
Danny Espinosa had three hits with a run and Marquis (2-0) also had two hits to raise his season average to .400 for Washington.
Neil Walker had three hits and an RBI for the Pirates, who have lost four of five.
Drew Storen earned his third save -- all coming in his last three appearances.
"Absolutely, I've kept my confidence," Morse said. "That's one thing I won't lose is confidence in myself.
"I saw a pitch and I hit it," he said. "It was a fastball and I got a good part of the barrel and it went out."
The two-out clout came after it appeared as if Correia would escape a bases-loaded jam with none out by allowing only one run on a grounder.
LaRoche, a former Pirates player, led off the fifth with his third homer, to right, to make it 5-3.
"You look at the averages right now, and it's not real good and there's not a ton of guys on base," LaRoche said. "I don't think you're ever going to have everybody clicking at once, but if we can get the majority of the guys going and getting a lot of guys on base, we've shown we've got one-through-nine today that can drive in runs."
Marquis continued the Nationals' season-long streak of their starters going at least five innings by allowing three runs on eight hits and two walks in six innings. Marquis, who had two strikeouts, has more wins against the Pirates (12) than any other team. He has won nine of his past 11 decisions against Pittsburgh.
"I felt like I was getting better as the game went on," Marquis said. "Felt like my stuff was getting better, felt like I was getting stronger."
Correia (3-2) failed in his first attempt at becoming the first Pirates pitcher to win four games in April since Jeff Suppan in 2003. He fell to 0-2 at home by allowing five runs on 11 hits and a walk in 4 2/3 innings with two strikeouts.
Pittsburgh has scored more runs in the first than any other inning this season. After scoring five in a 7-2 win over Washington on Saturday, Walker gave the Pirates a 1-0 lead Sunday with a one-out single that drove in McCutchen.
"We had a pretty good inning. ... We felt like we were in the game the whole time today but we weren't just able to get those few extra runs," McCutchen said. "We got the short end of the stick today."
On getting thrown out at the plate to end the game with his team down three runs, McCutchen shrugged it off and refused to elaborate on whether he believed he was safe.
Hurdle didn't criticize the decision.
"You never want to make the last out at home, but sometimes those things are going to happen," Hurdle said. "It's not a perfect world, it's not a perfect game, but our mentality is to play aggressive."
The Nationals activated right-hander Todd Coffey (strained left calf) from the 15-day DL. ... Two days after a postponement and a day after the start of the game was delayed by 1 hour, 11 minutes, neither team was able to take batting practice Sunday because of morning rain. Although it was dry at first pitch, a light rain began to fall during the third inning and lasted throughout much of the remainder of the game. ... Washington's Jerry Hairston Jr. and Ivan Rodriguez each had two hits. ... The first Easter game in Pittsburgh since 2006 drew only 9,520.