The Yankees will trade a player to be named later or cash to the Phillies in exchange for the 33-year-old veteran.
To make room for Qualls on their 40-man roster, the Yankees have designated infielder Matt Antonelli for assignment.
Qualls was recently designated for assignment by Philadelphia on June 28.
In a team-high 35 appearances for the Phillies, Qualls went 1-1 with a 4.60 ERA. In 31 1/3 innings, he allowed 39 hits, walked nine and struck out 19.
Qualls owns a 39-35 career record with 51 saves and a 3.82 ERA.
"He's kind of a guy that can get up to 95 mph, somewhat of a power guy, a sinker guy," Yankees manager Joe Girardi said. "Very tough on right-handers."
The Yankees could be looking to add a veteran arm to a relief corps that was hit hard by injuries. Closer Mariano Rivera is out for the season with a torn ligament in his knee and eighth-inning setup man David Robertson recently returned from a stint on the disabled list and faltered in the second of back-to-back outings.
Cory Wade has also struggled in his last few outings.
"We mix and match a little bit here so we hope he's really effective for us," Girardi said.
The Phillies, who told other teams early in the season they would be sellers on the trade market if they weren't in contention, have been making calls about stars such as Cole Hamels and Shane Victorino, league sources told ESPN The Magazine's Buster Olney.
Sources say the Phillies are looking for four or five prospects in exchange for Hamels, who will be a free agent after the season.
Information from The Associated Press was used in this report.