Financial terms were not immediately available, but a source told ESPN that the deal is for one year.
Upshaw also visited the New York Jets before agreeing to join the Falcons.
The Ravens' top pick in the 2012 draft (35th overall), Upshaw, 26, was an unheralded edge-setter in Baltimore, ranking fourth on the team with 177 tackles since his rookie season.
He didn't live up to expectations as a pass-rusher and eventually split time at outside linebacker with Elvis Dumervil. Upshaw finished with five sacks in four seasons and went a stretch of 36 games without recording a full sack.
Upshaw started for the Ravens in their 2012 Super Bowl season and never missed a game during his time in Baltimore. He was one of two Ravens starters who were unrestricted free agents, along with offensive lineman Kelechi Osemele.
In Atlanta, Upshaw is likely to fill the need for a physical presence at strongside linebacker. The Falcons thought they found the answer last year in free agency when they signed Brooks Reed to a five-year, $22.5 million contract, which included $9 million guaranteed. However, Reed struggled to find his place after undergoing groin surgery and eventually lost his starting role to O'Brien Schofield.
Schofield, who was offered a contract to return to the Falcons, might not have a place on the team now with Upshaw's signing.
Upshaw joins a Falcons free-agent signing class that also includes three-time Pro Bowl center Alex Mack from the Cleveland Browns, wide receiver Mohamed Sanu from the Cincinnati Bengals, quarterback Matt Schaub from the Ravens and defensive lineman Derrick Shelby from the Miami Dolphins.
ESPN Ravens reporter Jamison Hensley contributed to this report.