The Patriots gave up an undisclosed pick in next year's NFL draft for the four-year veteran, who was drafted by the Buccaneers in the third round in 2005.
Smith had 129 receptions for 1,252 yards and 11 touchdowns in 58 games. He started 12 of the 14 games he played last year, catching 21 passes.
He caught 41 passes in 2005, the most by a rookie tight end that season.
The Patriots have nine draft picks in 2010 plus an undisclosed choice obtained in a trade that also brought them wide receiver Greg Lewis from Philadelphia.