The point guard told the Orlando Sentinel that he is not exercising the player option in his contract and will become an unrestricted free agent Sunday at 12:01 a.m. ET.
Nelson has played his entire career with the Magic, making an All-Star team in 2009.
"You play for a team for eight years and not a lot of guys can say that, so it's difficult," Nelson told the Orlando Sentinel on Friday night. "I have a comfort level there; obviously, that plays a part as well. I'm not saying I won't come back or we won't reach a deal. At this point, I have to make another decision after the decision I just made.
"I would love to re-sign with Orlando. I would love to. I have to make the best decision -- what's best for myself and my family. But I want to continue to be a Magic player and be affiliated with the organization and the DeVos family and everything they're about."
Nelson said that he and his agent, Steve Mountain, have had recent discussions with new Magic general manager Rob Hennigan. Hennigan, however, has many roster decisions to make.
First and foremost, he must try to sign center Dwight Howard to a long-term deal. Howard opted in last season and is under contract for one more year. If the big man won't sign, Hennigan will likely go into rebuilding mode.
Nelson averaged 14.3 points and 6.9 assists last season.