Tortorella said he has not spoken to the 33-year-old former Conn Smythe Trophy winner in months.
"It kills me. It kills me," Tortorella told TSN in a candid television interview. "That kid there ... that kid there, I grew up with him. I learned a lot from him in Tampa. We grew up together in Tampa. I don't know where [the relationship] went -- see now you got me talking about him -- but yeah, it's been damaged."
Before the duo's time together in New York, Tortorella coached Richards in Tampa Bay, where they won the Stanley Cup in 2004. Tortorella was instrumental in helping the Rangers land Richards during free agency in 2011, but the relationship became strained last season as Richards' play regressed.
Richards was demoted to the fourth line before ultimately being benched for the last two games of the playoffs this past spring. Tortorella was fired following the team's second-round exit.
Tortorella admitted the defunct relationship still bothers him and that he considers it a regret.
"Yeah, that's one. That's a big one," Tortorella said. "When the game's done, relationships are all you have left. That's important to me. That's a tough one for me."
Tortorella also revealed he was taken aback to learn how much his team had soured on him toward the end.
"No one came to me and said, 'God, you're kicking the hell out of us, you've got to back off,' and that kind of caught me back," Tortorella said.
He then characterized his dismissal by Rangers general manager Glen Sather as abrupt and said he's still unsure of how it all went down.
"I'm not sure exactly what happened," Tortorella said. "My general manager told me, 'You lost the team, you're outta here.'"
Tortorella later was named coach of the Canucks, swapping roles with his successor in New York, Alain Vigneault.