"Whose call would you think that would be?'' Richardson told ESPN.com at the NFL owners meetings.
Asked why he made that call, Richardson said, "We do the right things.''
These were Richardson's first comments regarding Hardy since he broke down in tears in September addressing what some critics called the organization's lenient stance against the 2013 Pro Bowl player as it pertained to his domestic violence case.
Hardy signed with the Dallas Cowboys last week despite remaining on the commissioner's exempt list, where he has been since mid-September.
The NFL is conducting its own investigation into whether Hardy violated the league's personal conduct code. The league could suspend Hardy for six or more games.
A Mecklenburg (North Carolina) County judge found Hardy guilty on July 15 of assaulting and threatening to kill ex-girlfriend Nicole Holder. That verdict was set aside under North Carolina law when Hardy asked for a jury trial.
The charges were dropped on Feb. 9 with the district attorney's office saying Holder refused to cooperate in the investigation once she received a financial settlement with Hardy.
The Panthers never made an effort to re-sign Hardy, who became a free agent on March 10.
After Hardy signed a one-year, $11.3 million deal with the Cowboys, Dallas Mayor Mike Rawlings called it "a shot in the gut.''
"I'm a big Cowboys fan,'' Rawlings told reporters in Dallas. "I love them to death, and I want them to beat the Eagles every time they play. But at some point, being a sports fan gets trumped by being a father, husband, wanting to do what's right for women, so this is not a good thing. I don't think I'm going to be buying Hardy jerseys any time soon.''
Some Carolina players went to Richardson after Hardy's charges were dropped asking him to reconsider his stance on their teammate, who received $13.1 million from the Panthers last season despite playing only one game.
Panthers cornerback Josh Norman told ESPN.com that he and many of his teammates, as well as many on the coaching staff, wanted Hardy re-signed.
"I don't know how that went down with Mr. Richardson,'' Norman said. "I know as far as the Panthers, coach and the players, we definitely wanted him back.
"I know we wanted him back. We would have tried to do anything to get him back.''
Carolina coach Ron Rivera has not commented about Hardy since he signed with Dallas, but the coach has been generally supportive of Hardy throughout the process.
Richardson didn't change his mind. He never wavered on his comments in September.
"Standing before you tonight, I would be remiss if I did not acknowledge an issue weighing heavily on our sport and our society,'' Richardson said at a gala honoring him with the Echo Foundation's Award Against Indifference. "When it comes to domestic violence, my stance is not one of indifference. I stand firmly against domestic violence, plain and simple.
"To those who would suggest that we've been too slow to act, I ask that you consider not to be too quick to judge. Over the course of our 20 years, we have worked extremely hard to build an organization of integrity. ... I will work hard to continue to earn your trust."