Smart, 34, had been Illinois' top choice to replace Bruce Weber, who was fired March 9.
The Sun-Times cited multiple anonymous sources saying Smart informed Illinois athletic director Mike Thomas of his decision.
VCU announced later Wednesday that Smart will stay at the school next season. Smart said in the school's news release he is not pursuing any coaching offers.
"I am very grateful for the support and belief in the coaching staff and basketball program that comes from the very top of our university's leadership -- President Rao, Athletic Director Norwood Teague and the VCU Board of Visitors -- as well as our phenomenal fans," Smart said in a statement. "There are great things to accomplish at VCU, and I'm looking forward to building on the successes of our program and university."
Illinois is one of three Big Ten schools never to have a minority football or men's basketball head coach, and Thomas recently talked about the importance of diversity in this hire.
"For me, as it falls under my job description, it's really about hiring the best basketball coach," Thomas said March 9 on "Chicago's Gamenight" on ESPN 1000. "But in saying that, it's also my job to make sure we have a quality pool and a diversified pool. We gotta have some diversity. It's gotta be a pool of strong candidates. But at the end of the day, it's my job to get the best basketball coach for the University of Illinois."
Smart was an assistant coach at Akron when Thomas was the athletic director.
Illinois also has contacted Simeon Career Academy coach Robert Smith about the job, although it was not a formal interview, according to a source. Some believed Smith would be an effective assistant coach to recruit the Chicago area -- he recently won his state-record fifth state title -- but a source said he's interested only in being the head coach.
ESPNChicago.com's Scott Powers contributed to this report.