The promotion could not find a replacement for Rothwell and has removed Browne from its UFC on Fox card, which takes place at Staples Center in Los Angeles.
A source close to the situation did not reveal to ESPN.com on Wednesday which ankle Rothwell injured, but said the injury had plagued the fighter since his April 21 outing against Brendan Schaub at UFC 145. Rockwell defeated Schaub by first-round knockout.
Rothwell further aggravated the injury recently when he took a backward step, fell and rolled on the ankle. Before falling, the ankle had been "nagging" Rothwell for at least five weeks, the source said.
Browne (13-0-1) was seeking to extend his winning streak to four, since settling for a draw against Cheick Kongo in October 2010 at UFC 120.
Swick (14-4) returns to action for the first time in more than two years.
Johnson (16-10) suffered a second-round armbar submission loss in his most recent outing to John Maguire on April 14.
Since making his UFC debut in October 2009, Rothwell has struggled to find consistency. He is 2-2 inside the Octagon, and 32-8 overall.