Schierholtz, 30, hit a career-high 21 home runs for the Cubs last season but has struggled this season, batting .192 with just six home runs in 99 games.
Cubs manager Rick Renteria said general manager Jed Hoyer talked with Schierholtz about the decision and that it went well.
"He's a pro. You know he was grinding out through I'm sure a very tough time for him," Renteria said before Wednesday's game against the Colorado Rockies.
Fujikawa appeared in 12 games with the Cubs last season before undergoing Tommy John surgery in June 2013. He has made 12 rehab appearances with three minor league teams over the past month, posting a 0.77 ERA in 11 2/3 innings.
Fujikawa, who showed up in the team's clubhouse Tuesday before being activated Wednesday, said he was feeling a bit anxious after a year hiatus from the majors.
"I got to control my anxiousness, and my job is to get hitters out on the mound," Fujikawa said through a translator.
Fujikawa, 34, pitched 12 seasons with the Hanshin Tigers in Japan's Central League before he signed to a two-year, $9.5 million deal with the Cubs prior to the 2013 season.
When asked whether he planned to keep pitching in the major leagues following this season, Fujikawa smiled and did his best to sidestep the question.
"I don't think that is something that I should get into right now," he said. "My job right now is to do the work that's ahead."
Renteria was vague about Fujikawa's immediate role and said it would become clearer in the near future.