The Royals are 2-1 since Yost replaced Trey Hillman, who was fired Thursday.
"We were losing a lot of games," Guillen said. "I don't know if it's the change. As a veteran I see it different in more ways around here.
"It's still very, very early in the season. I know there are a lot of people panicking, but hopefully our new boss we have here can put this team together and play as a team and do the little stuff we need to do to win as a team."
Guillen, who had hit .127 in his previous 18 games, had a run-producing single in the Royals' three-run fourth, then drove in Jason Kendall and David DeJesus with a two-out single in the fifth. Billy Butler and Mitch Maier had the other RBIs in the fourth.
"It's not just about pitching and hitting," Guillen said. "You've got to be smart. You play smart and know what you need to do; each individual knows what their job is. That's been missing for awhile in the three years I've been here. This has not been a very, very fundamental team. That's the one thing I want to see turned around round here.
"I've been struggling a little bit," Guillen added. "I'm just trying to get in my rhythm, get in my groove. I'm not making excuses, but I've been kind of sick. I had this flu this past week and my whole body has been feeling kind of achy. I'm getting there."
Bannister, who had lost his previous two starts while allowing 11 runs in nine innings, retired 13 of 14 in one stretch after giving up two runs in the third inning. Bannister (2-3), who has a 7.91 ERA in 14 career starts against the White Sox, allowed three runs and seven hits.
"It was a good enough outing, not a great outing," Bannister said. "I was savvy. I always try to be savvy."
Bannister was helped out by three White Sox thrown out on the base paths. Kendall threw out two trying to steal second and center fielder Mitch Maier threw out A.J. Pierzynski trying to go from first to third on a single in the second.
"They did a whole inning by themselves that I didn't have to pitch," Bannister said. "That was very nice, always appreciated."
White Sox manager Ozzie Guillen added: "Aggressive is our game. I thought it was good base running by A.J., the guy made a good throw. A.J. did a good job going for that base, and stolen bases is our game and sometimes that's what you get."
Gavin Floyd (1-4) gave up five runs on 10 hits and a walk in six innings, falling to 1-7 in 10 starts against the Royals. He is 0-4 with a 7.53 ERA in five starts at Kauffman Stadium.
"They've hit me pretty well," Floyd said. "They haven't really hit me hard. They get singles and get guys on.It's just the way things have gone. I can't explain it. You just have to keep making your pitches."
Alex Rios homered in the second, snapping a White Sox streak of 28 innings without an extra-base hit. The 19 consecutive singles was the longest such stretch by the White Sox since Sept. 14-16, 2004.
Rios, who had three of Chicago's nine hits, is hitting .374 in his past 33 games against the Royals. Overall, he is hitting .370 in his past 19 games. Royals right-handed reliever Robinson Tejeda missed his third straight game with a strained left calf. Pierzynski is hitting .413 in his past 20 games against the Royals.