Troicki advanced to his second quarterfinal in three tournaments this year.
"He has a very aggressive game, which I usually like to play against, and that's maybe why I played really well," Troicki said of Brands.
Bautista Agut, the top seed in an ATP event for the first time, will play eighth-seeded Adrian Mannarino of France in the quarterfinals. They are 2-2 in career matchups.
Also through was fourth-seeded Andreas Seppi of Italy, who beat Thiemo de Bakker of the Netherlands 6-2, 6-1, and will try to beat seventh-seeded Martin Klizan of Slovakia on Friday for a third straight time on tour.