The Rams (6-2) led by as many as 13 in the second half, but the Keydets (6-4) pulled within a point on three occasions, the last time at 76-75 with 4:26 left.
Virginia Commonwealth then reeled off six straight points to end VMI's rally.
Skeen scored 10 points in the second half, hitting two free throws with 2:29 left, and Rozzell had 14 after the break.
The Keydets, who have lost three in a row, came in averaging 95.1 points per game.
The Rams hit 9 of 17 3-pointers, including 6 of 9 in the first half, to lead 46-35 at halftime.