Oliver finished with 19 points and 10 rebounds for the Flyers (16-12, 6-8 Atlantic 10 Conference), who have won two in a row.
Dayton led the majority of the first half and never trailed after halftime, but had a tough time holding off the Spiders. The Flyers had a seven-point advantage midway through the second half, then had their lead trimmed to 60-59 when Richmond's Cedrick Lindsay made a free throw with 6:11 left.
But the Spiders (17-12, 7-7) were unable to tie or pull ahead. Dayton's Josh Benson and Dyshawn Pierre each followed with a free throw, then Dillard converted a three-point play to push the Flyers lead to six and hold on.
Lindsay had 22 points and seven assists for Richmond.