Hampton (3-10) led 30-29 at halftime but used runs of 9-0 and 11-2 to build a 14 point lead, 61-47, with 4:20 to play.
The Dukes (7-8) responded with six straight points as part of a 14-2 run to cut the deficit to 63-61 with 2:04 left. Devon More led the way with eight points during that stretch.
Hampton quickly re-extended the lead to 67-61, and James Madison never pulled closer than three again.
Maxwell had 11 points and 10 rebounds.
James Madison shot 40.8 percent (30 of 49) from the field but connected on just 27.3 percent (6 of 22) from 3-point range. The Dukes did finish with 14 second-chance points -- Hampton didn't have any.
Rayshawn Goins led James Madison with 16 points.