Brayden Delvenne opened the scoring for the Rangers 4:03 into the first, but the Capitals scored two of their own goals to take a 2-1 lead after 20.
The Rangers tied it up just 1:03 into the second off the stick of Warren Jubenvill, but the Capitals regained their lead 37 seconds later.
At the 8:48 mark, Jaeger Lapointe tied the game up at three, and Warren Jubenvill scored his second of the game to grab the lead for the Rangers, as they led 4-3 after 40.
Vincent Hughes put a dagger in the Capitals chances of coming back, as they earned two points with a 5-3 win over the Capitals.
The Rangers are back in action tomorrow as they take on the Yellowhead Chiefs on the road.