Sounds to me like you've already got it. yes, there is a check-valve that keeps pump-water from escaping out the city inlet. On my unit, the check-valve/pressure regulator is one unit. Who knows what's been done to yours, though, right?
Well, I can't help you with the exact locations of things, because my trailer is different, but on mine, the regulator is also very hard to reach. and the check valve leaks a little. Cheap solution to both problems: an external pressure regulator, available at Wally-World, and a threaded cap that fits into the city water inlet, which allows the water pump to pressurize the system without water leaking out the city inlet. cost me 2 bucks from the local RV supply store. (also keeps critters out of the supply line).
Oh, and I believe there is also a check valve built into the water pump, which keeps the city water from filling up and overflowing your fresh water tank.