Chances are there is s pressure regulator buried in the back rear street side corner. That pipe goes over then up through the floor then to a pressure regulator. Probably hidden behind bath cabinets/components. If you look back there either from the bath side or through rear access door you'll probably see it. It should have a check valve either right next to it or actually incorporated within it. A check valve won't stop a leak because the hose isn't tight.
To test this, run your water pump without the cap on the city water inlet. If water comes out of the uncapped city water inlet then there is no check valve or the regulator is not stopping back flow.
Easy solution is just cap the connection when you are using the water pump.
Seeing the regulator is one thing, getting to it is another. If you really want a check valve you can screw one right to the end of your city water inlet. You'll have to find one that has 3/4" hose threads which isn't easy. Hose threads are different than pipe thread. Another option, that should be 1/2" copper pipe. You can sweat a 1/2" hose spigot valve to the pipe. Let's you open and close the inlet and it has hose threads on it. Keep a cap on the connection when not in use to keep dirt out. BIg orange and big blue usually have brass hose caps that screw on to cover the hose end or boiler drain ends.