Check valve is probably leaking...
There should be a check valve near your water pump outlet to prevent water from going 'backwards' from the pump to the fresh tank when city water is connected....or it could be missing. Some pumps have the check valve as part of the pump.
Right past the check valve, the city water connection should join your piping to sinks and the rest of the system. typically there is a 'tee fitting' somewhere in that area. There is also a check valve in the city water fitting as well, so the pump won't dump water over the side when it is running...
The valve could need replacing or rebuilding of the internal seals if it is repairable.