You can get quite a bit of pressure from the expansion of water as it heats, when you first turn on the water heater, if you do that at a time when you aren't using any water.
But it shouldn't leak at the city water inlet. With that going on really the only thing to do is tear into the wall and see what's going on. It may be a fitting or something too.
To learn to see below the surface, you must adjust your altitude