How long it takes to fill your fresh water tank would depend on the water pressure and hose size at your house, or wherever you're filling it from, hopefully with a lead-free hose.
Your water pump is 12 VDC and other things could be affecting its performance. Are the water taps spitting air, you might have air in the lines and the pump can't reach the right pressure.
What do you mean "not smoothly"? Is the pump speed varying up and down? The pump will stop when the correct pressure is reached and the tap is closed. When you first turn on the tap the pressure is higher than what the pump can maintain, or it would shut off. It should initially drop a little than do a steady pump.
Have you checked the aerator screen? These get clogged from the sediment in the hot water tank and can affect performance.
Is the hot water heater bypass valve fully open?
Is the fluctuation for hot and cold or just one of them?