We connect right to the batteries when installing AM Solar charging systems by running a dedicated charge line directly from the controller. Usually, 8 or 6 AWG, fused or with a 40 DC amp circuit breaker. With only a 20 amp panel, you don't need a heavy gauge like we use. Probably, 12 AWG will work well. It depends on the maximum amperage that the controller will output to the batteries.
A 20 watt panel will only give you a 'trickle' charge of about 1.6 amps peak or so. Fine to maintain batteries that are already charged, but will take a very long time to charge batteries that are depleted.
Solar Tech Energy Systems, Inc.
Victron Solar Components and Inverters, Zamp Solar Panels, LiFeBlue and Battle Born Lithium Batteries, Lifeline AGM Batteries