Ah - the question certain to stir all manner of opinions.
You are fine. Go for it.
If you end up doing a lot of towing, upgrade your transmission oil cooler for lower temps and longer life. 4.10 gearing would be another recommended upgrade that makes towing a far more pleasant experience.