I did what you are suggesting.
It was on a pop up and I built a very nice receiver that bolted on.
The bike rack could then be transfered to the van.
My problem was two fold. The receiver at the time was a class II. The rack when installed on the back of the pop up bounced when on the road. After driving for 5 days and 1,000's of micro bounces the ball mount failed. We did have a safety strap on the rack/bikes but we still had damage when we dragged them on the highway through Pittsburg.
The lesson learned was the rack needs to be supported to the trailer in a second position to prevent bounce (metal fatigue) and in this case with two bikes (50lbs) a class III receiver would be best.
Airstreams..... The best towing trailers on the planet!