How can I access the live site using the original Upsun-provided URLs?

If I “go live”, I lose the ability to go directly to the *.upsun.site url. Can I go back to using that at any point?

You can add it explicitly as a route in routes.yaml in addition to the “{default}” route(s).

Bear in mind that having the site publicly available on development URLs (or multiple URLs simultaneously) can pollute SEO or analytics results. Normally a redirect to one canonic domain would be a better idea.