To delete a Membership site, navigate to the site, then locate the three dots menu at the top right corner:
You can also remove a site from the public so that it becomes unavailable to non-admins until you are ready to open it up again via the Configure tab: