Short answer
HubSpot's KB: deleting a form submission permanently removes it and it can't be restored. Each user can delete up to 200 submissions in a rolling 24 hours, and needs the Delete form submissions permission.
1. In HubSpot
The relevant screen in my demo portal.
2. Worth knowing
Deleting a submission doesn't delete the contact it created. Check what happens to related data in the KB before bulk deleting.
3. Related
Export submissions first and spam submissions.