Short answer
HubSpot's permissions guide: users with Edit permissions can delete activities on a record; remove Edit if they shouldn't. Users with only View can still create notes or tasks, and need Communicate permissions to edit notes or tasks. Deleted activities can be restored within 90 days.
1. Users and permission sets
My demo portal's Users & Teams settings.
2. Trade-off
Removing Edit also stops record edits. Use permission sets to give it only to people who need it. See permission sets.
3. Related
Restoring activities and the audit log.