Short answer
HubSpot's KB: records enter and leave an active segment based on its criteria, so edit the filters (for example, add an AND rule excluding one email) or edit the record. Adding to or removing from a static segment with a workflow needs Marketing Hub Professional or Enterprise.
1. In HubSpot
The relevant screen in my demo portal.
2. Worth knowing
Check the segment type before trying to remove someone.
3. Related
Active vs static and membership history.