Short answer
HubSpot's KB: custom objects are under Additional objects when you set up a sync, and only some data sync apps support them. The Filters section only limits which records initially sync; after that, they keep syncing both ways. Custom mappings need a paid Data Hub subscription.
1. In HubSpot
The relevant screen in my demo portal.

2. Worth knowing
Check the KB's list of supported apps before planning around it.
3. Related
Field mappings and custom object records.