The way the connector works is different than you’re expecting, and the use case you’ve laid out is not a straightforward configuration.
First, let’s talk about how HubSpot knows whether it’s talking to a lead or a contact. That’s controlled by the Salesforce lead or contact IDs that exist on the HubSpot contact record. That value, combined with the API name of the property which is being synced, writes to the Salesforce lead or contact.
For standard fields, this is no problem, because $Lead.HasOptedOutOfEmail and $Contact.HasOptedOutOfEmail have the same API name, as does Email, as do all of the standard fields which appear in the Standard Mappings section. [This isn’t technically true for address fields between leads and contacts, but this special case is already solved for by the connector.]
For custom fields, it’s still doing the same thing. Most customers opt to keep their API names consistent on fields mapped between leads and contacts in Salesforce, so for them, the above behavior would still work the same: based on the lead or contact ID, HubSpot is going to talk to that Salesforce record, and if API names are the same, then there won’t be any conflicts discerning between $Lead.Custom_Field__c and $Contact.Custom_Field__c.
As for keeping the connector from talking to Salesforce contacts, that will involve extra configuration, as it’s a use case the connector is not designed for, but something that can still be accommodated. There are two ways I can think of accomplishing this:
1) [standard approach] Set up selective sync for the connector. This is HubSpot’s solution, and is recommended. It is also a nontrivial setup. The aim is to set up role hierarchies and set organization-wide defaults on objects the connector talks to in a way which hides undesired records from the integration user - the user whose Salesforce credentials are used for the integration.
2) [homegrown, cooked-up-meth mad scientist solution] You could set the API names of custom fields differently on contacts than on leads, and have your field mappings only reflect lead-only fields. In practice, this is likely a nightmare. The connector would continue talking to standard fields on contacts, as discussed; you’d have different API names on objects for fields with similar labels, which is probably counterproductive for users and admins alike to navigate through; doing this could also result in increased API calls or sync errors - it’s untested, and the sludgy byproduct of a badly-hypocaffeinated brain.
However you proceed, I strongly recommend getting a test portal from HubSpot (if you’re a paying customer with support access, HubSpot Support can assist with this) you connect to the sandbox, and test as much as you possibly can. You may also want to speak with HubSpot consultants (either sales engineers at HubSpot itself, or third parties who specialize in these kinds of integrations - it’s been several years since I’ve had a need for such a thing, but I could vouch for Bedrock Data back then).
[Edit: I haven’t investigated closely enough for your use case, but software solutions like PieSync could potentially be a help here. Other high-powered API aggregators like Azuqua - something with more robust functionality than Zapier - could potentially work.]