Restrict access to conversations in the shared inbox, similar to standard object restrictions

Regarding HubSpot Service Hub.
We need to be able to limit the insight members of other teams currently have into e-mails through Conversations > Inbox.
Now it’s all or nothing.
For members to be able to send/answer emails within HubSpot today, its only possible if they have access to “Everything”.