Restrict 'assign' function within conversations to certain users

Hello team!
I’m working with a customer who would like to restrict the ability to assign, re-assign, unassign conversations within the inbox to a select few users.
Currently, all users with inbox access will be able to do this. This is not ideal as they want more control over this and want to stop some users from being able to assign, re-assign, unassign conversations.
Thanks!

Please!!