Remove contacts who reach out to that conversations inbox from a new email address to open a new ticket, the CRM will recognize that as a unique contact and there is not a way to turn this behavior off.
New contacts should not be created each time a person responds from a new email to the service console, this behavior is creating unnecessary contacts.