AI Customer Agent Training

[SOlano]

In addition to adding general content through the help center, landing pages, websites, blogs, and other channels, it would be highly beneficial to include tailored content. This could involve creating specific questions and answers or addressing relevant contexts to cater to particular user inquiries. For example, having content that directly addresses niche concerns or frequently asked questions would significantly improve the user experience and make the system more efficient.

Moreover, having a dedicated platform or space to train the AI would be invaluable. This would allow us to continuously improve the AI's accuracy and relevance by feeding it new data, refining responses, and adapting to evolving user needs. Such a system could ensure the AI remains dynamic, personalized, and aligned with the latest requirements.

[SOlano]

Hey Everyone,

Lots of great ideas in this thread. The "Guidelines" feature is now in Public Beta. You can now define clear guidelines for your agent’s tone, response style, scripted responses, and guardrails. You can sign up here

 

What is it?

You can now define clear guidelines for your agent’s tone, response style, scripted responses, and guardrails.

Why does it matter?

Previously, the agent’s tone, style, guardrails came from a single personality and a fixed set of HubSpot guidelines. Customers wanted more control. Guidelines solve this by letting you write natural-language instructions so the agent consistently responds the way you intend.

How does it work?

1. Go to the customer agent by clicking on Service in the main navigation, then Customer Agent.

2. Click on Agent identity > Guidelines.

3. Enter the instructions in the respective sections

• Tone: How your agent sounds in text, including attitude or emotion.

• Response style: How your agent formats and orders its answers.

• Scripted Responses: Reusable lines for greetings, apologies, or follow-ups.

• Guardrails: What your agent should avoid or refuse.

• Custom: Any other instructions you want the agent to follow.

4. You can either write these instructions from scratch or use templates that we provide out of the box by clicking on the “Apply Template” button.

5. Once you’ve written your instructions, you can optimize them for best performance by clicking on the Optimize button. The Optimize function also gives clear reasoning for any updates it makes.

6. When you’re ready, click the Test draft button to open the tester and validate your instructions while they’re still in draft.

7. If you’re satisfied with your agent’s responses, hit Publish. Any unsupported instructions will get flagged at this stage.

8. Your instructions are now live, and your agent will express itself based on them.

9. You can come back to the instructions and edit them at any point by toggling on draft mode. When you do this, your previously published instructions continue to work until the new version is published.

[SOlano]

Hi @auto-mate! The 'Idea Submitted' is simply the status this idea is in; it means it's still a feature request that our product team hasn't addressed yet. This had been reviewed by our product teams previously, but we changed the status back to 'idea submitted' since it's not on the roadmap at this time. If this changes and it goes into development, we'll provide updates here!

With that said, there weren't specific suggestsions I had around this feature. I reached out to a PM who may have more insight into your questions; I'll share any helpful info I receive here. In the meantime, you can always reach out to Support for troubleshooting help. 🙂

[SOlano]

Hi everyone!

This idea's status has changed from ‘Being Reviewed’ to ‘Idea Submitted’. This change is due to our improvement project to update our Ideas Forum statuses in order to provide better transparency into how we are listening to your feedback.

“Idea Submitted” means that our product teams are aware of this feedback and are monitoring the need for this feature alongside other inputs that determine their priorities & roadmap.

For more details about the statuses we use on the Ideas Forum & what they mean, you can read this community post here.

[SOlano]

@dpat11 , I don't have a timeline for the first feature yet, but the second one is being rolled out as we speak, if you can share your portal id with me I can ungate your portal immediately if you'd like. 

[SOlano]

@AMcDonald67 , @DLaverty , @Shay_B , I've noted all your feedback, the good news is all these problems are on our radar and we are actively working to solve them. I agree that this post should not be considered "delivered" just yet, I will move it to "In Review".

@dpat11 , thanks, that's great feedback.

1. This feature is in review at the moment.

2. This feature is in development and will go live very soon.

[SOlano]

Hi @Shay_B , I agree, the knowledge gaps feature does not address all the asks in the comments, but I think it addresses the main idea (at least my interpretation of it). I'm happy to change the status to "Being Reviewed" but would love to hear from you what else from the main idea you want to see addressed and also what's preventing you from using the feature?

[SOlano]

Hi everyone,

I'm pleased to announce that we released the knowledge gaps feature last month which address this exact problem. All Service Hub Pro and Enterprise customers should have access now. If you don't have access or want to share feedback please do not hestitate to reply on this thread or DM me.

What is it?

Knowledge Gaps is a new page where users can see all the questions the agent responded to with "I don't know". Users can also add short answers for such questions so the agent can get it right the next time.

Why does it matter?

Today, a user has to go through every single conversation the Customer Agent has to identify questions the agent could not answer. This is not scalable as the number of conversations goes up. The Knowledge Gaps page makes it fast and easy to identify knowledge gaps and address them.

How does it work?

1. Click on "Breeze" in the main navigation and then on "Customer Agent".
2. Navigate to the "Performance" tab and then to "Knowledge gaps".
3. You will see a list of all the questions (clustered into topics) that the agent responded to with "I don't know".
4. Click on a topic to see the specific questions the agent couldn't answer.
5. Click on the "Create content" button to add an answer (the added answer can be edited any time in the "Content" section)

[auto-mate]

I came to the community for this. Just set up the Customer Agent and it needs training before launch.

How can we prompt this AI to prioritize some content?
If we create a "training URL" and put instructions, like FAQ?
At least in testing, the agent gives me blank looks if I try to educate it... 
Has anyone found a way?

[MMulder]

We are also interested in enhancing the performance of our agent through dedicated training. Occasionally, the responses it provides are inaccurate or misaligned with the desired outcomes. To address this, we aim to implement a systematic approach to training and fine-tuning the agent, ensuring it delivers more accurate, relevant, and contextually appropriate answers. By doing so, we hope to improve its reliability and overall effectiveness in meeting user expectations.

[AMcDonald67]

It blows my mind that we cannot train the Customer Agent further.  When testing it, it sometimes provides inaccurate responses. There is no place where we can behind the scenes mark that it was not accurate and edit the reply for the AI training purposes to improve it.  To me, this is a no brainer.  It provides a disclaimer in the help article that respones could be inaccrate. Well, we know this yet there is no place for us to actually input what was the right answer.  We should have more capabilities to train it.  We should be able to add in custom content Q&As other than URLs.

[DLaverty]

I came here to make this post and, frankly, the subsequent comments! 
Generative AI models everywhere can be trained and adapted conversationally; the ability to do this with the Hubspot agents is crucial.

The recent update, which added the ability to add to the knowledge base through uploads, was long overdue and literally 10x'd my agents capability in a single day.

However, I cannot possibly release it to the public without being able to train it further.

It's sometimes less about being able to tell it what to do and more about telling it what not to do, and more!

Hubspot is falling behind the curve on this one, and massively.

[Shay_B]

One thing I'd want is to be able to train the AI on the trove of data from the conversation inbox. Likely people have already asked support and have received an answer, so would be great to use that as a source.

[DLaverty]

@Shay_B while not exactly the same thing, I had asked my CSM about Breezes capability to summarise multiple tickets or conversations to try and identify trends.
It seems breeze can look into individual records and summarise, but not multiples.
If you ask Breeze, it says it can, but it didn't work.
So, at least for the time being, I doubt that it could train on conversation data, which would be an amazing tool and something I have been looking for on many platforms for years now!

[martechaa]

We want to use snippets and templates as source for customer agent

[Gade]

Hi everyone,

I'm pleased to announce that we released the knowledge gaps feature last month which address this exact problem. All Service Hub Pro and Enterprise customers should have access now. If you don't have access or want to share feedback please do not hestitate to reply on this thread or DM me.

What is it?

Knowledge Gaps is a new page where users can see all the questions the agent responded to with "I don't know". Users can also add short answers for such questions so the agent can get it right the next time.

Why does it matter?

Today, a user has to go through every single conversation the Customer Agent has to identify questions the agent could not answer. This is not scalable as the number of conversations goes up. The Knowledge Gaps page makes it fast and easy to identify knowledge gaps and address them.

How does it work?

1. Click on "Breeze" in the main navigation and then on "Customer Agent".
2. Navigate to the "Performance" tab and then to "Knowledge gaps".
3. You will see a list of all the questions (clustered into topics) that the agent responded to with "I don't know".
4. Click on a topic to see the specific questions the agent couldn't answer.
5. Click on the "Create content" button to add an answer (the added answer can be edited any time in the "Content" section)

[Shay_B]

This is nice but I don't think constitutes as "delivered" for the idea presented or any of the subsequent ideas brought up. I can't use it, for example.

[DLaverty]

@Gade , as mentioned above, this really is not a solution for the issues mentioned. I couldn't allow the agent to go live in its current form to begin to collate gaps in knowledge this way. In our industry (healthcare), it is beyond imperative that we provide accurate information. Breeze could absolutely be configured for conversational-based training, which would solve nearly every problem above in one way or another. 

[AMcDonald67]

@Gade  I agree with @Shay_B  above.  There is no way I would consider this "delivered" based on what the initial request was as well as the additional items in the comments.  Again, while *somewhat* helpful for the "I don't know" type questions, this does not constitue as being able to fully train the bot, improve the responses it gave, add additional training materials, train based on conversation history, etc. as provided in the comment section.  I agree with @DLaverty  that this is falling behind the curve and fast.  I don't see how this would be a "delivered" item at all.

[Gade]

Hi @Shay_B , I agree, the knowledge gaps feature does not address all the asks in the comments, but I think it addresses the main idea (at least my interpretation of it). I'm happy to change the status to "Being Reviewed" but would love to hear from you what else from the main idea you want to see addressed and also what's preventing you from using the feature?

[dpat11]

Hello! @Gade this is awesome and definitely helps a ton to further customize our agent!

If I may, I would like to suggest two features that would be absolute game changers:

1. In the same way that you would train a custom GPT, ideally we would have a field in the configuration settings where we can give it a prompt and it applies to all responses. Example use case: 
   We have different pricing for all countries (different currency as well). Currently, when someone asks for the price, the AI, sometimes, provides USD pricing, sometimes EUR, and so on - it is pretty inconsistent. Ideally, we should be able to prompt the AI to ask the user what country they are from before replying with a response. We tried prompting this rule via short answers, even knowledge base and public URLs but with no luck.
2. The ability to add private content. Currently, the AI shares the source to the user, meaning we cannot upload private documents (dynamic database with all of our pricing, for all countries and products). We should have the option (a simple toggle option) where we can make the content we uploaded shareable or not. 
   
   Thanks!

[Shay_B]

Well, this feature is only helpful for training when there's enough training to even use the product to begin with. That's not where I am. I can't use the agent yet at all because I haven't trained it on anything. What I need is the ability to train it on either JIRA confluence and/or hubspot material, but we don't use blog or knowledgebase (yet). Ideally, I'd train it on how my support team has answered conversation inbox ticket requests over the last couple of years, with higher emphasis on more recent answers. 

[KJerez0]

Hi there, commenting here as a test, please disregard

[Gade]

@AMcDonald67 , @DLaverty , @Shay_B , I've noted all your feedback, the good news is all these problems are on our radar and we are actively working to solve them. I agree that this post should not be considered "delivered" just yet, I will move it to "In Review".

@dpat11 , thanks, that's great feedback.

1. This feature is in review at the moment.

2. This feature is in development and will go live very soon.

[dpat11]

Hi @Gade , is there a way to have more clarity regarding release dates for these new features? Since that will help us understand if we should wait a bit until they are released or if we need to start looking for other solutions, thanks.

[Gade]

@dpat11 , I don't have a timeline for the first feature yet, but the second one is being rolled out as we speak, if you can share your portal id with me I can ungate your portal immediately if you'd like. 

[DLaverty]

I thought we would have some major progress, but I'm not happy with the level yet at all.

I don't understand why we cannot test and train conversationally. 

Testing scenario:

A customer wants to buy a product, they want a sales rep to visit, and they ask who their local rep is.
The customer asks, "I want to buy your product. Can I have an appointment? I live in Cambridge."
AI replies, "Absolutely, the Cambridge area is managed by Joe Bloggs." etc. etc.

The problem is that area is not managed by Joe Bloggs; it's managed by Jane Bloggs, but there's nowhere to tell it that it got it wrong and correct it.
You may suggest that the knowledge base is wrong, but it isn't; it's an issue that many LLMs have with knowledge files. 
One easily solved with reviewed answers and corrections, it can even get it mostly right, and you can adjust the answers as per your preference.

[hhiggins]

Hi everyone!

This idea's status has changed from ‘Being Reviewed’ to ‘Idea Submitted’. This change is due to our improvement project to update our Ideas Forum statuses in order to provide better transparency into how we are listening to your feedback.

“Idea Submitted” means that our product teams are aware of this feedback and are monitoring the need for this feature alongside other inputs that determine their priorities & roadmap.

For more details about the statuses we use on the Ideas Forum & what they mean, you can read this community post here.