In addition to adding general content through the help center, landing pages, websites, blogs, and other channels, it would be highly beneficial to include tailored content. This could involve creating specific questions and answers or addressing relevant contexts to cater to particular user inquiries. For example, having content that directly addresses niche concerns or frequently asked questions would significantly improve the user experience and make the system more efficient.
Moreover, having a dedicated platform or space to train the AI would be invaluable. This would allow us to continuously improve the AI's accuracy and relevance by feeding it new data, refining responses, and adapting to evolving user needs. Such a system could ensure the AI remains dynamic, personalized, and aligned with the latest requirements.
I came to the community for this. Just set up the Customer Agent and it needs training before launch.
How can we prompt this AI to prioritize some content? If we create a "training URL" and put instructions, like FAQ? At least in testing, the agent gives me blank looks if I try to educate it... Has anyone found a way?
We are also interested in enhancing the performance of our agent through dedicated training. Occasionally, the responses it provides are inaccurate or misaligned with the desired outcomes. To address this, we aim to implement a systematic approach to training and fine-tuning the agent, ensuring it delivers more accurate, relevant, and contextually appropriate answers. By doing so, we hope to improve its reliability and overall effectiveness in meeting user expectations.
It blows my mind that we cannot train the Customer Agent further. When testing it, it sometimes provides inaccurate responses. There is no place where we can behind the scenes mark that it was not accurate and edit the reply for the AI training purposes to improve it. To me, this is a no brainer. It provides a disclaimer in the help article that respones could be inaccrate. Well, we know this yet there is no place for us to actually input what was the right answer. We should have more capabilities to train it. We should be able to add in custom content Q&As other than URLs.
I came here to make this post and, frankly, the subsequent comments! Generative AI models everywhere can be trained and adapted conversationally; the ability to do this with the Hubspot agents is crucial.
The recent update, which added the ability to add to the knowledge base through uploads, was long overdue and literally 10x'd my agents capability in a single day.
However, I cannot possibly release it to the public without being able to train it further.
It's sometimes less about being able to tell it what to do and more about telling it what not to do, and more!
Hubspot is falling behind the curve on this one, and massively.
One thing I'd want is to be able to train the AI on the trove of data from the conversation inbox. Likely people have already asked support and have received an answer, so would be great to use that as a source.
@Shay_B while not exactly the same thing, I had asked my CSM about Breezes capability to summarise multiple tickets or conversations to try and identify trends. It seems breeze can look into individual records and summarise, but not multiples. If you ask Breeze, it says it can, but it didn't work. So, at least for the time being, I doubt that it could train on conversation data, which would be an amazing tool and something I have been looking for on many platforms for years now!
I'm pleased to announce that we released the knowledge gaps feature last month which address this exact problem. All Service Hub Pro and Enterprise customers should have access now. If you don't have access or want to share feedback please do not hestitate to reply on this thread or DM me.
What is it?
Knowledge Gaps is a new page where users can see all the questions the agent responded to with "I don't know". Users can also add short answers for such questions so the agent can get it right the next time.
Why does it matter?
Today, a user has to go through every single conversation the Customer Agent has to identify questions the agent could not answer. This is not scalable as the number of conversations goes up. The Knowledge Gaps page makes it fast and easy to identify knowledge gaps and address them.
How does it work?
Click on "Breeze" in the main navigation and then on "Customer Agent".
Navigate to the "Performance" tab and then to "Knowledge gaps".
You will see a list of all the questions (clustered into topics) that the agent responded to with "I don't know".
Click on a topic to see the specific questions the agent couldn't answer.
Click on the "Create content" button to add an answer (the added answer can be edited any time in the "Content" section)
This is nice but I don't think constitutes as "delivered" for the idea presented or any of the subsequent ideas brought up. I can't use it, for example.
@Gade , as mentioned above, this really is not a solution for the issues mentioned. I couldn't allow the agent to go live in its current form to begin to collate gaps in knowledge this way. In our industry (healthcare), it is beyond imperative that we provide accurate information. Breeze could absolutely be configured for conversational-based training, which would solve nearly every problem above in one way or another.
@Gade I agree with @Shay_B above. There is no way I would consider this "delivered" based on what the initial request was as well as the additional items in the comments. Again, while *somewhat* helpful for the "I don't know" type questions, this does not constitue as being able to fully train the bot, improve the responses it gave, add additional training materials, train based on conversation history, etc. as provided in the comment section. I agree with @DLaverty that this is falling behind the curve and fast. I don't see how this would be a "delivered" item at all.
Hi @Shay_B , I agree, the knowledge gaps feature does not address all the asks in the comments, but I think it addresses the main idea (at least my interpretation of it). I'm happy to change the status to "Being Reviewed" but would love to hear from you what else from the main idea you want to see addressed and also what's preventing you from using the feature?
Hello! @Gade this is awesome and definitely helps a ton to further customize our agent!
If I may, I would like to suggest two features that would be absolute game changers:
In the same way that you would train a custom GPT, ideally we would have a field in the configuration settings where we can give it a prompt and it applies to all responses. Example use case: We have different pricing for all countries (different currency as well). Currently, when someone asks for the price, the AI, sometimes, provides USD pricing, sometimes EUR, and so on - it is pretty inconsistent. Ideally, we should be able to prompt the AI to ask the user what country they are from before replying with a response. We tried prompting this rule via short answers, even knowledge base and public URLs but with no luck.
The ability to add private content. Currently, the AI shares the source to the user, meaning we cannot upload private documents (dynamic database with all of our pricing, for all countries and products). We should have the option (a simple toggle option) where we can make the content we uploaded shareable or not.
Well, this feature is only helpful for training when there's enough training to even use the product to begin with. That's not where I am. I can't use the agent yet at all because I haven't trained it on anything. What I need is the ability to train it on either JIRA confluence and/or hubspot material, but we don't use blog or knowledgebase (yet). Ideally, I'd train it on how my support team has answered conversation inbox ticket requests over the last couple of years, with higher emphasis on more recent answers.
@AMcDonald67 , @DLaverty , @Shay_B , I've noted all your feedback, the good news is all these problems are on our radar and we are actively working to solve them. I agree that this post should not be considered "delivered" just yet, I will move it to "In Review".
Hi @Gade , is there a way to have more clarity regarding release dates for these new features? Since that will help us understand if we should wait a bit until they are released or if we need to start looking for other solutions, thanks.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
