It would be an extremely helpful feature to have the ability to A/B test a bot. This could be a functionality added in the "Target" section of the bot editor. The option should allow the option to, based on the other targeting display rules, just show the bot to a certain percentage of visitors who navigate to the page, such as 50%. This way, we could have a straightforward way of testing the impact a bot has on conversion rates, for example.
Yes, this is an idea I emailed into Hubspot support about! Adding a chatbot is a significant change to your customer experience and it's important that you can use A/B tests to measure how it impacts conversion metrics.
We are currently using Drift and this feature is one of the reasons our team refuses to migrate to HubSpot's chat. If we could get this feature and some better conversation routing rules it would be great.