topic Real-time audio streams in Conversations API Beta

Real-time audio streams

Jupita_team — Wed, 06 Apr 2022 12:18:04 GMT

Hi team!

Hope you're all well! How do we feel about taking a look at audio streams from HubSpot calls?

I believe all audio is sent as base64 encoded streams... it's not my field of expertise, but I think this is correct... if we can use some sort of markup language & access these audio streams in real-time we can fork them via websockets to a server of choosing, i.e a speech-to-text API like Google Speech. This allows for real time analysis of audio calls, something that has been around for a few years now.

The benefits of this are providing a full 360 degree view of customer interaction & communication, not being limited to just web chat & emails, + keeping up with the CX demands of 2022.

I think it's a great add on. As a fall back option/temporary solution, an API that provides real-time transcriptions of calls direct from HubSpot might suffice. Looking forward to hearing your thoughts on this!

Regards,

Alexander.

Re: Real-time audio streams

pleninger — Wed, 06 Apr 2022 19:51:26 GMT

Good morning,

I believe what you’re describing should already be possible with our existing APIs. One of the features of HubSpot Calling is called Conversational Intelligence which transcribes and analyzes calls that are made in HubSpot.

If you’d like to download calls for your own analysis, you can do so using the Engagements API - querying there will give you the URL through which you can download the call after its complete.

At this time, we don’t have plans to offer realtime audio streaming via API. However, if you want to leave an idea in the Ideas Forum, our product managers use that forum as a data point when considering our product roadmap, and I’m sure the Calling team would love to know more about this idea.

Best,
Paul

Re: Real-time audio streams

Jupita_team — Thu, 07 Apr 2022 13:55:01 GMT

Hi Paul,

Thanks for your response, I've submitted the idea to the ideas forum but I'm not going to hold my breath! They seem to have a lot to look at there.

Is there any other way to get these audio streams or transcriptions in real time? The conversational intelligence function is limited. Recorded calls are currently able to be retrieved and or transcribed via Google's Speech-to-Text, is this currently in real-time on your backend?

Is there any reason the conversations team can't run with this? It would complete the conversations offering to include calls.

Re: Real-time audio streams

pleninger — Thu, 07 Apr 2022 14:16:32 GMT

Good morning,

There isn't a way to stream audio or transcriptions in real time. I will be sure to pass on your idea to the team that handles calling, and let them know that there is interest in this feature.

At the moment, the only way to access calls is via the Engagements API.

Best,

Paul

Re: Real-time audio streams

Jupita_team — Thu, 07 Apr 2022 14:23:03 GMT

Thanks Paul, appreciate you flagging this with them. So I assume then calls are kept separate on purpose to conversations.

Does that team happen to have a group in the community forum, or any way we can communicate directly?

Re: Real-time audio streams

pleninger — Thu, 07 Apr 2022 14:30:55 GMT

@Jupita_team wrote:
Does that team happen to have a group in the community forum

None that I'm aware of. The Ideas forum is our main entry point for community-inspired feature ideas.

Re: Real-time audio streams

Jupita_team — Tue, 19 Apr 2022 00:58:33 GMT

I see... I'll need then to get in contact with the team directly. I've sent you a DM Paul.

Regards,

Alexander