069: Announcing Agent Studio, the world's only voice-first conversational AI platform

In this buzzy episode, Damien hosts PolyAI's Senior Vice President of Product, Devidas Desai, to announce the launch of Agent Studio, the world's only voice-first omnichannel CX platform for AI agents. Devidas provides an in-depth look into what Agent Studio is and discusses how the platform has been rigorously tested with live enterprise deployments and includes proprietary technologies like advanced automatic speech recognition and specialized language models. Hear about flagship tech like 'Talk Tracks,' and more of its future enhancements.

• Follow PolyAI on LinkedIn
• Watch this and other episodes of the Deep Learning pod on YouTube

Transcript

Devidas 0:00

The future is now. The future is here. We've been working hard at this for the last. Many months and the platform has been ready and we've been trading it, we've been making it perfect. And now it's time to let the world know that

Damien 0:18

Hello and welcome to another episode of Deep Learning with Poly ai, the only podcast that's operating on three functional hamstrings. Yeah, there's a story behind that. I'm your host, Damien, and with me is Poly AI's Senior Vice President of Product Devidas Desai. Hi Devidas. How are you?

Devidas 0:38

Damien, lovely. It's good to see you always.

Damien 0:40

It's great to see you as well. All right, three hamstrings, who's short, A hamstring who's operating on a non-functional hamstring. It's, yeah, give us the backstory.

Devidas 0:48

You can say three and a half. The backstory is that, yeah. Terrible mistake. Four weeks ago I was trying to, join my daughter in her wheels and I pulled a hamstring and then the first time around it wasn't so bad, so I thought I might do it the right the next time around, and then that definitely hurt. I'm still recovering. All right. I'm at down and stand up more comfortably.

Damien 1:07

We'll keep you nice and iced. Good thing about being on this podcast is it's not a highly active, a highly physical podcast. So welcome to the show. It's always a pleasure to have you on. The reason that De das is on with us today is a very big day for poly ai. In that two day, the day that we air this program, we will have launched Poly AI's Agent Studio. So this is a very significant milestone in our growth as a company, and Dedos is the captain of the ship. Dedos, why don't you tell us a little bit about what Agent Studio is?

Devidas 1:39

Yeah, absolutely. I'm super proud of the team. It absolutely takes a village. Each and every member of the company, each and every person in the company has contributed towards making this happen. Think of Agent Studio as the next version of our platform, our studio that helps us build, manage. On the performance of the agents all in top of one platform, irrespective of who the persona using the platform is, whether it is us. Whether it is our customers wanting to care for themselves and wanna spend 30 minutes making their assistance better and they don't have to be conversational AI experts there, we have tools built in with which they can care for themselves. Whether it's our partners building assistance for our customers on top of our platform there's something in it for everyone and it's what brings all of the personas together. All on top of one platform totally depends on your appetite for how dirty you want to get your hands in terms of being hands-on with respect to building and managing assistance.

Damien 2:34

Amazing. And obviously that isn't possible with what we could consider traditional agent builders, right? Anything that's just super co code laden, even leveraging LLMs externally to you just can't do it like you can in a platform that's made for, as you said, everybody to use. Help me understand a little bit about how we evolved as a company. To get to this point where we're now putting out a platform that we use ourselves internally to build those agents and manage, and now we're offering to enterprise clients to run and operate themselves.

Devidas 3:07

Yeah. Yeah, absolutely. The whole, when chat GPD happened, when generative AI came to the picture there was definitely a very clear use case with respect to us. Leveraging the power of generative ai. And make conversations better, more contextual, more personalized for everybody. But along with that came a few risks, right? What if the agent uses its own brain without, sticking to what it has been told or taught? How does it respect the boundaries? How does it take care of safety and security? The amount of work and. Proprietary tech investments that we've made in the platform give us a lot of confidence with respect to how we keep our assistance on track, how we keep our agents on track, how we make sure that they don't hallucinate, they always stay on track and provide the most accurate response. Can detect threats, manage risks on a real time basis, and take care of customer sensitive information. So those are all of the capabilities that are built into the product, and we have a high amount of confidence in terms of how the studio, the agent studio, provides tools. For the agent to perform accordingly and then on simultaneously give the managers the tools to review conversations and make sure that the performance has been accurate and the performance has been good. And because of, these controls being in place now the product has been battled, tested, right? We have a variety of live enterprise deployments. Over and over again. So this is not just fluff. We have live deployments built on top of the platform that we are managing on top of the platform. We are seeing incredible results with respect to the, how Agent Studios powering these agents how customers are using Agent Studio in order to keep the agents optimal, to perform them optimally. And making them better with the time that they're spending on Agent Studio. And now is the time we would like to show the world what we've got.

Damien 4:59

It's really interesting because as you mentioned, this isn't something where it's oh, let's put a flashy UI on something and call it a product. This has been battle tested. It's been used for live deployments that are actively been supported. Unfortunately, I don't think we're at liberty to name the clients who have been utilizing this on the backend, but very significant names in brands and travel utilities, et cetera. It's massive news. It, it's, what's that? Massive

Devidas 5:22

names. These are some massive names. Incredibly, the hugest,

Damien 5:26

The biggest logo. But you said, as you said, this has been time and battle tested. So that begs the question, we are just now announcing this platform out for, greater enterprise adoption, but we've been using it for a while and our clients have been using it for a while. What prompted the release now?

Devidas 5:42

Yeah, I would say it's more than the release. We are announcing the availability of the platform, right? It's been around for many months. We are a team. We are a company, like I said. That like to measure things 10 times and then cut once, because Sure. We absolutely wanna make sure that the platform works. Agent Studio just works and everybody loves it. Because we want to have that confidence that when we let the world know that, hey, we are here. We know that everyone's gonna love us, right? So these are not just theories or ideas about what the future could be like at Poly ai. The future is now. The future is here. We've been working hard at this for the last. Many months and the platform has been ready and we've been trading it, we've been making it perfect. And now it's time to let the world know that hey, here we are. This is Agent Studio, these are the logos that are already supported on Agent Studio. These are the customers, and I'll let the marketing team decide whether and when we can talk about it. Customers are using the platform. It is so real. It is absolutely real. And with this launch and with us getting Agent Studio into more and more customers hands. It'll only create a feedback loop that's gonna make all of our agents better. It's gonna make the platform better. It's gonna make our underlying tech better for everybody, for all of our assistant, even the ones that are already deployed over the air, because those are the investments that we are making in the product.

Damien 7:03

It's very true. I'm very excited about it too. I've seen what it's capable of. I've also seen some, let's be honest, competitive agent studios. There's a lot of access to very potent LLMs and generative AI solutions out there that can power with a nice little interface. We always refer to'em as wrapper companies, coming in and basically putting a slick interface on top of an LLM and calling it a product. Yeah. We are not that. So what is it that we're doing? What sort of tools and functionality, and you've already alluded to a few, but what sort of tools and functionality are available in Agent Studio that really take it to the next level and differentiate us from that? From a very noisy market?

Devidas 7:41

Yeah. Yeah. No, that's a great question. And I think it's a combination of what the platform offers and the investments that we are making in the underlying. Technical stack. And that is what really makes the entire offering and Asian studio truly unique. I'm, I can very proudly and confidently say we have the best in class. SR if not one of the best. Automatic. And that's automatic speech

Damien 8:05

recognition for That's our automatic For the acronym. For the acronym averse.

Devidas 8:10

That is correct. And like I said, we are already among the best in the industry, if not the best. It and it continuously improves with usage. And we are repeating that formula across. The world's most widely spoken languages globally. So that is proprietary tech. That's what we, like we are one of the best at in the industry. Yeah. I would say another example is us essentially tuning our own LLM building and tuning our own LM, which is, which I like to call it, it's purpose built for customer service. Okay. Of course we are plugging into some of the best. Models out there that are off the shelf, but we are also building our own. And that's what we are essentially using. Because this particular model, we call it poly, LLM, it's tuned specifically for customer service. It means that it's better at handling tasks. The responses are safer, they are correct as far as customer service use cases are concerned. And it's a model that understands the tone of the caller and appropriately supports those conversations. Now coming to the platform, coming to Asian Studio we are building tools truly meaningful and valuable for our customers, our builders. Whether you're an AI expert or you are not, we have tools that helps everybody make the assistance better. If you're a builder, you have access to a different set of tools and you would roll your sleeves and you would get your hands really dirty in terms of how do you make the performance better. If you are a manager or a reviewer of conversations, you can look at these conversations, identify improvements, make the updates yourself in terms of, a SR corrections in terms of knowledge base updates, in terms of getting to problematic calls much faster and being able to update your assistant, being able to make your assistance better. Those tools are built into the platform itself. One of the other differentiators that. We have in the product is and without going into too much detail because it's our flagship builder feature called Talk Track. Oh, we Secret sauce

Damien 10:03

stuff. Secret sauce talk. I love it.

Devidas 10:06

Alright. I'll try to stay as high level as possible and just press the red beep alarm. If I'm going into details that I shouldn't know but basically it helps our agent designers combine the flexibility of generative AI with programmatic code. And we have made the tools available in Agent Studio and they're very simple to use, and that is what allows you to bring Gen AI and code together and keep the agent creative as they should be, but also on track, which they must be. Especially because the assistant is serving or the agent is serving enterprise enterprises and enterprise use cases and all of this in, under the roof of one platform that everyone has access to. And I believe those are some of the things that we are doing very uniquely and in a way that we know works at scale for our deployments. And we are bringing everything under the roof in, in, in one platform essentially.

Damien 10:58

Now talk tracks. It sounds like you said it's applying some rules to the power of generative ai. Is that just a flashy term for rag or retrieval, augmented, generative, or is there something else that's going on with talk tracks?

Devidas 11:11

No, I would say a lot more things are going on in, in addition to talk, in addition to rag, right Rag is one of the competence in terms of talk track working as well as it does. But there's a bunch of things coming together and if I talk any further, I think I might be entering the

Damien 11:26

divulging too. Too much information. We're sending the IP police your way. Yeah. Hey and it seems like you're really passionate about all this. I know this has been your baby. I. When it comes to what you are most excited about, is it something you've already discussed? Is it that functionality that user provisioning that allows anyone within an organization to affect and, fine tune their representative agent? Is there something else that you haven't gotten a chance to talk about that you're really excited? Is it about the potential stickiness or the adoption that you're seeing from existing clients or something beyond? Yeah. Tell us what you're most excited about Agent Studio.

Devidas 11:59

Yeah. No, I, that's a great question and I'm so glad you asked me because I think I'll be saying something that I've never said before or hopefully I'm able to package it appropriately. It's all of those things. Absolutely. But then I think what I'm most excited about with Asian Studio is the fact that it is allowing us to get into a world where. Where truly we are not just building great agents, but we are building smart behaviors for all of the users of Agent Studio. Instead of having to plan for every little thing that the caller might say, we are now giving teams and agent designers, a way to build agents that are knowledgeable, a way to build agents that are learning constantly a way to build agents that can adapt on a real time basis. And agents that can just improve on their own based on real conversations and feedback provided by the supervisors or the managers, so to speak. It sounds like the reli,

Damien 12:52

The reliability piece has gotta be huge. I know that's one of the things that as generative has become more widely adopted within enterprise and even consumer use. That there's always questions of reliability. And it seems like we've made that a central focus point as we develop this platform. Absolutely. A hundred percent.

Devidas 13:08

And not just, we are not just playing the trust as card. We are reliable. We right. I think reliability with transparency is super important because you can see this work in the platform. You will have tools in the agent, in Agent Studio that. Shows you that the agent is not hallucinating it's citing the topics that it's picking from. Or if you look at the safety dashboard and you wanna drill into certain conversations that you would wanna review and make sure that the agent handled the threat appropriately, you can actually review the conversations and see that. Okay. The agent absolutely handed off to the appropriate destination when it should have, and it's following the safety controls that I've put in place. So it's not just reliability. Reliability is one part of the puzzle, but the other is transparency around how the agents are behaving and bringing all of that information in a very simple, transparent fashion on top of the platform.

Damien 14:02

We know that's been in high demand for recent reasoning models of LLM, so it only makes sense to see that same level of transparency and process like visibility in this enterprise grade product. Yeah. And you've already told us a lot about what we can expect now go to our site check out our Agent Studio product page. What can people expect in the short future? I know that we're making this announcement and it's already very feature rich. What's in the pipe with regards to how we're gonna make it even better in the short term?

Devidas 14:32

Yeah. A lot of things, right? Time is not enough. Time is not your time in terms of how much we want to do and the amount of time we got to, bring all of those things into the platform. But we have a very. Like power packed roadmap ahead of us. In addition to all of the voice capabilities, we will be expanding our omnichannel capabilities within the platform and bringing those controls on top of the platform as well. On top of Agent Studio we will be building seamless multimodal capabilities because we believe that. There are other channels with which the voice experience can be better. You could leverage like information sent through a text or filling out a form the web on a website while simultaneously being, on the phone with a voice agent. And it really doesn't matter how the information goes to the agent, the conversation can move forward. So we are building seamless multimodal capabilities within voice interactions. We are gonna make investments with respect to. If I may use the word low, codifying the platform even more for our customers, for our external builders, so they can build and manage assistance or agents even more seamlessly. And simply, we will have an army of micro agents working for you. In addition to the poly AI agent, of course, that's the primary agent. But we imagine you'll have a caller agent, which is simulating, callers of different personalities. Calling for the poly agent and making sure that the poly agent is staying on track and is responding well. We'll have supervisor agents running on the platform that are reviewing conversations and giving you information about which conversations are good versus not so great, versus the ones that require a little bit more tuning. We believe the future is anti dashboard, and so that brings to life a bi agent in the future. You shouldn't have to look at the dashboard and deduce information. You should be able to just. Chat with your bi assistant, your BI agent, who can look at everything that's going on with respect to your deployment and give you information by just responding to your questions around top call drivers, some of the best calls that you would like to listen to. What percentage of calls led to successful reservations and so on and so forth. It's so

Damien 16:31

difficult to conceptualize that because we're such a, we're especially in the enterprise environments where everything is, tab dashboards. Analytic data stores that you can access and compile yourself or at least do a little bit of personalization. Can you imagine when the time comes where everything is just prompt based, give me an output of the last six months of calls and tell me how many of them included non resolutions. Yeah, absolutely. That's

Devidas 16:57

what, that's what AI is doing. AI is powering no ui. It should be as simple and the fastest form of interaction. And how do I interact with you in the fastest way? Like before I make a call, I just send you a message, right? Just right, I wanna send, but in the, I wanna talk

Damien 17:10

in the interim, you might as well have a really slick user, inter or user interface and a great user experience and some robust dashboard. Absolutely. To give you a deeper dive into that data,

Devidas 17:19

dashboards are not going away. We'll always have, amazing dashboards using which you can click and drill into things and find out. More insights, but you are basically drilling into what you're seeing. If you want to customize and personalize your information finding, then it has to be creative. It's coming from your mind. And then we wanna make sure that there's a bi agent there that's willing, ready, running to support you and giving you answers to the questions that you have.

Damien 17:45

in addition to just the general excitement, I've been showcasing a lot of the agent studio capabilities to a lot of industry analyst firms out there. Yeah. And the feedback has been going back to differentiators that nobody's touching. Our audio management and voice management piece, like voice has always been front of mind for us. And ensuring that seamless experience and that exquisite experience translates in this platform is fantastic. The function calls the API library that we have. That to your point, takes a little bit more code. All right. We're still pro code coders. We still need you. All right. But the the interface for tagging function calls within certain conversation flows. Nobody else is doing that. There's just so much positive feedback that we've gotten with regards to functionality and the experience itself as an end user has been top notch. So I just wanna let you know that you and your team, that you're getting warm accolades from a lot of those a lot of those analyst firms out there that, we ride on the unbiased, we're gonna tell, we're gonna call it like we see it Mo and they're loving what they see so far.

Devidas 18:48

Oh it's the team. It's the team. I genuinely, every function in the organization has come together. To build this platform and shape it in some way, shape, or form. And it's amazing that it's working really well and I really cannot wait to, see where Agent Studio takes us and how quickly. I think the platform is really gonna propel the future iterations, especially now that we are launching it and we're gonna give, we are gonna put it in more and more hands. It's gonna really drive additional feedback additional iterations, and I just can't wait for the future.

Damien 19:18

It's a fun rocket to be on, that's for sure. And for those of you who are frequent listeners, you know that we don't often sit here and do a little self-promotional episodes. So give us this one, grace. We're very excited about Agent Studio. We hope you are too. Deas, it was wonderful to have you on the show. Thank you so much for stopping by, and we'll bring you back on again in shorter time so you can tell us about all the magic that's happening behind the scenes. Okay. Thanks for having me. Of course. for the rest of you, we will talk to you next week. Take care.