Tom Arbuthnot: Welcome back to the show. This week we are talking AI integration into Teams and some of the journey of bringing together Microsoft Facilitator, which is Microsoft's kind of multiplayer meeting Agent. Really interesting conversation with Madhu, who's a Director of Engineering at Microsoft. We talked back through some of his history, going back to the Skype days, and then we talked about the evolution of adding AI capabilities into Teams and some of where it's going as well.

Thanks to Madhu for jumping on the podcast, and many thanks to AVI-SPL, who are the sponsor of this podcast. Really appreciate all their support On with the show. 

Madhu Sudan: I'm Madhu. Uh, I am Director of Engineering at Microsoft. Uh, my journey with Microsoft started with Skype, and from Skype to we built Skype for Business, and then whole Cloud wave hit.

We transform our need from Skype for Business to Teams a- and which was a great savior for a lot of people during the pandemic as well, um- 

Tom Arbuthnot: Yeah, it was lucky we had that Cloud scale because, uh, a lot of server infrastructures weren't ready for that level of load. 

Madhu Sudan: Yep, it was really, really, uh, you can say learning and full of, uh, you can say ex- n- um, the growth time at that time for Teams to see what and how people are using, and there are use cases which were really amazed to see how people are having their bread and butter on Teams, whether it's a hospital surveying or whether it's, uh, um, property manager showing properties over Teams or even airline doing their businesses over Team.

So th- this is really, uh, great to see how Teams, uh, uh, was able to hold the value and, uh, serve as a communication channel at that time. And even now I see that now Teams is becoming more of, uh, uh, you can say go-to, uh, tool for enterprise communication. And, uh, with the inflection with AI, I think it's very exciting time for Teams, how we communicate, uh, in groups, as well how AI can transform our communication, and that's where my role is in, uh, t- Microsoft Teams to add and adopt more AI into the, um, day-to-day communications.

Tom Arbuthnot: Yeah, it's a really exciting time. The, like, Teams is obviously one of the, if not the heaviest use case for Copilot with the meeting transcription. That's kind of a given for lots of people now. But I feel like the communications platform is a very natural place for the Agentic conversation, for interacting, obviously for meetings and actions and collaboration.

So it feels like to me the, the surface where Copilot and Agents are gonna come to life for, for a lot of certainly business and enterprise kind of customers. 

Madhu Sudan: Yeah, absolutely, and some of the things which we learn here is meeting itself has a cognitive load, and, uh, the more, uh, y- the intensive meeting is, the more load is on human.

And if AI can help, like for example, taking the notes on behalf of you or finding the right document at the right moment. Uh, imagine you are in a meeting with a customer and you are trying to figure out, "Hey, I had this thing, but I need to share with customer." What if AI can proactively bring it to you?

So all those use cases are getting built and helping our customer. So I'm very excited about, uh, one of the product which- Which we built is Facilitator, and that is... Th- think of Facilitator as your companion in group scenario or your teammate there who is helping driving how you can run your meeting effectively.

If you think of, uh, uh, this is a very often question asked here. You have Facilitator, you have Copilot, how do I see working hand to hand? So Copilot is like think of your own personal assistance or, uh, or companion which is helping you one-to-one, which all the conversation is privately happening between you and AI or you and, and Copilot, where y- whatever you are typing or you are receiving is between you and AI.

At the another hand, Facilitator, Facilitator is our group Agent, where Facilitator is hel- helping whole group rather than, uh, each individual level, for example, taking a notes or helping answer a question publicly and all those things. So Copilot is when you want to have a h- a help which is during the meeting or after the meeting, which you want to have at a individual basis.

So this is the Facilitator which is working alongside, uh, with you and your team, uh, taking notes, moderated meeting, showing your agenda, trying to, uh, help, uh, achieving the goal of a meeting, and you can, uh, mention any time Facilitator to get, um, uh... You can say any question related to meeting. It is grounded based on your chat, your, uh, active conversation, and it can also go and find information on web for you.

This is already available 

Tom Arbuthnot: Yeah, this is, this is quite a big jump because this is like, so yes, uh, I think meeting transcriptions are pretty well understood now. It can summarize, but this is much more proactive as in it can engage in the meeting, it can warn you when there's a certain time left to go, or you're not on the agenda topic, and even as you say, reach out to the web to potentially get content for you during 

Madhu Sudan: the meeting.

Yeah, it has more like advanced feature as well. For example, it can detect whether you have a quorum or not, or for example, due to any conflict you are unable to join but people need your help, it can a- mention you as well, "Hey, Tom, um, people are referring you for this reason. You might want to jump into the call."

So there are a lot of a- advanced Agentic features. So what we call as of now, this is a V1 of Facilitator, which is, uh, really reducing the cognitive load from the, uh, participant as well as organizer, let me play this video. It will give you, uh, what currently we have and in which direction we are going. 

Tom Arbuthnot: Great. 

Video: You and your team to help you manage your meetings effortlessly. Facilitator can be activated either when you're scheduling the meeting or during the meeting. To add Facilitator during a meeting, select Turn on Facilitator in the More tab at the top right of the window pane.

Facilitator moderates the discussion by using the agenda or identifying the goals set at the beginning of the meeting. At the halfway point and before wrapping up, Facilitator recaps key decisions and open questions, ensuring that important follow-up tasks and details are not overlooked. Throughout the meeting, Facilitator follows along, taking real-time notes and assigning follow-up tasks.

Meeting participants can co-author the notes, revise, and add critical details as the meeting progresses. For meetings where attendees have joined from a Microsoft Teams room, the in-room view will automatically adjust to display the real-time notes for those attendees After the meeting ends, you can easily access the meeting notes in the recap tab.

With Facilitator, your meetings become more productive and focused, leading to better outcomes and enhanced collaboration

Madhu Sudan: Yep. So this is a very high level, like what we have shipped for, um, our customer, like really establishing a shared goal and agenda, and then during the meeting capture notes, taking action item, answer any question what people have which can bring a clarity. And if somebody join late, it can recap as well as ask more question about the con- conversation.

So th- this is, uh, like at a very high level our, uh, um, like what pres- uh, Facilitator, um, is, is capable- Mm-hmm ... bringing to, to our customer 

Tom Arbuthnot: Yeah, it's really exciting. It's, it's interesting. It's a new paradigm to have this multiplayer AI Agent where we can all contribute to the notes and kind of it can achieve quorum of notes.

Previously, we would have a Copilot, a transcript, and a summary, and it would be our generated summary of the meeting, and then we might share that around. Whereas really what you want to be doing is working together in real time to make sure you've covered off the tasks and, and things like assigning actions is really interesting.

The idea potentially to go out to things like Planner and assign tasks is a really exciting area as well. 

Madhu Sudan: Absolutely. And, uh, o-o-one of the, like, challenge you can say or what we build is very unique from industry perspective here is lot of AI Agent, like if you talk to, um, Copilot, ChatGPT, or even Claude, like they are really one-to-one with you helping.

When it is in group, we need to really take care of the permissioning model as well as because releasing any conversation from your one-to-one or any other grounding than what we should be releasing is, is, is very key to Facilitator, how we have built permissioning model. It won't leak anything, whatever is happening in the group context, and that is a very unique, uh, aspect, uh, in any group Agent.

How does it secure and it abstract the information within that group, uh, so that it won't leak anywhere else, as well as it still provide all the AI capabilities because model are very eager to spit out the, uh, information, whatever it has. But having a, a right harness and data boundaries were pretty much needed, uh, in, in Facilitator.

So Facilitator is working alongside with other Agents as well. For example, there is a Agent called Channel Agent- Mm-hmm ... where Facilitator as of now can give more insight from meeting to channel meetings to help better grounding in the channel area as well. 

Tom Arbuthnot: Yeah, it's interesting. At the moment we have a lot of different single purpose Agents and kind of how the future comes together as to whether we have a, an Agent around the, the team that can deal with the meetings as well, or is it, is the Facilitator of the live meeting and the channel Agent the async content.

U- ultimately as a user, I kind of want to address one- Yeah ... thing and say, "You have the context of the entire project. I can't remember if we discussed it in chat or the meeting, but, like, here's what's going on with project XYZ." Uh, 

Madhu Sudan: absolutely, and this is, uh, one of the thing which AI is bringing, uh, really is the right memory, right context at the right time for our, our user in a, in a communication world.

So, um, it will be really, really helpful for lot of use cases and customers, especially they are like, uh, working, dealing with the very complex projects or dealing with customer communication, bringing right context at right time will be the key, and this is where Facilitator Copilot is very helpful for a customer to get those insight on the, on the, on, on the moment when they really need it.

Tom Arbuthnot: Yeah. You mentioned permission context as well. That's quite a complicated problem to solve, isn't it? Because if I'm in a meeting and I have the project plan on my OneDrive, me as the human decides to share that into the meeting or not. Uh, if the meeting needs access to that content, does it, does facili- is Facilitator allowed to get it because I'm in the meeting?

Do I have to ask the Facilitator to get it? Like, like where the, where the line is drawn on... Obviously it's in SharePoint, it's common context, but if it's- Well- Uh, if, if we all have different permissions to SharePoint, is it the lowest common denominator or the highest common denominator? 

Madhu Sudan: Yep, absolutely.

So mostly what we, we try to be a low- lowest common de- uh, uh, denominator. So w- what we normally, uh, does in this situation here is first it has to be user initiated. We don't go beyond meeting to figure out as of today what else is in the wild, because m- user like M365 data is like a big, uh, layer- Yeah

as to, uh, look into it. But still we, we, the user has a way to bring that data into the conversation, and that is where when user bring based on the access level and human in the loop, we definitely ask permission before sharing. 

Tom Arbuthnot: Yeah, I can see some kind of like, uh, on behalf of model where like if I ask Facilitator to get the thing, then it's, it's temporarily inheriting rights to get that thing on my behalf, but there's a very clear audit log that I asked it to go and get the thing. 

Madhu Sudan: And then there are two aspect. One is reactive, another one is proactive.

Reactive s- uh, skills from Agent or Facilitator perspective is when it reacts to user or it, it reacts to on behalf of user. Facilitator has a lot of proactiveness as well. For example, it can even detect a knowledge gap. You and me are talking something and it detect that, or in the group people are discussing and it detect that there is a knowledge gap in a group, but Facilitator has that access to web and it can bring the knowledge here.

Uh, so that is really, uh, very powerful, you can say, uh, tool we have in a meeting now. A Facilitator can bring a new context based on its, uh, data boundaries. 

Tom Arbuthnot: Nice. And as, as you've been developing Facilitator, how's it been in terms of model evolution? Because I feel like the exciting thing about a lot of these AI things is every time we get a new frontier model, we inherit some extra, extra power and extra capability, and also the, the, the, the, the knowledge around the harnesses seems to have got a lot stronger in certainly the last six months.

Madhu Sudan: Oh, absolutely. And, and this is, uh, the exciting or you can say challenging as well as exciting part of, uh, this whole AI journey is AI is evolving and it is evolving at a very rapid pace. And what we have seen here is model to model when the new model come, all the prompts and instruction which we have given, it start behaving differently even though the next model is for good reason.

But the same instruction which we previously gave may not be relevant. So that's where- Yeah ... we have lot of automations as well as our, uh, offline eval, online evals to make sure, uh, things are better in new model where we, we progress towards the quality of AI responses. Uh, th- that's where, um, lot of code engineering and evaluation is in place.

Like for example, uh, people can ask, "Hey, share, uh, recap this meeting." Some of the people like a longer response, and some of the people like shorter responses. Some of the people like bulleted point. Other will say, "I need a paragraph." So how do we learn and adapt to user expectation is the key fundamental of AI engineering here when we are dealing with in a group context.

Because, uh, in one-to-one it is easy to learn what user is doing in last five thing, and when you are in a group context you really need to establish that, uh, paradigm of way of communicating how Facilitator communicates to, um, uh, different people. So that's where is lot of learnings we have. Uh, with new model, uh, things change and sometime, uh, it's, uh, it's formatting is changing, sometime it is, uh, giving more context than it should be.

So all those things are interesting, but I am very passionate about new and new model. We always try to be on latest and greatest and get, uh, maximum out of it 

Tom Arbuthnot: No, it's really cool. I, yeah, I hadn't thought about the memory challenge. Like you say, when you've got personal engagement, you can the build a, a memory with the AI of, like, "I prefer bullets, and I prefer this style."

I guess the other challenge you've got that's really interesting and quite unique is you've got hundreds of millions of users. So, like, a me- meeting is not the same in my company as it is in your company- Absolutely ... as, as your country, as your territory, as your vertical. L- like, we all say meeting summary, and we all mean a different thing.

Madhu Sudan: And, and, and meetings are happening in a very diverse and culturally diverse, uh, ways as well. In, in some meeting, uh, talking over each other might be really fine, and in some countries it feel like no, this is not right in a corporate se- uh, setup. So how do we handle the challenge that it stay neutral? Uh, but if it stays very neutral, then it will not look like a native to my team only.

So how do we learn and adapt? So those are the challenges which we are still learning, still progressing. Lot to learn from it and, and enhance, uh, in, in our, uh, group conversation. And that's, that's where I'm very passionate because this is a industry first, or you can say very, uh, one of the first, uh, group Agent, which is running at this scale- Yeah

uh, in, in, in real-time communication. So it's, it's lot of learning, uh, and we have lot of good feedback coming directly from our customer. We- they are helping us getting, uh, getting it improve as well. 

Tom Arbuthnot: Yeah, I feel like M- Microsoft are poised to be the, the, the like the certainly one of the biggest scale multiplayer services for AI.

We haven't seen anybody, uh, you know, kind of from any of the other companies nail multiplayer. It's all, as you said earlier, kind of one-to-one, and multiplayer brings in all these different, different challenges of keeping everybody happy. 

Madhu Sudan: Nah, absolutely. 

Tom Arbuthnot: Awesome. Well, Madhu, thanks for jumping on the pod.

Really interesting to hear. I know there's more we're gonna talk about Facilitator when new features get announced. Uh, but, uh, hopefully we'll have you back again when we can talk about some of the newer stuff. 

Madhu Sudan: Oh, absolutely, and very nice talking to you, Tom, as well. 

Tom Arbuthnot: Thanks so much. 

Madhu Sudan: Thank you. Bye-bye.