SPEAKER_01 0:05

Hi, and welcome back to the Teams Insider podcast. This week we are going deep on the TAP program, the technology adoption program. So this is the program where customers and partners get to test features early and give feedback to Microsoft. We talked to Alex Ivanov, who's a product manager there. In particular, he looks after Teams devices, Teams Android devices. I'm going to get deep into how that testing program works, how it works with the OEMs and with the customers, how you give feedback and what happens to all that feedback. Really interesting conversation with Alex, and also there's an opportunity to join that program. So if you listen and feel like it's a fit, there is a link in the description to submit your interest. Thanks again to Alex for joining the pod and giving us such a great conversation. And also many thanks to Loggy, who are the sponsor of this podcast. Really appreciate all their support. On with the show. Hi everybody, welcome back to the podcast. This week we're going deep on Microsoft Teams and Android devices. Alex is a known face in the community, and uh he just recently did a session at Comms V Next talking about this, and uh I grabbed him afterwards and said we've got to do a podcast on the on the same topic. So uh welcome to the show, Alex.

SPEAKER_00 1:15

Thank you so much, Tom. It's my pleasure to be with you and uh have a chat about uh TAP and Teams.

SPEAKER_01 1:22

Awesome. So let's start off with um you and kind of your journey in Microsoft and your current role.

SPEAKER_00 1:28

So I'm a product manager at Microsoft, uh CMD organization. Well, now it probably has more letters, but uh basically the focus is on call-in meetings, um, devices, immersive experiences, and uh as a product manager my product is uh quality, quality, and right now my focus, primary focus is devices, and also I'm working on the customer feedback, so all things uh NPS uh on devices, it comes through me. I I'm I'm reading that. Um and as a product manager to gather all that feedback and and turn it into constructive feedback and actionable feedback. I also a program manager, so I run program uh called we we call TAP 100, uh, which uh which stands for technology adoption uh program. Uh, but the focus is to gather all this feedback from our customers and uh make sure that uh we address all uh feature requests and all issues that customers discover. And uh again, primary focus is to make sure that whatever we build is a high quality and very delighting to our customers. So, yeah, that's my job.

SPEAKER_01 2:52

That's awesome. I love the the product being quality. That's such a uh a great way to think about it. And and devices is quite a challenging area because it's uh partnership ecosystem model, so it's Microsoft and and the OEM partners in the case of devices.

SPEAKER_00 3:10

Yeah, absolutely. It's uh we we have a huge ecosystem and a huge variety and diversity of devices and uh and OEMs and partners, and uh uh we we have two platforms. I mean, we have uh uh meeting rooms. Uh we started with meeting rooms on Windows and then we expanded into Android area. So uh our biggest growing area with Android devices, bunch of uh new OEMs and partners uh joining this program. Everybody wants to build uh meeting rooms for teams, and and we we have phones uh to support you know our customer needs, and we have new you know, panels and uh whatever other product, of course, peripherals. Peripherals is also very important in Teams world if you want to get high quality. And uh again, you know, we we build in new features, we certify devices, and uh I'm deeply involved uh in in every aspect that when it comes to quality.

SPEAKER_01 4:20

Awesome. So so the TAP program, like obviously some people listening to this podcast might be familiar, but plenty won't be. Talk us through what that actually is and how that works.

SPEAKER_00 4:32

Yeah, absolutely. So TAP program is designed as uh engineering program, it's uh it's a private preview by invitation only. And uh uh why we're doing this, we want to get customer feedback um in early stages. We we get uh our customers engaged uh at very early stages when our PMs are dreaming out the feature, and you know, they build some mockups and they come to customers and get feedback if you know from multiple uh points of view, like does it even make sense? And uh how you know our vision to execute on that feature. So we get that feedback from customers and we take it, and you know, if necessary, we adjust and and start building the product, and then we kind of check in with our customers on um you know our progress and and and how how how features work. But why we do this, uh basically we want to ensure high quality, we want to ensure that great experience with our uh you know teams, with product, with and teams devices specifically. So and um what we do, we have this preview program where we recruit and on board uh our customers. Uh it's uh it's a unique group of people and customers that are dedicated uh to, you know, I I look at them not as the customers but as as our partners who are comfortable to be uh on the bleeding edge of technology, of Microsoft technology and Teams. And uh my goal is to take all that feedback and turn it into constructive, actionable items, actionable feedback, something that I can take it to engineering, I can take it to PMs and uh translate it into technical speak and uh uh you know make sure that uh whatever customers are looking for is implemented and and the quality is high. And uh uh with Step program, basically we're using Teams as a platform, so all collaboration is done in Teams, so we have Teams Team. Um makes sense, right? Yeah, absolutely. And uh, you know, uh we have multiple channels dedicated to you know each area. Of course, I have uh an own and manage devices channel and I manage my customers, and then there are multiple.

SPEAKER_01 7:15

And there are different there are different tap programs for different subsets of features as well, aren't they? So there'll be different different taps either temporarily or permanently for different areas of the product.

SPEAKER_00 7:24

Yeah. I would say that the program, there is only one program, but there are multiple work streams happening uh within this program. Uh, you know, there are big areas for you know, like colon meetings and devices and immersive meetings, and uh, you know, copilot obviously is a big part of it now. Um, so uh and and we have dedicated uh PMs uh that drive those areas. Again, I'm I'm focused on devices, and as part of my portfolio, like what do I do, right? So I do all things uh uh as a software validation, so Teams app uh for devices, authentication app. You you know we had a huge project that we've wrapped in up.

SPEAKER_01 8:13

Yeah, it's been a busy uh busy 18-24 months, hasn't it?

SPEAKER_00 8:17

Yes, yeah, it was AUSP, which is I I'm very pleased with the result. I think we we we took very balanced and uh um you know highly focused approach to deliver and and uh test out, and uh that's that's again where our partnership with customers uh came big time. Basically, you know, again we recruited customers uh for this validation and uh we were releasing uh AOSP-based firmware to customers, they were testing it in their environment. That's a big part of uh private preview because we we built features um and um in our environment, right? And we test features in our environment. And when we test in, we know how they supposed to work, right? So we essentially have this uh creator bias, and when we test in in our environments, of course it works. Of course it works as experts.

SPEAKER_01 9:31

You know what combination of buttons you're expecting to press, you press them in that order, and the code works.

SPEAKER_00 9:36

Yeah, we absolutely know how it works, and of course it works perfectly, it's like flawless. But uh with Tab, we bring our Tab customers with their environment, with their unique uh uh you know uh policies and network setup and security setup, and totally different user expectations or life expectations. And uh that's where it actually might break, because you know what we think, you know, uh fantastic things and you know, something brilliant, you know, since SliceBreat was invented. Uh, for customers based on their experience, it could be not exactly what they're looking for, even though we're talking the same words, right? Or the same feature buttons.

SPEAKER_01 10:24

That's where tap's really interesting for me because it's a combination of technical validation, like does it work, but the feedback from the customers. And I I've heard calls where it's interesting where two different customers are complete opposite opinions of the feature, like it really suits us and our way we work, and it doesn't really suit this customer. So it's it's not black and white, it's uh kind of sea of grey.

SPEAKER_00 10:45

Yeah. Yeah, absolutely, because everybody has their unique experiences and they they have perception of those unique, you know, the features based on their unique experience. And then, of course, there is reality kit that you know your network uh built in one particular or architected and built in one particular way, and there is security settings that you know might not necessarily allow, you know. So we we we find all those things in this validation process, and uh that's that's how we we learn, and uh we uh you know again I run very structured validations as you know. And yeah, so you know, my file debug, it's like running line on my forehead, and and but again, it allows to take that uh you know freewheeling feedback into something uh you know uh structured and constructed that I can take it to engineering and they can they can translate it, they can look at the logs because logs are attached, so they can get like uh much more uh down-to-earth technical information, environmental information, like to fit to to to actually enable them to rot cost and uh uh fix it or you know redesign. So um that's that's part of process and that's a lot of fun.

SPEAKER_01 12:13

Yeah, no, it's great. I think it's a really great opportunity for customers because they get to see some of the early process. So they see what's coming down the pipe because this is stuff that they're going to eventually probably deploy to their business, and they get a unique opportunity to voice their opinion. Doesn't mean it's always going to get accepted or rejected in either way, but you have that chance to have your say on on direction, which is is huge, I think.

SPEAKER_00 12:37

Yeah, absolutely. And again, uh it it and actually it never gets rejected. You know, it's we we listen, uh our ears wide open, our eyes are wide open. If customers have any suggestions, we we document it and it goes on the backlog, it doesn't go anywhere. It's I mean, it's it's in the backlog. And you know, if if it makes sense, if enough customers talking about one particular feature, uh, you know, we we will prioritize, we uh we have a process and we prioritize and we will introduce at some point.

SPEAKER_01 13:12

Yeah. So you mentioned it like I I I kind of I think we've talked about some of the real advantages of being on it. There is a commitment for customers to engage and spend some time on doing testing and and and being engaged in the weekly calls and things.

SPEAKER_00 13:26

Yeah, absolutely. Uh that's actually a very good point. I mean, it's very cool to be uh uh uh in a in a private preview program for for teams. You get uh again, in order for us to get feedback from our customers, we actually need to expose you to all those features very early. Um and and uh basically uh we have uh rings, and like I, because I'm in product group, I I I live in ring zero, but our customers' IT admins actually get placed very close to the product group. And basically we dog food in hard you know big time internally, and uh at some point uh when we have enough confidence that uh it works very well for for us from usability point of view and from quality, because again, I do my business day-to-day job uh based in you know dog food environments. So it's it's actually pretty pretty good. Um so at some point we expose our customers' IT admins to those features very early. Uh and uh the main focus is um to uh uh to get initial feedback like uh at the at a functional level. Does it work in a customer environment? Again, customer environment is unique. And does it make sense? I mean, in addition to IT admins, we also uh uh onboard power users. So again, you know, adoption specialists or trainers uh from from the customers. So again, they kind of represent their uh user base in their organization and they provide feedback, you know, aside from break fix, like does it even work in their environment? But then does it make sense? Does the uh feature use flow make sense? Does it do what uh they expect, what their user expects? So that's a that's a very first kind of line of defense on on the customer side. Uh and and uh another benefit of having IT admins in this early ring is they're technical enough. So when they find the problem and they can file a bug and they can actually uh you know can speak engineering talk, right? Yeah, and we can go back and you know uh get the like in more structure, like, okay, what are repro steps and what did you do and what did you expect to happen? And then can you get more locks? And then you know, it's they're technical enough. I mean, they very technical people, so we can work at engineering level uh uh to gather information, to road cause, and then um when we fix it, again roll out to them and and and get this high highly technical feedback that did we fix it, does it work in again a customer environment? And then next step, actually, we have feature testing at scale. Basically, uh the next string uh is where uh you know customers can onboard their uh basically main population of users, business users, and uh again we do get uh feedback uh at the user level, meaning you know, usability point of view, but also we get some uh test uh scale uh testing uh from that because you know feature might work uh perfectly, or you know, backend can handle anything when there is only 10 users, but when we have millions of users, well, you know, that's a scalability question.

SPEAKER_01 17:11

You try you throw these AI features out and suddenly the cogs are uh quite extravagant as the you get into the millions of users.

SPEAKER_00 17:18

Yeah, absolutely. I mean, even feature like features uh as simple as uh read receipt, it could uh you know, it could have a significant impact on your back end. So so yeah. And then when it comes to devices, it's even more interesting because it's not just uh feature enablement on the back end and rolling out to the client, but we also have to get you know that particular Teams app version on that particular device, and and there are so many variables like there is a Teams app, there is authentication app, there is an Intune app now with OSP, and then there is a uh OEM partner uh management app, and then the whole firmware package. So there is a lot of work streams happening just on on a physical device to get physical device up to date in addition to features. So basically um a lot of focus is on uh software quality, so all uh updates like you know, right now we're testing update number eight for phones, right? And and we're testing hotfix for MTRAs. So all that going through, I mean it it went through internal testing and uh and now it's released to our preview customers, and you know, now I'm uh basically doing a lot of marketing for for my customers to to install that software and test all those exciting features that come in. Um and uh you know, again, if they find something, they they they they file a bug and and we go and fix it. Um in addition to that, you know, all those apps and and we release firmware packages uh again before they go GA. So there is a lot of going on in this area. Um in other work streams that I own, we we do certifications, obviously. So uh let's talk let's talk through that.

SPEAKER_01 19:21

I mean I've talked about before, but like like certification in the sense of not just anyone can say they've got a team's device, they have to go through this process. So, what does that look like?

SPEAKER_00 19:32

So, yeah, it's uh it's it's it's an industry standard, right? I mean, I remember good old times when anybody could slap uh Skype logo on on headset, and you know, you you never know uh you know if it really works. Usually it works, right? I mean, uh the most popular headset at some point was uh Apple earpods because everybody got them with their phone. Uh but if you really want you know high quality audio with uh you know advanced features and uh call controls, uh you probably want to get, I mean, you definitely want to get a certified headset and and and certified, I mean, especially headset for audio quality because it's like everybody using uh devices.

SPEAKER_01 20:20

It's not just uh vendors can pay money and get the badge, is it? They have to submit hardware, it's it's thoroughly tested, like there's a lot to it.

SPEAKER_00 20:28

Absolutely. They so basically they go through uh third-party uh certification uh company, which you know they partner with, we partner with. Uh, but uh on top of like you know, this industry standard certification, we get a we we we add additional layer of TAP certification. Basically, you know, it's more f it's it's more comprehensive, it's not as deep technical, you know, 300 plus industry standard tests on decibels and you know, levels and all whole nine yards, but it's it's again user experience comes into play. And and uh we we get you know we we we get some very interesting uh scenarios where customers find some specific uh issues specific to their environments uh that might not necessarily even get covered in industry standard because industry standard uh testing is like you know, there is uh micro steps, right? And and each step checks out, it's all good. But when you put it all together, there might be hiccup. And you know, we discover it and we fix it with our partners. So it's it's a it's a great um uh user user focused, I would say, uh uh additional layer of uh certification validation. Um and you probably see my some of my demos. um so yeah it's it's it's it's great um and then um another uh another s another area that I own and I I mentioned is NPS so um uh we we track uh you know user delight uh we need to track it somehow right and uh there is again industry standard uh for user delight tracking or user satisfaction tracker tracking is uh net promoter score uh it's uh the the beauty of this platform is that uh applicable it it's not uh industry specific or area specific it's across the board you can you can look it up uh nps for Apple for Tesla for Ford for you know for hotels you know anything any company and there is a it's it's it's a it it's it's a that the number nps score you know translates uh just fine uh across the board um so the way we get this net promoter score is we ask users questions right so uh some customers some users are not happy about it totally understand it because uh you know you you you you just got of the call you just got of the meeting and we popped the question but it's not because you know we we want to keep you busy or we want to destruct you or we want to annoy you we want to know uh what you think about our uh uh product and what you what you uh what your experience is um and uh many people think it's uh it goes to black hole well it's it doesn't go to black hole or I am the black hole I actually read every single uh uh you know comment that uh uh our users put on their devices and I deeply appreciate you know their effort because you know it's it's uh it's a lot of fun to type uh you know comment on uh and I'll be able to do that.

SPEAKER_01 24:10

I really care if they've taken the time to give the feedback.

SPEAKER_00 24:12

Yeah yeah well and and then again mostly it's negative because people care to type this in the off to go back your day it's uh if you've got a complaint. That's the nature of the beast isn't it yeah yeah but but it's it's uh uh I again I read every single comment and then I I actually share all those comments uh I you know aggregate the summary I share it with the team I have monthly call so it doesn't go to the black hole I am a black hole that actually read in all those messages and um um sometimes there is a lot of passion uh totally understand it um uh but again our focus is to deliver high quality user experience on every single you know surface that you interact with teams as and and teams is a big part of it uh especially with our uh you know uh offering on on Teams rooms you know where uh it's it's it's an amazing uh solution and uh we want to make sure that everything works my my ask to to the users you know when you type in and and you're passionate I really appreciate the passion but be more specific you know if you're just saying that you know teams you know in very adverse terms not so good some of that yeah if it did not meet your expectation please let me know what exactly because when it's generic like hmm you know but if you say something along the lines like oh my microphone you know wasn't wasn't good or people didn't hear me or video or there was some flickering or there was some you know interruption or cold drop or something. I mean give me something that I can categorize and and actually drill down and and focus. So that's that's a lot of fun to to to do that review and and share it with the team because the team is amazing in in listening feedback. I mean I get uh you know GMs and GPMs uh so you know all the way up uh everybody you know listening in and uh we have a very lovely conversation uh with drill down on on some topics I mean when I read uh you know all the feedback when I review all the NPS scores um you know I pinpoint some particular areas that kind of jumps out right but but uh the team can can zoom in on one particular area and we drill down really hard and real deep uh because um again we we're looking at uh you know multiple uh feedback sources you know we have our telemetry we have our conversation with customers even at events you know uh like Infocom or uh there is one in Barcelona so again yeah we we go to those conferences not just to you know to show off right but uh also listen to our customers we have multiple customer meetings so we're all ears we we listen uh and we take that feedback and you know we look at telemetry we look at nps we look at the what's happening in tap and uh it's really great to hear that because it's it can kind of feel like Teams is such a huge product you know hundreds of millions of users m well over a million rooms in play it's nice for customers to hear that that stuff is going back and somebody does you know like really care about that feedback and it not just your job but the whole product team then are taking that feedback. Yeah we are extremely customer focused we're listening we you know I'm reading Reddit I'm I'm getting daily I'm I'm I'm getting daily updates from Reddit uh about you know Microsoft Teams subreddit um you know LinkedIn is is a place where I'm you know monitoring and and proactively posting out I mean um I'm I'm monitoring all comments uh on your post and all MVP's post uh you know uh even in in um our technet articles also have a comment section that that's another area for feedback but again my goal is to take that feedback and and make an act make it action actionable so because when it just uh you know just a comment then you know I I I might even go on an investigation if this customer particular customer is in a private preview and you know I would reach out and hey you know I saw this feedback from you know your organization can we get it in in a bug format or you know can you can you open a support case with Microsoft so again we get some uh real technical uh background and logs and stuff and we can uh get it uh looked by uh engineering.

SPEAKER_01 29:23

Awesome and and as we go into the new FY you've got opportunity potentially for some more customers to come into tap if they've if they've been excited about this conversation and everything and you've got this far in the conversation if you're listening so you're probably definitely interested in in the the weeds of how this stuff works. Yeah absolutely so I'm actually very excited right now because um I am recruiting uh customers for uh teams android devices uh with some potential for uh MTRs uh on Windows as well but my focus right now is to get uh teams android customers and uh uh you know if if you as a customer are comfortable to be on a bleeding edge and you know want to uh be part of this very cool private very private program um you get to interact with me weekly on a weekly basis you get uh exposure to PMs where PMs come to my meetings uh uh uh every Thursday and they talk about features they present yeah we've very early briefings on what they're thinking and again they're willing to hear the feedback and the suggestions like it's uh it really is a chance to get an early view on what's coming down the pipe. Yeah yeah if you if you want to be part of the school and Viber community for private preview you can go to aka ms slash join tap 100 and select option five Teams Android devices tab um that uh that will that nomination uh will get into my hands I'm monitoring that pipeline and you know uh you you get uh uh opportunity to see early preview um and uh at this point uh again why I'm excited we we have this preview for uh teams android teams uh teams android devices software preview it's very now user friendly or IT admin friendly yeah let's think through what you've done there because you it's a big step forward I think for for the typical customer isn't it because they can now select specific devices within their environment so if for example put all the MTROAs in the IT building on tap and and see how those work first.

SPEAKER_00 31:43

Yeah I mean now there is literally granular control uh which devices can be provisioned for preview uh and uh you know lots of flexibility and uh it's it's it's a great opportunity for you to be to be become our partner in this uh exciting uh bleeding edge technology journey with teams yeah if you think you're interested I'd definitely say um put put your name in in the list and and you might get a chance to talk to Alex about it.

SPEAKER_01 32:16

There's a commitment there to to do some work and test the features but the the visibility you get particularly if you're a bigger Teams customer like you get this chance to see what's coming down the pipe before it hits your mainstream users as well.

SPEAKER_00 32:27

Yeah absolutely I mean and and speaking of commitment I I'm I'm I'm glad you mentioned it there is definitely commitment but uh in terms of time uh because uh to get constructive feedback you know you need to file a bug but and uh first time it might take you like five minutes to file a bug um it's but then you get better and it just uh seamless you know you you you run into issue or you don't you you you don't you know it's it's might not issue but you don't like you know what happened or you know how things behave you just file a bug from whatever uh from team's device and then um you know hopefully uh uh you you put a good description I mean and there is follow-up right you know we we we will need uh to get the full picture so there is some time investment but I look at this uh and I hear from my customers that that time investment is well worth it because uh you you get exposure to all those features you get influence over how those features are executed how they work um you get uh opportunity to influence you know with your knowledge and and how your environment built that's what happened in in AOSP because when we were testing AOSP we were I mean uh we were releasing AOSP in a private preview and uh initially some customers had challenges you know things just didn't work because you know of course and and and not surprisingly they work just fine in our environment in our testing but in their environment there were some caveats around you know Intune configuration or um um MDM configuration so there was all those questions that uh customers were asking and they were able to kind of influence what what we prioritize it's not the best engagement you're ever going to get it's not like you're raising a generic support to the generic support there you are right on that edge of like okay we need to work this out together because it has to work for customers. Yeah and it also helps us to prioritize like what what we should address first because you know we how we have our in you know person like internal uh vision and uh data what's uh what's priority but uh you know voice of customers is very important and and again we'll listen and and we execute on it so yeah awesome Alex thanks for kind of uh opening the curtain a little bit on that TAP program I I appreciate the opportunity to be be on it as well it's uh definitely if you're interested we'll share the link in the uh description as well for the TAT program so take a look at that and uh Alex maybe we'll uh see see how the the year of features go and catch up with you again in the future yeah absolutely and um again I know everybody wants to you know look behind the curtain and uh I get a lot of questions like when things come in I I highly recommend I mean uh obviously public roadmap for Teams public roadmap I know you do a great job of uh posting on LinkedIn here is what's coming because you know some people get too busy with day-to-day jobs and keeping lights on uh but also uh message center post it's a it's a huge uh source of information and it's even better than a public roadmap because it has like very detailed like you know 10 paragraph feature description with pictures I mean you know it it helps so much to understand what it actually does when there are pictures so we post all that information in message center and all the updates to feature um and uh you know I I talked about the USP I should mention mDAP as well because it's it's a it's a next wave of big things big chains that uh we're working on and we already released uh you know multiple devices with mdap support and again Chris had a great talk with you about what mdap like what yeah yeah coming in about a month to do a delete on it as well but I have it on uh uh I I have that flow uh going through private preview on on uh consistent basis so again another reason to join uh teams android devices uh tap um uh so yeah awesome well thanks for coming on Alex really appreciate it and uh talk to you again soon thank you so much Tom such a pleasure talking to you