Tom Arbuthnot: Welcome back to the show. This week we are talking Copilot adoption at a massive scale. We talk to Tom at EY all around their adoption journey in a huge organization, how they're handling it, how they're doing comms, his journey in the Copilot program, and how they've dealt with kind of early access and TAP and all that good stuff.

And also some really good tips on how to manage adoption and usage-based billing. Many thanks to Tom for jumping on the show. Really appreciate the conversation with him. And many thanks to Neat, who are the sponsor of this podcast. Really appreciate all their support of the community. On with the show.

Hey, everybody. Welcome back to the podcast. I'm really excited to have Tom on the podcast. First time I think we've had Tom on the podcast. Tom do you just wanna introduce yourself and your role?

Tom Crinnegan: Yeah, sure. Tom Crinnegan. So I, I'm at EY at the moment, so been here for, I think just over four, just over four years.

Always worked in the sort of with the Microsoft stack from the sort of Intune, early days of Intune. Think, I think when I was working at Qantas, to be one of the first sort of larger customers on Intune, so suffered all the, the, the interesting teething issues with Intune. The, the growing

Tom Arbuthnot: pains.

Tom Crinnegan: The growing pain. But you know, it ended up being a sort of a good deployment in the end. We, we, we got off a competing product, shall we say, that we were very happy to, to move away from. But yeah, worked in, you know, Teams, like looking after sort of SharePoint and Teams and Exchange over the years and then moved into the wonderful and ever-changing world of Copilot when I, you know, when I was at EY and the private preview sort of came along, or was it called in the days early access preview?

Tom Arbuthnot: Yeah. Sort of. Yeah, it's been renamed about three times or probably more.

Tom Crinnegan: It has. Yeah. We're back to Microsoft Copilot now, aren't we? Yeah. So- Yeah.

Tom Arbuthnot: And Fr- Frontier program and Frontier as a philosophy and Frontier as a ring.

Tom Crinnegan: Well, we've actually got other layers to that internally 'cause our Frontier resource, there's also a sort of huge cyber program that's all geared around Frontier as well.

So yeah, the, the term Frontier these days is has many, many meanings depending on who you're talking to. So yeah, look after the sort of Copilot sort of stack internally, not... Doesn't all sit with me, but my team generally look after all the kind of main, the main parts of M365 Copilot and, you know, things like Cowork and obviously with the emergence of things like Cove and things that are, they're coming into my team.

So yeah, we do, you know, enablement bit of change and adoption. That's pretty hard. We might get into that at scale sort of our scale. But-

Tom Arbuthnot: Well, your scale is unreal, right? And, and, and not only- Yeah ... is your scale unreal, but EY are a very progressive company now, all in on the E7, lots of consulting, like, like, like lots of I would say power users.

Tom Crinnegan: Yeah, for sure. Yeah. I mean, we've got 400 and sort of 20,000 people around the world, I think in over 140 countries. So we deal with, you know, all the sort of different regulatory sort of challenges that that introduces. So we, we, we are, we are always using all the possible admin controls to sort of divide policies up between what different parts of the firm are comfortable with.

Spend a lot of time with data protection and risk, you know, around what, you know, what we're comfortable enabling. So, so yeah, and, and you're right. Like, we, we are out there in the market talking to clients about, you know, AI tools so there's, you know, ex- a very, very high demand for the, for the latest access to the latest tools so that we, yeah, the, so that we can talk to clients about them, that, that we've got sort of a customer zero story in.

And we have, you know, the number of teams that are doing things that sort of blow me away from, like, tax through to RSAT sort of strategy and transaction team is, is kind of incredible. So we're ... If, if we, if my team are sort of ever feeling like we're behind the some of the client-facing groups sort of drag us into the, drag us into the future pretty quickly.

So yeah, it's very, very ... I mean, the, the, the, this world is fast-changing, right, in AI as it is, but, but the, the teams that we work with who are often in client-facing roles are, you know, they, they're often be- You know what it's like. Something comes out, there's a new model released, there's a- Yeah, yeah

something Microsoft announced, and clients within 12 hours, maybe less, are asking, "Yeah, what's, what's your perspective on how this fits in?" So we're, we're often sort of trying to preempt those types of questions which is where, you know, getting involved in things like the Frontier sort of TAP 100 programs come into play, right?

'Cause we, we spend a lot of our energy on, on those and participating heavily in them and you know, involved in customer advisory sessions so that we can stay, you know, as, as, as current and as, and as ahead of the things that are coming as we possibly can.

Tom Arbuthnot: And how, how's that ... I just really wanna get into kind of usage-based billing and the new code and also pilots and stuff, but, like, how's that journey been in such a big organization?

You know, I've, I've worked with EY in the past, and it's a, it's a beast of an organization. As you say, you've got the regulatory- Yeah ... requirements, you've got the global, you've got the different use cases. Yeah. It's been so fast-moving, even by Microsoft cloud standards

Tom Crinnegan: Yeah. I mean, so we, I mean, we, you know, we, when we were deploying, sort of starting the deployment journey for Copilot, and I think, you know, we- we're in a...

I- I feel like personally, you know, with me and my team, we're in a pretty good position because, you know, we, we have the E7, right? That makes, that, that, that removes some of the, the challenges around, like, have you got the right components of the licensing? Have you got, you know, Agent 365? You know, as that-

as that product's kind of maturing and has more capabilities coming into it, we don't have to sort of battle the, the sort of licensing conversations. Because I know, I know it's a real,

Tom Arbuthnot: a really big- Yeah. The, the, the, the have-have nots of the, the now, the kind of- Yeah ... Copilot and the E7 stuff is definitely a challenge- Yeah

in lots of organizations still.

Tom Crinnegan: Usage-based billing, as you said, that is, that is going to introduce in most companies potentially another layer of, like, the sort of haves and have-nots, right? Around like, can you justify the can you justify what, what you wanna use this tool for? Can you sort of show the value?

But yeah, I think from the early days of sort of deploying the licenses and getting nodes into the hands of people yeah, we spent a lot of time on... You know, we had things like, you know, mandatory training courses for people and before they could get access to the product, you know, that, that really wasn't about value.

Like, it was about... Because, because it was the early days of AI and, you know, lots of people were getting access to tools and didn't necessarily know the sort of guardrails they should be thinking about, so we spent a lot of time sort of educating people on the sort of guardrails and, you know, how existing you know, controls in SharePoint and OneDrive were, were key, right, to the way that Copilot could behave on your, on your behalf.

And then, you know, with the emergence of things like, you know, the ability to enable people for Frontier access, that was when, that was when we sort of went back out to some of our key sort of stakeholders in, you know, whether it's, like, the enablement teams for, for Copilot and AI in different regions, plus sort of data protection and risk teams, and we sort of said, "Well, we, we can give people access to these features."

We, we, we, we don't wanna have tens of thousands of people with access to Frontier because, you know, often those features are kind of very raw You know, we might find that they land in our environment and they don't work right because they're maybe they're requiring like a, a really open configuration in the Power Platform that, that j- we just don't have, right?

W- we've got- Yeah ... pretty strict controls. So, so we've, we've kept that group pretty small because, A, we, we say to people up front, we, we require them to complete a form. We say to them like, "We don't provide formal support for these features," right? We're not... That there isn't a service desk for these features, right?

This is about you being able to help us understand them from a business context, and also you being able to sort of prepare your business potentially for some of the bigger changes that are coming in. So there's a bit of a sort of a reciprocal deal that we have with people. But we but yeah, we do struggle at times when there are issues with those features if people have started embedding them into their ways of working.

Tom Arbuthnot: It does make it some- Well, and that's part of the deal with Frontier, isn't it? It's, it's these are early access. This is subject to change. And it's all well and good telling people that, but they, they might get hooked on a Frontier feature, and sometimes Frontier goes on for months and months and months.

And it's suddenly like, oh, actually now that's going to consumption, or that's changing, or it's going to this product stack. And you have to go back to those early access users and be like, "Sorry, we're moving your cheese a little bit," 'cause that's, that's how the world works at the moment.

Tom Crinnegan: Yeah, for sure.

For sure. There's even things in it now that I sort of look at and I go, "Is that going to ever reach GA?" And that was the other thing we've, we've sort of had to pr- probably warn people about a bit more as, as, as the Frontier programs evolve. We've sort of said to people like, "Don't, don't get too wedded to these features because they may never reach GA."

Tom Arbuthnot: They may- Yeah ...

Tom Crinnegan: they may disappear. I think I saw someone on LinkedIn actually last week talking about- I think it was the eq- was it the formula in, in Excel, like equals Copilot? Oh, yeah. The

Tom Arbuthnot: Copilot equals or whatever

Tom Crinnegan: it was, yeah. Yeah. I, I s- I saw someone sort of saying, "Oh, you know, I, I was using that in all these kinds of business processes," and, and it was like, yeah, but that...

It was Frontier, and it was clearly sort of labeled as such. And you, you do have to be quite careful not to, not to get too,

Tom Arbuthnot: It's kind of a challenging thing for Microsoft as well, because they're using Frontier to gauge interest levels. And it's like, well- Yeah ... if people are nervous to get committed to it because they don't know if it's gonna come to GA or not- Yeah

I guess they've got a big enough sample size that they can tell what the percentage is that it's a goer potentially.

Tom Crinnegan: I think they, yeah, I think they do. I think they, you know, between sort of customers of our size that might have sort of a few hundred people, if you scale that across, you know, I think they, they end up with sort of enough yeah, enough, enough telemetry.

But even telemetry, you know, is difficult. We, we struggle with this even today, right? If we're, if we're trying to... If we wanna provide feedback on something, Microsoft want the logs, they want the prompt flows, and we're sort of sitting there going, "Well, we actually..." Like, we don't allow people to send feedback, including their prompts, right?

Like, as you can imagine, right, it's, you know, I think most pr- companies would probably have a similar posture. So it can be really tricky, 'cause then we're sort of saying to someone, "Well, can you try and replicate the experience you received, but without any EY data?" Right? So, like, turn off Work IQ, you know, just use sort of web data and- Can be, yeah.

It can be pretty difficult. So, so we, we try and sometimes in my team we try and sort of replicate something that someone's seeing, and we, and we almost do it on their behalf if we think we can do it quickly, and it's a-

Tom Arbuthnot: Yeah ...

Tom Crinnegan: repeatable kind of issue or yeah, I mean, some of them are bugs, some of them are, some of them are just, you know, the nature of LLMs and how they might- H-

Tom Arbuthnot: how do you handle the user comms with those people?

'Cause, like, is that all part of your team? Is there a different comms team? What does that look like?

Tom Crinnegan: Yeah. So for Frontier, we manage it all within all within my team. So we, we... It's pretty straightforward. We have a sort of sign-up form that, again, we don't e- we don't even advertise, right? We sort of keep it we, we, we kind of bring people in if they find us, which is n- which has worked quite well, right?

If people are really trying to explore something, they normally can find their way to my team. Yeah, we-

Tom Arbuthnot: Yeah, that kind of sets the de facto bar of, like, they're interested enough to poke- It does ... IT for, like, I'd like to play.

Tom Crinnegan: Yeah. Which is, you know, might sound a bit, might sound like we're trying to sort of operate as a bit of a back office team, but we, we are pretty...

Yeah, I've got, like, a guy in my team who hosts you know, like a What's New and Next session every week with, yeah, with John Moore, another one of your fellow MVPs who, I mean, do a great job on it, right? I think we, the one yesterday had sort of, I don't know, anywhere from 7 to 8,000 people that joined us.

So we are quite- Amazing ... for GA features, we're quite public internally, if that makes sense. So we do try and be quite visible. Like, we've done that very intentionally. But yeah, for Frontier, we keep it, yeah, we keep it in, in the teams channel and we, yeah, we don't... Sometimes people sort of request access to it, and we just ignore the requests and-

Tom Arbuthnot: Yeah

Tom Crinnegan: sort of funnel them back into the sort of request process. But yeah, so we keep that very sort of separate. But then for GA features, you know, we do have... We're, we're pretty lucky. We have a service adoption team internally who have, you know, people that can write content with us. So we sort of co-author a lot of that content together.

Again, we're trying to find the right balance there between- 'Cause what we've realized is we don't, we don't want most of that team to actually have all the, all the latest and greatest 'cause it actually, I'm sure you find this in your role sometimes. You'll, you'll see... You'll, you'll be using something and then you're like, "Oh, that person next to me hasn't got it."

And it, it's, it's hard to keep track of like- Yeah ... who's in Frontier and

Tom Arbuthnot: what's not. Yeah. So. It really, really is. Yeah. It's hard, again, harder than it ever was and like, like, yes, 'cause we do lots of news groups, and podcasts, and content. You're like, "Ooh, am I allowed to talk about that thing yet? Hold on."

Tom Crinnegan: 100%. Yeah. Yeah. Oh, did we sign up to that private preview, and we can't talk about that publicly? So yeah, we- Yeah ... we have to be, we have to be quite careful with that as well. But yeah, we keep the sort of adoption teams on, generally on the sort of GA sort of track, and then what we normally try and do is, you know, give people like give people their own account that is maybe in Frontier that's just like a service account, like a non-HR account that has its own virtual machine.

Tom Arbuthnot: So we can sort of- Yeah ... they can kind of- Got that. Like, like a- ... jump ... when it hits- Yeah ... your GA j- account, that's when you can talk about it 'cause it by definition has rolled out- Yeah ... to the GA audience. Yeah,

Tom Crinnegan: exactly. Yeah. So that, that's the way we, that's the way we try and manage it internally. But yeah, we, we, again, we're pretty lucky 'cause we've got, you know, we've got a strong, sort of a strong team there.

We have an internal portal we maintain. You know, it's not easy though, right? It's, it's very difficult. We've been experimenting, you know, recently, a guy on my team's been experimenting with Cowork with how can we, how can we sort of use Cowork plus a couple of skills to actually create videos that are not- They're not traditional sort of screen recording r- videos, right?

They'll- We'll just take maybe, like, 15, 20 screenshots of something. So very, very quickly take 15, 20 screenshots of, say, how to use something, and then we just throw those screenshots at Cowork with the skill, and we just say, you know, "Create the video." And then it's pretty surprising what it does. I mean, the quality's actually far better than I would've expected.

So we've been, we've been experimenting with that a bit recently 'cause, you know, the, the rate of change is so, is so rapid that ideally we don't wanna be having to constantly sort of re-record videos. So if we can just, you know, swap out a screenshot, throw the new one in and get Copilot- Yeah ... to rebuild it or Cowork to rebuild it, it's it can speed things up quite a bit.

Tom Arbuthnot: I wonder if we'll hit a point where with either the, the public Microsoft knowledge or more likely for someone of your size, like an internal knowledge base, because it will take account for we've enabled this feature, not this feature, or we wanna point people in this direction, not that direction.

Does that knowledge base itself become... To have an agent in front of it, it's almost like a Copilot coach- Yeah ... but it's not pulling from the generic internet 'cause that's not very helpful for you because it will say, "Do it this way," but actually inside of EY we're, EY we're, we're nudging people that way.

Tom Crinnegan: Yeah. Yeah. We, I mean, we've experimented a bit with... We've actually got... Yeah, we, we, we experimented quite a bit actually with to the point where we've kind of got something that we use internally for Viva Engage communities, 'cause that was where we were struggling a lot, right? With just so many questions, and when you've- Yeah

got a community that has, you know, the, basically almost the entire company in it, the, that's licensed, the range of the questions you, you get goes from-

Tom Arbuthnot: Yeah ... you know, "Oh, someone told me that there's this thing called the

Tom Crinnegan: researcher agent," right? Like right from that sort of level right through to someone that's, you know, been reading the, the sort of-

Tom Arbuthnot: Yeah.

They've just, they've just heard Satya say something on stage and like, "I wanna build a, an autopilot." So like, "

Tom Crinnegan: Well, well," Yeah. Or Anthropic released a model and it was announced on a blog post three minutes ago, and someone's already-

Tom Arbuthnot: Yeah, yeah ... like, "I need one." Yeah. You're the experts, right? What, what's, what, how do I get this?

Tom Crinnegan: Yeah. So we've, we, and we've, we've experimented quite a lot with that, and we've, we've got that to a pretty, a place we're pretty happy with, right? Where we actually have an agent that I think's actually using... This is where it's interesting, right? We had, we didn't actually need an agent for that. I think, I think the, in the end, we built like a and I'm not gonna do this justice 'cause the guy on my team who supports sort of more the technical SME on this build, I think he's actually using like Power Automate With some of the sort of AI workflows you can leverage there, right?

Like AI Builder to, to kind of reply to the posts. We, we found that we ha- we could get, like, really good control over that. We could sort of customize the, the response to the way that the response looked. Yeah, we could set you know, the sen- the sort of sensitivity level around, like, how confident we were with the answer.

And if we're not confident, you know, we, we sort of then funnel that off somewhere else so that we can review like, oh, here are the 15 things we got asked this month that we had, like, really low confidence with. Then we sort of figure out, well, do we need to update the knowledge source?

Tom Arbuthnot: Yeah. How do we refi- Yeah

I love that idea of refining the experience of, like, this wasn't- Yeah ... the knowledge base didn't have a great answer for this. It's gonna come up more than once. Like, rather than answering once and losing that data, let's refine the knowledge base.

Tom Crinnegan: Yeah. I, we talk about that quite a lot in my team around, yeah, because we get, like, I, like, email is, is the, is the sort of death of knowledge sharing.

'Cause you get this email from someone, super deep, sort of some- something really specific around something they're trying to do, and then you reply to them and you think, "Oh, that's just locked away in that person's mailbox and in my mailbox. It's absolutely hopeless." So we've been looking recently at how do we sort of do something there with automation across our team?

Because I think all of my team, and probably colleagues of mine in the Power Platform world and, and, you know, in more of the pro dev world, we're probably all getting these queries all the time. We're probably all answering them in slightly different ways. Like, how do we build something almost like a s- something that all of us contribute to on, you know, even if it's something we look at sort of weekly or, or monthly.

Tom Arbuthnot: Like, very- Yeah. And if you can do that in Teams or Viva Engage, you're signaling to the Work IQ database layer, you know, the, the, like, this is shareable information, so actually you're building up the work IQ layer, which is a ben- a benefit.

Tom Crinnegan: Yeah. Yeah. 'Cause I think I've seen, I think I've, I think one of your other podcasts in the past, I think I was, we were playing and it was talking about, you know, Teams chats versus channels, 'cause that's the bane of my life as well, the number of times I'm sort of like, "Oh, this should really be in a channel that all this other, all these other people have got access to and

Tom Arbuthnot: not- I'm, I'm ha- I'm harboring a secret hope that Chat- this will be the thing that persuades business users why channels, because they want all the information in one place in an accessible way.

And like, it, the, the benefit of channels has always been for somebody else, not for you. It's been the, the next person to join the project will inherit the benefit, but in the short term, chat is the muscle memory. Whereas now, if I can go to an agent over a project and know that every meeting, every chat, everything, it is in one place-

me as an individual, I'm benefiting 'cause I can ask questions of the project corpus of information. Yeah. So I'm, I'm... Yeah. There's a lot of work going on at Microsoft at the moment, y- I'm sure you know as well, like, to try and cement Teams and channels as an excellent experience for kind of an agentic data repository.

Tom Crinnegan: Yeah. Yeah. And we've been using it a lot more internally recently. You know, I was... 'Cause I've got colleagues that work, you know, across, obviously there's, like, people that kind of look after SharePoint and OneDrive and, and Exchange, right? 'Cause of our size and scale, you know, we've got sort of dedicated teams for most of those sort of product areas 'cause of, because of the level of sort of complexity that we deal with.

Tom Arbuthnot: H- how has that been for your team? Because your product, quote unquote, touches every surface in the M365 stack. I mean, not even- Yeah ... not even potentially, literally does. Like, there's nothing doesn't have a touch- Yeah, it does ... of Copilot in there.

Tom Crinnegan: I mean, so we, like, we, I mean, the g- the good thing is, like, so I mean, we work with, with every single one of those teams.

We've actually sort of got to a, a model where we tr- we try and do as much as we can asynchronously, right? 'Cause otherwise you're just in meetings all the time. So we, we've, yeah, we've got, like, channels with every team. So I've got my... I mean, the trouble is my l- my team's left rail is becoming so busy 'cause it's now just full of

It's just got almost now too, too many different categories of things. But yeah, I've got like a Copilot BAU sort of grouping, and then within there we've got channels for pretty much everything, like, like Copilot and SharePoint. We've got one around like agent governance, right? 'Cause agent governance straddles even broader groups than just the traditional sort of M365 product stack, right?

You've got, like, Power Platform coming into there. You've got InfoSec. You've got, yeah, lots of other teams that have different roles to play there. So but yeah, we, we, we, you know, we run things like- We run, like, fortnightly feature update calls, which is, you know, targeted at sort of risk and data protection type communities and A- AI enablement teams in all of our regions and our service lines.

And we're very specific. Like, it's, it's, you know, it's, it's not like Tom and his team that do all the work. We bring in the SharePoint team to come and talk about, say, Copilot and SharePoint and what's evolving there. So we bring them in. We, we just, we, we sort of perform a, perform a bit of a role as, I guess, an aggregator of...

So, so that- Nice ... people don't have to know who to talk to. They just know they can come to this forum, and if there's a question about, you know, basically anything in Copilot if my team don't know it, we, you know, we can funnel them to the right, to the right group. So, you know, even, like, even down to, like, Copilot and PowerPoint, you know, some of my colleagues have been doing loads of work recently with our brand kit, so that's not me and my team leading it.

But then when we wanna talk about it, we will bring that topic into the sort of live event that Lee and my team runs. Yeah. And we'll bring on, like, a guest speaker from that team to talk about it. So yeah, we sort of try and sit, like, above and, and generally we are across pretty much everything that's going on because, you know, if we're not, then someone will ask us and we won't even know where to funnel it to, so but yeah, it's diff- it is definitely difficult but- You-

Tom Arbuthnot: you're in a challenging posit- position from a coordination point of view, but kind of a privileged position from a you have SMEs in, in lots of different areas to lean on, collaborate with, I guess.

Tom Crinnegan: 100%. Yeah. It's not like I'm having to sort of go into the weeds of something in SharePoint. It... I, I know that the team would already be looking at it and, you know, there was something the other day I think that came in around- Is it authoritative sources, you know, based on like SharePoint? You know, you can sort of list authoritative sources for Copilot to kind of rely on and, and you know, I pinged the team and said, "Oh, what, what have you guys been thinking about this?"

Right. And they, they're already thinking about it. We're talking- Yeah ... to like our brand marketing team. So it's, yeah, that, that having those relationships with those teams is, is critical 'cause we, you know, my team would not be able to scale to look at absolutely everything that comes in. And I think we, we've talked about this before, right?

Like, there are some organizations just probably do not have the level of resourcing needed to kind of triage everything that's coming

Tom Arbuthnot: in. No, no. I'm talking to plenty of organizations, and not even small organizations that are, you know at the 20,000, 30,000 Copilots each level. Yeah. And the service has kind of just been rolled into the M365 service owner.

Yeah. This is a big business transformation. Like, like, like, like M365 was already a big job, and now they're like, "And Copilot." It's like that is a, a, I, I don't... Really challenging.

Tom Crinnegan: It is, yeah. I mean, I'm like, I mean, sort of, we've got... There's, there's a guy in my team who we, we built like a homegrown sort of app actually to manage that level of complexity because, because there were so many people often looking at things, we, we look...

Th- and this actually predated me. This wasn't my idea. This, I'm not, I can't take any credit for this. This, this was something that sort of got moved into my team in the last year or so. But we, yeah, I mean, we've got a Power App that, that aggregates everything because often something that comes in will actually be maybe like my team needs to look at it, but maybe the Power Platform team have an opinion here.

So we kind of bring in, you know, Message Center, Roadmap. We're even bringing in like blog posts because sometimes things get announced, and it's on a blog post, and there's not- Yeah ... a Message Center post for it. So we bring it all in, and then we can see, like I can see that we have decided to maybe out scope this and not look at it because another team is already looking into it.

So, so yeah. We but yeah, if you're just sort of a person that's traditionally looked after SharePoint, OneDrive, you know, Teams, and Exchange, and then you've suddenly got this incredibly fast-moving thing called Copilot coming in- It's, yeah, it's, it is like, I, I always describe it as like drinking sort of from a fire hose.

Yeah. It is just... And, and you have to, you have to really, I think, decide where you spend the time and, the time and energy, right? And where you, and where you don't. If you investigate it, absolutely everything that comes through, even some of the sort of minor s- sort of minor user features, we often look at them and go, "You know what?

That's not even worth telling people about," because it's so, you know, some, it updates, like, the way that chat history looks. We're like, "Okay, descope it. Let's move on 'cause we've got- Yeah ... 20 other things to look at." So, yeah.

Tom Arbuthnot: How's it been for you like reporting on and understanding usage and, and I guess extension to that question, now usage-based billing, how are you thinking-

about that? 'Cause that's s- sup- huge change for Knowledge Work, I think.

Tom Crinnegan: Yeah. We spend, we spend... There's a g- there's a guy in my team, Simon, who leads and because it's such a big area we, you know, Simon kind of leads on it and spends, like, a lot of his week on that topic. I'm,

Tom Arbuthnot: I'm mentally logging here, Tom, we're gonna end up doing a podcast where we bring half your team on 'cause it's gonna be- Yeah

it'll be really interesting to, to query some of the stuff.

Tom Crinnegan: Yeah, no, potentially, yeah, 'cause we've got, you know, I, I, again, I, I sort of, I'm repeating myself, but I am lucky 'cause I've got people that are kind of really good with, like, content generation and can work with our service adoption team. I've got, like, tech- like, really deep technical minded people like Emiliano in my team, who's got like a really deep background in the M365 stack.

So if something new comes into Copilot, he doesn't have to educate himself on how that might integrate into something 'cause he, he, he knows the stack really well. Yeah. So, so yeah, I mean, analytics-wise, we, I mean, we've always... Because there's, there's always sort of a data privacy and a data protection angle to this conversation, right, as well around, well, who, you know, who should be able to see how much someone is using a tool.

So we've built yeah, we, we've, we've taken all the, and, you know, this, this, over time this has moved to sort of API ingestion. In the early days it was pretty horrible, right? It was like exporting-

Tom Arbuthnot: Yeah, export from the portal in CSV.

Tom Crinnegan: Yeah ... export portal files and all, all that kind of horrendousness. So yeah, we, we, we basically created sort of internal Power BI reports that had, you know, role-based access control to, like, right, this person needs access to, say, the data in one country for this, you know, say on this line, like, for tax in this country.

So, so we ended up with a fairly complex sort of web of permissioning there. But yeah, that, that was what we needed to do to make sure that we could give the people access to what they were basically allowed to, allowed to access. So we had things like, you know, identifying, I guess, power users for champions networks, all the, all, all the sort of traditional change management stuff you'd, you'd probably expect.

And then we had things like... W- what we tr- we also tried to add different layers on top of what Microsoft had, 'cause what we wanted to understand was, were things like- More like trends, so things like habitual use. So we started to define some KPIs around, like, what, what does habitual use mean? You know, is it...

A- and we saw them obviously steadily increasing, and then we saw, I think as the mo- as, as some of the models started to improve as well, people were like, "Wow, this is actually...". Like, Copilot just started to dramatically improve as the models improved dramatically. And so then we started to see, like, ma- massive increases in habitual use 'cause people were realizing, "I...

Wow, I can actually really, really get, like, that PowerPoint file generated or that Excel spreadsheet sort of data crunched in that," and then

Tom Arbuthnot: sort of- And that's, that's an adoption challenge, isn't it? 'Cause it was, I think it's fair to say Microsoft were early to this. The models were new. Like, like, it was a growth journey, and a lot of people tried, and they're busy doing their day job, and getting them to come back a second and a third time, be like, "Well, it's very different.

We were on Sonnet 4, and now we're on 4.5," and all, like, like... And the latest models, Microsoft got a lot quicker- Yeah ... at getting the models into production and more or less neck and neck with the, the-

Tom Crinnegan: Yeah ...

Tom Arbuthnot: other tools out there. But yeah, I, I've seen the same thing. Like, I've had a lot of customers where we've been like, "It's worth a revisit now.

Like, this is a very different story to two years ago."

Tom Crinnegan: Yeah. I think Copilot in Excel demonstrates that the best. I, I, I think- When I have... Yeah, we, I, I was doing something a few, probably about a few months ago now with Excel, and I was having to, like, divide up, you know, the entire user base. I was trying to how...

think about how we were gonna deploy stuff in tranches, and it would've ta- it definitely would've taken me ages to even think about how I was gonna approach it to sort of- Yeah ... balance load, and I, I, I threw it into, yeah, Copilot in Excel, and I, I can vividly remember when it first came out, it was, it really struggled to do really anything of much sort of- Yeah

Tom Arbuthnot: value and use, right? It was at the moment of like, "I can't do it, but I can tell you how to do it." It's like, great.

Tom Crinnegan: Yeah. Yeah, I can give you the, I can give you the- Yeah ... here are the 50 steps to performing, thinking, "Oh my goodness, that's not what I want." But I know, I know the team spent, like, a lot of effort sort of basically rebuilding it from the ground up, and it is incredible what it does now to the point where I'll ask it to do something, and it will sort of come back and give me, like, four options, and I'll be like, "I thought of two of those."

Two of those- Yeah ... I hadn't even, I hadn't even sort of thought of, and actually that, that one that you're recommending is actually by far, yeah, far better than the way I was approaching it. So that, that rolls into our training approach as well, 'cause now we're... And we don't do lots of training. It's not like we're running, you know, hundreds of training sessions a week, right?

But we do try and... We've, we've got there, there's a new woman that joined the team that leads on the training, and she's doing a cracking job at just, like, looking at, like, where we should bother spending the time. And then one of the things we've also been trying to do is bring in, like, subject matter experts.

Like, because of how big EY is, right, we've got like deep SMEs in Excel. You know, people who are, like, at, like, Excel world championships, you know, like, competing, right, i- in this sort of world and, and so we're trying to bring people like that who... into help with our training content because- Yeah ... who better to hear from than, you know, someone that knows Excel inside out?

And when they're saying, "Hey, Copilot in Excel

Tom Arbuthnot: is fantastic"- Yeah, this is worth a shot. Yeah. Like, I'm, like, like, I I'm quite literally a world-class expert in this, and I think it's useful. Yeah, I'm literally- That is, that is quite a blessing ... I've literally won

Tom Crinnegan: awards.

Tom Arbuthnot: Amazing. Yeah.

Tom Crinnegan: But I think, but I think your question was probably about usage-based billing as well, wasn't it?

Tom Arbuthnot: 'Cause that's Yeah. I'm super interested in how you're thinking about this 'cause it's just, it's- Yeah ... it's a change for everybody, and you mentioned about RBAC roles and visibility. Like, I'm having a lot of conversations at the moment about this is an IT and business conversation, and the business should own the usage-based budget.

But the business is gonna come back to IT or, like, your team and say, "What are you recommending here? What's good? What's bad?" Or how are you thinking about that? Yeah.

Tom Crinnegan: I think we're, because we yeah, because there's other sort of gen AI tools internally that are in use that are already usage-based, I think we're probably a bit ahead and w- and I'm sort of fortunate in that respect 'cause we've got...

So our, our process is very much- Requiring people to, you know, request what they need, but then tying it back to the lowest level of sort of budgetary control that we can, right?. So down to, like, we, you know, we use internal charge codes and things for people's time. Like, even for me as an internal sort of resource who doesn't charge their time to clients, like, you know, I charge my time to, to sort of like a sort of a code internally.

So, so that's where we require people to,

Tom Arbuthnot: Yeah. So actually you've got a cultural benefit there that you already have a concept because you bill clients in a kind of client coded type way. Yes. They're used to it. That, that's not, that's not, not normal outside of legal and pro services. So that's inter- I hadn't made that- Exactly

leap before of, like, maybe some, some industries are better suited to this concept of granular cost models on the business side.

Tom Crinnegan: Yeah. Yeah. I mean, peop- like, I guess, like, you know, the traditional model in other companies might be more, you know, submit your expenses, right? And, and sort of... But even then, normally when you log into those systems, right, it's probably pre-populated for you.

It's got your cost center. So, but I mean, that, that I think to me is the, you know, that's the model that allows both the user t- it, it kind of allows the user to justify to the right person, to the right decision-maker, like, what they need this for and why. So we, you know, we've worked on sort of some...

And, and different, yeah, different parts of the firm are sort of approaching this obviously in different ways, as you can imagine. But yeah, I think that's the key for us in sort of technology roles is making sure that we... And, and this again, I'm lucky that there is a team that built all this tooling, right?

That we're just leveraging, we're just onboarding Cowork onto it. So there's some great we, you know, we've got a whole internal, like, marketplace of, of tech services internally that has been, you know, around for a long time. You know, you can order, na- name the product or service, you can, you can go and order it from there.

So we've onboarded sort of Cowork into their ecosystem. So, but yeah, I mean, the reporting side is still a bit of a pain because the, the, the sort of API sort of level reporting is not, not quite where we need it to be. So we're even the pilot we ran, you know, we were having to, you know, we, we, we built something in Power BI, but people were like, "Oh, but it's two days out of date."

And so yeah, we were like, "Yeah, use forward slash cost, like, to see..." And obviously Microsoft are putting more features into the app itself.

Tom Arbuthnot: Yeah. And this is classic Microsoft, isn't it? As we said at the start. Like, it like, it starts off in portal. We've been doing some horrible stuff of, like, taking the web token and scraping the portals and things.

But, like, like, hopefully, I mean I'd say because of the, the need for this, it'll come to API coverage, and we'll be able to tidy it up hopefully.

Tom Crinnegan: Yeah. Yeah, for sure. Yeah. We, yeah, we're fully expecting that to come, and then we'll be able to sort of give people the, the level of insights they need. I think the, the big challenge is and I think we, there was actually a report that came out this week, and I wouldn't normally do the sort of a plug for, for an EY report 'cause it's not usually something that I get that involved in.

But there was a report that, that is, is public, so I can speak about it. Nice. I will

Tom Arbuthnot: link it in the description.

Tom Crinnegan: Yeah, and I'll, I'll, I'll grab that and send it over. It's the- It's our EY Parthenon team who do, like, a CEO survey. I can't remember how often they do it, but they do it sort of on a, on a regular basis, and they...

And you can imagine AI is, like, a top topic in, in that these days. Like, it was probably, I don't know, s- I think when I skimmed through it, it was probably, like, 30% or 40% of the whole report talked about, sort of talked about AI. But, but it was, it was really interesting 'cause it was talking about it, like, I think a lot of CEOs, and it's, like, 1,200 CEOs from different companies across the world, like, you know, generally quite large companies and, and they...

The, the general consensus was, like, lots of these CEOs were saying, yes, they're seeing benefits and, and some of, some of them have got, like, insights into ROI. But, like, the big bulk was, like, some, whereas the, I think it was, like, 16% were, were saying they've got, like, really great control over all the spend and really great insights around R-ROI, right?

Which is pre- the 16% isn't, isn't sort of that big and the, and the biggest chunk was that, "Yeah, we- we've got visibility, but we, but the ROI stuff is sort of the hardest piece." So, and I think that's the key. It's, like, under- and that's why I think tying the request for usage-based billing services back to the, literally the person that is governing the work you're doing is the key, right?

'Cause otherwise, how can you s- Yeah ... how can you provide with value is-

Tom Arbuthnot: We, we, we were talking on our, our last session around, like, I, I think it's an imperfect metaphor, but I equate it to managing people in the sense of, like-

Tom Crinnegan: Yeah ...

Tom Arbuthnot: you've got the, the French marketing team, and there's five people in that team, and they've got a manager.

And then eventually that goes up to country marketing, global marketing, CVP of marketing. But, like, it's, it's gonna be the manager of those five people that knows whether $1,000 spent on Co- on Cowork is the best spend ever because they got the product- Yeah ... out fast and they delivered the, the requirement, or it was just someone checking their email every hour and summarizing it and, like, not a great spend.

And- Yeah ... it's very hard as you scale up to get that understanding and, and it kind of poses a challenge of the, the manager needs to know, but also you as a, the person doing adoption globally would kind of like to know that the French marketing team smashed it and are getting loads of value out of Cowork.

Tom Crinnegan: Yeah. And I think that's the, that, that's the thing we're seeing. 'cause it's interesting even that example you said around, like, someone's mailbox, right? Because we've, we've, we've had that conversation internally where I was like, "Well, actually, if you were, let's say you were-" I don't know, c- responding to a bunch of RFPs for a bunch of clients, right, and trying to w-win work.

Well, if freeing up your mailbox using, say, Cowork, right, just to kind of remove, you know, 60, 70% of it, even if it's, like, cleaning stuff out, drafting responses for you. Like, if that, if that freed you up, if that allowed you more time in your week to respond to more RFPs, or basically be a bit more effective and productive because you don't burn sort of,

You're not, you're not stressed out from just the deluge- Yeah ... of mail that's coming into you. Like, maybe that is a good investment for someone who- I-I-It- Yeah ...

Tom Arbuthnot: it could be the best, the best $30 or $40 ever spent. Like, if you're- It could, yeah ... if you, if you're the right level in the org or the right, or have the right impact, 100%.

That's why it's so hard for IT to judge because-

Tom Crinnegan: Yeah ...

Tom Arbuthnot: it, it... I 100% agree. And, and again, kind of this bad metaphor, like, a certain tier of person in organizations get an EA or a PA. Like, that's an FTE whose job is to help that person perform. We potentially get to democratize some of that power but the CFO is sitting there going, "Well, I haven't budgeted for everybody sort of like to have $100 a month of consumption or hundreds of dollars a month of consumption."

Tom Crinnegan: Yeah. Yeah. No, it's, it's, it's def- definitely disruptive. And then when you think about, you know, things like Copilot Code coming along, and you've got this real, I think, blurring of- Like who, like people that would traditionally have used like a CLI to sort of do things. It feels like you're getting more people that are just not from a dev background and are like, "Oh, yeah, maybe, maybe I should give this sort of CLI type approach a try."

And then Copilot Code, I think, is trying to sort of bridge that gap a bit by giving a bit more of a user-friendly interface for people that ma- And, you know, like Lovable and other sort of consumer services obviously have done this really well, right? Where people can kind of build, sort of build apps themselves and I don't know, probably still at the, the jury I think is still probably out around, like can you then actually really launch that, scale it, manage it, evolve it, maintain it in a, like in a, in a reasonable way.

Tom Arbuthnot: There was... I was... Was it an article the other day?

Tom Crinnegan: It, it was a LinkedIn post. I, I'm not gonna... I can't remember who it was, but they were talking about, yeah, these, all these like mini businesses that are spinning up that, where their, their entire model is like, "Oh, if you've built something that's actually doing okay, but now you're realizing you can't manage it, we'll come and manage it for you.

Like, we'll actually bring some of the pro dev

Tom Arbuthnot: skill set to you." Interesting. So like a, yeah, the, the b- and I imagine part of their pitch there is y- your corporate IT might not be interested, but your head of marketing and you know it's a big deal, do you just wanna sub us in to, like-

Tom Crinnegan: Yeah,

Tom Arbuthnot: yeah

Tom Crinnegan: bridge the gap kind of thing. Or even like small startups that are like, you know, s- that came from someone who had used, you know, a vibe coding app to kind of build something, and they've actually, you know, made some sales. They've actually got a bit of revenue- Yeah ... in, but now they're looking at it and they're going, "Well, I, like this vibe coding stuff is not gonna get me all the things- Yeah

I now need to do." They're kind of coming in to try and bolt themselves onto-

Tom Arbuthnot: Interesting ...

Tom Crinnegan: sort of small businesses. Yeah, so I mean, there's a whole ecosystem, right, of companies that seem to

Tom Arbuthnot: be- So will there be that equivalent ecosystem inside of organizations where you're kind of like a COE or a tiger team for like...

A bit like we saw on Power Platform, I guess. Like, like, like the business can get so far, and then they need the support of that kind of dev-esque or proper dev understanding of how to architect things.

Tom Crinnegan: Yeah, exactly. I think that's the key. It's the sort of architecture piece because especially in sort of big, you know, big corporates, right?

The, the architecture piece, if you, if you start trying to do that when you're too far down the line, yeah, you'll realize pretty quickly, oh, wow, we really wouldn't have designed it- Yeah ... this way, and now we've got all this, this sort of legacy or, or this sort of tech debt that we now need to deal with. So and obviously we've got, you know, there's like tens of thousands of people internally that sort of help people with that.

So yeah, we've got like whole, whole functions to help people with that. But it's, but no, there, there are still lots of people that are trying to experiment, right? At, who generally maybe have never built an app in their life or never, you know, never like logged into sort of the Power Platform or tried to even create, you know, like a flow in Power Automate or something.

And now they're, they're sort of having a go at, you know, Copilot Studio and building sort of an agent in there. And we definitely see that, you know, some people just get to a point where they can't quite sort of make that step to get it ready for production. Yeah. Or when the governance sort of world comes along, and they have to justify how they're running evals for their agent, and pe- and they're, they're like, "Evals?

What do you mean?" Like the, they're, they're just, they conceptually just don't, don't fully understand that they're, they're... I mean, they're build- They're essentially building an app, right? It just happens- Yeah ... to be an AI-enabled sort of app. And, and so like maintaining it and support model and like how are you, you know, regression testing this and like, you know, how are you thinking about model drift and all this kind of stuff.

What are you gonna do when a new model comes out? It's, you know, some of these teams then do need that support to, to help get it to production. So yeah, it's quite common.

Tom Arbuthnot: So are you finding like with code coming in, like y- Power Platform becomes another one of those teams that you'll end up working alongside with?

'Cause that's, that, those worlds feel like they're coming quite... Well, they are coming together, aren't they? We've seen it inside of Microsoft. The Dynamics team and the M365 team are very close now.

Tom Crinnegan: Yeah, they are. I know. Yeah, I mean, I feel, I feel like I've spent more time in the last probably six months working with yeah, working with some of my colleagues in the Power Platform team, and then, and then the Dynamics sort of world as well than I have sort of in the prior three sort of years of the Copilot- Yeah

journey. Yeah. 'Cause you know, as you say, right, it's, it's storage, it's like it, it's where these kind of things are gonna, are gonna live, where they're gonna be stored, where the data's gonna be processed right through to like, right through from that through to like where you allocate Copilot credits, right?

Because today it's kind of split between different portals and, you know, that, that, that, that's all sort of changing. And so we're- Yeah, we're, we're working very closely together 'cause... And in fact, I think the, the call I was on before, before the session today was with was with the, the sort of plat- platform team where we were talking about you know, like some of the, some of the A365 stuff that we've been looking, some of the newer things in A365 that are coming along that we need to sort of think about how we're gonna use.

So, so yeah, they, they, they have to be like cross-team conversations, whereas tr- traditionally you could sort of probably kind of separate that off, like Power Platform was kind of, it was its own sort of beast and, oh, you might publish a, something back into the Microsoft ecosystem, but a user probably wouldn't know, and they wouldn't have to know.

Whereas now that I think is completely-

Tom Arbuthnot: Yeah. And that usage-based billing conversation again is like, Power Platform used to be very seat based. Like you would build and then you would have a seat cost for the users using it, broadly speaking. And now that's gone more usage based both for build and use.

So you've got- Yeah ... like a, again, I'll use the marketing team, like we've built an app and actually there's consumption for using that app on an ongoing basis, so you factor that into your budget.

Tom Crinnegan: Yeah. E- even things like, you know, the, like I think the GitHub Harness sort of when that was brought into the Copilot studio world, I mean, that was like, and may- maybe this is my memory, but I feel like that, that was announced and then happened within Two days or something, and it sort of- Yeah,

Tom Arbuthnot: it was very fast.

Tom Crinnegan: It was extremely quick, right? And then you sort of... I mean, you're trying to help educate people on what does that actually mean, and people that traditionally were just able to build an agent, you know, in the, in the standard harness, and you never had to tell someone what a harness was or, or what it meant before, before, before the la- sort of, I guess it was really like Anthropic, like with Claude that kind of introduced the sort of harness concept.

I'm probably misquoting that, but I, I feel like it was really them that-

Tom Arbuthnot: No, it did. It's, it's one of those- Or- ... it's one of those terms that kind of came out of nowhere and then was everywhere. And it was like, and it was this distinction of like the, the, the experience is a combination of the model and the harness, and the harness actually became a bigger part of the conversation in some contexts as like that's- Yeah

unleashing the power of the models.

Tom Crinnegan: Yeah. So I think, yeah, we've seen people struggling with that, right, as you can imagine, 'cause they're like, "Oh, now I need to think about credits for this? But I just wanna build something." It's like, well, you need credits for, you know, for, for even sort of mocking some of this stuff up, so.

Yeah. Yeah, we're, yeah, we're figuring out how we give people... 'Cause that's what the Power Platform team have done, I think, really well in- internally, has always been, you know, letting people experiment and letting them... Because if you can't experiment, then you can't learn, right? So you kind of, i- if the door is, if the entry gate or the door is so hard to get through to even, say, like get a dev environment to kind of do something in, yeah, you, you're sort of quashing sort of innovation from the- Yeah

from the get-go. So yeah, that, that's what we're still sort of trying to figure out internally as well is, well, how do, how do we equip people and make it clear, like, what they might need, how many credits they might need to explore. But then, yeah, if you are then going sort of to production or you're trying to get something into the hands of lots of people, that's where we, yeah, then funnel them into s- the governance process and, and make sure that, yeah, they're equipped to really manage it and actually deploy it and actually govern it over time and, and deal with, you know, the inevitable, you know, "Oh, this model is actually being deprecated in two weeks," and so you need to-

Tom Arbuthnot: Yeah, yeah.

The lifecycle management or, or the, very often we've seen on the Power Platform side is, you know, the person who's innovative, impressive and builds it moves on to the next role, and now what's the lifecycle of that suddenly business-dependent app or agent?

Tom Crinnegan: Infra

Tom Arbuthnot: as a service. Yeah, exactly. Yeah. A- another fun conversation I'm having at the moment is some of these agents are gonna become business critical workflows.

Like how do IT, quote unquote, "know" that these ones are burning good credits and have to stay on because they're mission critical, and these ones are experimental and actually those can be turned off. Like all that kind of facts, this kind of governance and comms conversation gets really interesting.

Tom Crinnegan: Yeah, it does, and I think that's where you just have to... Yeah, I th- think to me, like my, my, maybe it's oversimplifying it, but my view is always you, you have to sort of... And sometimes the tooling doesn't help here, right? 'Cause sometimes you can't easily show someone, like, what their agent is. I think, and I think Microsoft are plugging some of those gaps, right?

But yeah, the ability to sort of go in, into your, say- Power Platform environment and look at, like, your top agents and what they're consuming, all those kinds of things. Like, I think the key is empowering people with that and then saying... And, and sometimes it might be a fairly blunt conversation, right? It might be, look, if it is business critical, then someone in, someone in your team needs to have that role for consumption management, like understanding- Yeah

what it's consuming, governing it, setting up alerting to sort of you know, order more credits if, if that's where you're heading towards, right? Like, do, like, look at projections, try and figure out what your trend is. Again, start small, right? Don't, don't throw this thing into the hands of, like, 20,000 people on the first day be- because you'll probably find out you'll blast through all the credit straight away because, like, three people out of that 20,000 will, you know, will go sort of wild with it.

So- Yeah ... so yeah, we, we try and give people that sort of guidance and, and try and educate them on it. But it's, yeah, it's not easy 'cause some of the people you're talking with are not, they don't, they don't sort of come from sort of tech backgrounds, so you're having to start sort of right from the very beginning with them, with some groups.

Tom Arbuthnot: But, you know. Yeah. It's gonna be in... I think, broadly speaking, I think that's true as well. Like, for the business, it's just gonna be a new conversation. I think we're gonna see a lot of the, the business trying to say, "Oh no, this sounds like IT's problem. Like, you, you guys do the IT budget for stuff.

Like, we're just gonna do the thing." It's like, no, no, this is a, this is your budget. Like, you can spend a million a year on tokens if you want. It's, it's the business. We're facilitating, governing, ensuring it's done in the right way. We're, we're advice and guidance. We're not the gatekeepers of the, the tooling in that way.

Tom Crinnegan: Yeah. Yeah. And I think because we've always had, you know, because we've always had this sort of model with services that sit outside of my team, you know, like Azure services, like pe- generally the business is very used to that, right? They are used to, you know, if you use it, you pay for it. And-

Tom Arbuthnot: Yeah ...

Tom Crinnegan: you, you can see it every month.

Your finance business partner can show you exactly what, you know, you spent money on, and then you can go into a Power BI and you can see exactly where that was spent, right? Like, you can see exactly what the costs were. So I think that level of understanding is actually pretty good. And, but that might be more in big firms, right?

They've got, like, sophisticated sort of processes and report... Do, you know, reporting by default. Whereas, you know, may- maybe it, maybe it's sort of harder in, in other businesses where the reporting stuff is, you know, a sort of a, a thing that's on the backlog that's, you know, that's hard to get to because the rate of change is

Tom Arbuthnot: coming sort of- Yeah.

I mean, look, plenty, plenty of reasonable-sized organizations don't have that chargeback model, 'cause then it's part of the kind of, again, the- Yeah ... the, the, the franchise or country model. They're just like, "Oh, IT's just a global cost." Like, we haven't... Now it's not. Yeah. Because it's gonna be one country is gonna get really progressive on AI, and they're gonna consume more, so it's gonna have to be a cross-charge, chargeback model.

Tom Crinnegan: Exactly. Yeah. Do you want everyone that's not using it to be paying for everyone that's using it? It's not, doesn't seem like a-

Tom Arbuthnot: No ...

Tom Crinnegan: particularly sort of fair model, does it? But yeah.

Tom Arbuthnot: Awesome. Well, Tom, thanks for sharing so much. It's such an interesting time. I think you're in one of the most interesting positions, both from a, a, a visibility point of view working kind of closely with Microsoft and your scale, so I appreciate you sharing so much.

Tom Crinnegan: Yeah. No problem at all. Good conversation. Thanks, Tom.

Tom Arbuthnot: Thanks a lot.