Karoliina Kettukari: Most of the use cases with M365 Copilot have, have been meetings. Of course i, I think that's the one that gives, uh, immediately like hours of savings per week. For example, today I followed three meetings, and then I just used like, uh, three minutes to read through the meeting notes. Instead of sitting in three meetings, three one hour long meetings.

Tom Arbuthnot: Hi, welcome back to the Teams Insider Podcast This week, a really great customer perspective on Copilot and Copilot adoption in a large financial organization is OP Financial Group. We have Karoliina who is head of modern work, really interesting conversation. She's also an MVP, so very, uh, switched on as to what's going on in the space and a really great perspective on the challenges and opportunities of Copilot adoption and agents.

So really hope you enjoy this conversation and many thanks to Neat who are the sponsor of this podcast, really appreciate all their support. On with the show. Hi everybody. Welcome back to the Teams Insider Podcast. Excited to have this conversation. Uh, Karoliina and I have been going back and forth for a while trying to get dates in and stuff, so it's, it's one I've wanted to have for a while.

Um, Karoliina, maybe we could start off by you introducing yourself and your role. 

Karoliina Kettukari: Cool. Thank you Tom for inviting me and I'm happy to found a time for this as well. I'm Karoliina Kettukari. I come from Finland, that's why I also have a difficult name to pronounce. Um, I'm the head of Modern Work at OB Financial Group, which is the biggest financial services group in Finland.

And basically I'm responsible for all of our modern work Teams adoption, M365, and of course, Copilot. 

Tom Arbuthnot: Awesome. And an MVP as well? 

Karoliina Kettukari: Uh, yeah. Uh, just been renewed for the sixth year. 

Tom Arbuthnot: Awesome. It's, it's rare. There's a rare breed of MVP that work customer side, so it's really interesting for me to talk to you in particular because I feel like people that work customer side, live with the decisions that they're kind of, uh, espousing and making and pushing through.

It gives you a different perspective to the, the consulting side of the fence. 

Karoliina Kettukari: Yeah, I'm a recovering consultant, so I switched from consultancy, which I did for about 10 years, uh, to to customer side about 10 months ago. So this is, uh, fairly, fairly new to me. And 

Tom Arbuthnot: how did you find the transition? 

Karoliina Kettukari: I think it was, uh, pretty, pretty smooth actually because, um, the basic things, so for example, Teams adoption, modern work, best practices, Copilot.

I've been doing all of those things, uh, as a consultant before. So basically it was more about, uh, how to create a strategy and vision for one company. And then, as you said, I have to live with my decisions, so, so we also have to execute the things we have in our roadmap. 

Tom Arbuthnot: Yeah, I think it's very easy as a consultant to come in and, uh, espouse a lot of advice and be like, this is gonna be the best thing, and then onto the next gig.

So actually living with the decisions and, and having to drive the result and not just recommend the result, I think is really interesting. 

Karoliina Kettukari: Yeah. And, uh, as a consul, as a consultant, even I was, I was never about the person who was the biggest in the hype. So I always wanted to have like concrete results.

But even though when your consultancy gee, is maybe for three months or six months, then you never maybe see the client again. So it's a very different point of view, uh, coming in as an external consultant than as an opposite to like actually living to the reality. Yeah. Inside an organization. 

Tom Arbuthnot: Awesome.

And OP financials, uh, for people who don't know it, quite a big organization, isn't it? 

Karoliina Kettukari: Yeah, we have about, uh, 20,000 employees. So basically about, uh, 13, 14,000 of our own and then about five to six, 6,000 external employees, which, uh, mostly work as part of part of our team. So, yeah, uh, quite a big, in a Finnish scale as well.

Tom Arbuthnot: Awesome. So, um, you mentioned it at the start, like modern work, Copilot. Obviously AI is the big conversation. I'd love to get your perspective on how, how you thought about that strategically, what you're doing inside the organization. Good points, bad points. Like it's, it's so again, so interesting. It's such a hot topic and particularly in financial, it's interesting because I think it's gonna be hugely disruptive to that space, but also there's all the regulation and all the considerations there.

Karoliina Kettukari: Yeah. And, um, I think we had a great start in the beginning because our CEO, our top management has always been very eager on utilizing new technologies. We have a very strong, uh, background with Microsoft. For example, we are doing cloud journeys, um, to Azure, et cetera. So we have had a very like, uh, good, good basics, which to build on.

Yeah. Then when Copilot came in, uh, 2023, almost two years ago. 

Tom Arbuthnot: Yeah. It's crazy how fast it's been. Uh, it was like how long it's been and how short it's been all in the same breath. 

Karoliina Kettukari: It, it feels like five years, I think. But we started, um, accumulating more licensees. In the fall of 24 and now we have licensed about half of our, uh, normal employees, which is quite a lot, so to say.

Hmm. And I think it's mostly thanks to the eagerness of our top management and also the eagerness of our employees. So our employees are very excited to use new tech, new technologies, and we want to give them the tools. And hope they don't go to touch bd, for example. 

Tom Arbuthnot: Yeah. And how's the adoption been Because it's super interesting, like this conversation about AI is like, it's gonna change everything you need to get on it.

And at the same time it's a, a real, or I feel like I a real effort to get sustained adoption and cultural change. 

Karoliina Kettukari: Yeah, that's actually a very interesting topic because of course we have the first people, first teams, who are very eager, very competent, already building agents in Copilot Studio, et cetera.

And then we have the big masses who are just thinking about, what should I prompt with Copilot? So it's the scale of different users and the scale of knowledge is very, very large. In our organization. I, I think in any organization, yeah, has the same thing. And we've been trying to have very different kinds of support methods, different kinds of trainings, um, ideation, workshops, et cetera, so that every, uh, different person, every different type of learner can find something they can relate to.

And like, they take just one thing with them to the next working day. 

Tom Arbuthnot: Yeah, that's interesting to think about it like that. 'cause you've kind of got the super forward leaning people within any cohort or organization. And actually they're probably already playing with chat GPT or Gemini or something, and they're coming back to the organization saying, I want to do this.

Let me do this. Can I do that? Like, like they're pushing the boundaries. So that's one type of conversation. And then people who haven't yet got there in their personal life actually you're starting from a different position and, and they're not, they're not chasing you for technology. It's the other way around.

You are trying to espouse the value. 

Karoliina Kettukari: Yeah. But it's been, I've been, uh, like working with modern work technologies for, for over a decade. Uh, and, uh. This has been the first time people come so eagerly to us to say that, Hey, give us trainings. Give us the license. I, I want to try this. I want to do this. So it's been very nice to see this kind of like progressive thoughts also from like those kind of knowledge workers who usually are busy with everything else than technology. 

Tom Arbuthnot: Yeah. It's nice when you are the, the area you work in becomes kind of mainstream important to the business in the sense of the business know it's going to impact individuals as well.

Uh, like things are changing. I need to be on the front of this, not the back of it. It's, uh, yeah, it's rare. I, I think, well, certainly in my, you know, uh, oh God, almost two decades of kind of working with organizations, it's rare where you are. The hot topic. We saw it with Teams during COVID, right? Suddenly everybody was like, oh, I need to get this working.

It's really important to the business, and I feel like it's, it's even more so with AI particularly. In, in knowledge work type scenarios, everybody's aware it's going to have an impact. So actually on a personal level, they want to be on, on the front of that conversation, not being disrupted themselves by it.

Karoliina Kettukari: Yeah. And we also have quite a lot of conversations where people are actually worried that, am I, am I behind? Is it too late for me to start using Copilot? So we are trying to, uh, calm them down and tell them it's okay. You can start right now. You can start tomorrow. You can start even next week. And you'll get along with, uh, aboard the train.

Tom Arbuthnot: Yeah. How's it been for your organization in terms of the M365 Copilot license versus Copilot chats? Have you got different users using both of those? How's that worked out? 

Karoliina Kettukari: Yeah, we have, um, about 50% of users that, like the amount of users who are using licensed version. We have about 50% of them who are using a chat version.

So for example, the imaginary numbers. If we have 10,000 licensed users on top of that, we also have 5,000 users using the chat. 

Tom Arbuthnot: Yeah, yeah. It's interesting to me that that's, uh, free slash included in your costs. And actually there's quite a lot of value to be gained out of chat. I'm not seeing many organizations like they're, they're doing work around the M365 Copilot, 'cause they've invested the money in the license.

They want to drive it. But actually there's a tier of users that can still get value out of Copilot chat and they're the type of users that potentially are the business risk of jumping to the consumer versions of other products and actually you, the data getting into the wrong place. So, uh, it's interesting to see how many organizations are saying, look, actually there's value to be had here even without the $30 per month Copilot, you know, M365 Copilot license.

Karoliina Kettukari: And we, we believe that we have some roles or, or people in the organization who are fine with like just a chat version because for example, most of their work or documents are in our customer, um, customer systems or customer solutions. So in that case, they don't even, they can't even use the intrinsic Yeah.

Tom Arbuthnot: They're not getting much upside of an embedded experience in Word or, or Excel or, uh, yeah, or Outlook if they live in the line of business app. 

Karoliina Kettukari: Yeah, so, so that's why we also want to give trainings and inform people about the chat, chat version as well. 

Tom Arbuthnot: Yeah, and you mentioned agents at the start of the conversation that that's, that's really progressive from what I see in organizations like Microsoft are hammering on agents and I think part of that for Microsoft is.

A user adoption is hard. If we can find automated business use cases, we can find additional value fast. Um, is that fair? Like how, how much, how much has been user use cases versus kind of Microsoft or the business saying, can we automate this process? 

Karoliina Kettukari: Yeah. Topics agents have been a hot topic for, I don't know, about half a year maybe, and I, I think they're like the next.

Step for, for what's to come, what we, what we have at the moment, what, what users can do at the moment. It's not that great. You, you can basically create an agent, which does have some good meta prompts, like background information for your agent, and that's a nice tool for our own team. Or for example, myself, I use a lot of the researcher and analyst agents.

Mm-hmm. They, they work really well. 

Tom Arbuthnot: Yeah. 

Karoliina Kettukari: But when it comes to like business processes, then we go to Copilot Studio and then it's not that simple anymore. 

Tom Arbuthnot: Yeah, it's interesting, this kind of, uh, again, like pitch of like everybody's been, create, create these agents and processes, but the big jump between very basic use cases and then into low-code, no-code, and then into pro code and, and that kind of, that journey accelerates quite quickly.

Karoliina Kettukari: Yeah. Uh, I think when we have discussions with end users, they have great use cases, so they have amazing use cases for, for agents, but I think nine out of 10 of those use cases, um. We stopped there because we can't use Copilot like agent builder to build that use case. Mm-hmm. And then you have to jump to the studio and it's the, it's the money and it's the licensing and it's the, you, you really have to have the technical capabilities to use that.

Tom Arbuthnot: Yeah. Do do you think there's going to be a place in organizations for. Kind of a, a specialist team that is like part of it, that actually is going in and helping bring those solutions through. I, I can see a case where you could have, we used to see it with low-code, no-code, potentially kind of tiger teams going around being like, I can build a power app for that process for you.

Like the marketing team aren't gonna build it, but they could use it. I feel like there might be the same use case with agentic workflows. 

Karoliina Kettukari: I really hope that because for example, our team, our mod work team, we are like that for Teams or Copilot or M365. So people, uh, come to us and we, we have trainings, we have the workshops.

We, we do a lot of, uh, talking in different events internally. So I really hope that in the future we also have those like, um, Cowell Studio, uh, consultant, internal consultants going around different teams. 

Tom Arbuthnot: It's, it's, it is an interesting funding conversation from an internal point of view of like, where's that cost go?

But I can, if you could get that team humming, they could potentially move between departments, between locations and, and even if they're picking off, like it doesn't have to be world changing, end to end automations. It can just be time saving pieces for different departments that can really add up. I think.

Karoliina Kettukari: Yeah. And I think that from an internal point of view, uh, I think like many consultants are doing that at the moment and of, of course want to try, uh, help, uh, want to help companies doing that. Yeah. But, but from an, like an internal point of view, when it comes to like business processes, it's quite a difficult task to come into a.

Uh, to like, um, finance organization and try to gather all the information about regulations and everything. So it's actually easier to do that kind of role internally, 

Tom Arbuthnot: I would a hundred percent agree, like, like the best consultant in the world, their, the time bound. So they can only spend so long with each person.

Then they present the result. What tends to happen with these processes is you build the V1 and then the person use it and they go, oh, I forgot. There's this exception and there's this use case in that use case, and then six months later, the business processes change for some external reason. So actually now I need you to tweak the solution again, because it used to be PDFs, but now we're using online forms or whatever it may be.

So I, I think it's great. Consulting is great for, here's what's possible. Here's an example. Mm-hmm. Um, or if you're in a long-term engagement, then you're, you're the equivalent of getting to know the people. But I, I, I would agree with you. Like the, the potential of also for the business, trusting. A new process and knowing they can go back to Sarah in six months and be like, it is not working for us.

Can you change it like that? That trust of there's someone going to support us in the future as well. 

Karoliina Kettukari: Yeah, exactly. Yeah. 

Tom Arbuthnot: So digging back into the M365 Copilot, as much as you can share, what have been the hero use cases or experiences that have really caught on with the users? 

Karoliina Kettukari: Um, we've done some research, so we have used the, uh, analytics.

Microsoft is, is giving us, and also we have done some our own, own end user research and, um, most of the use cases with interest of Copilot have been meetings. Of course. I, I think that's the one that gives, uh, immediately like hours of savings per week. Yeah. If you use it correctly. That, that's like, that's one for sure.

Yeah. And it's 

Tom Arbuthnot: a problem really everybody has, is too many meetings, meeting minutes summaries. Yeah. Like I apparently internally, like the Microsoft numbers, that's the one that gets Yeah, the, the biggest immediate hit because it's just obvious value. 

Karoliina Kettukari: Yeah. Uh, for example, today I followed three meetings, and then I just used like, uh, three minutes to, uh, read through the meeting notes.

Uh, instead of sitting in three meetings, three one hour long meetings. 

Tom Arbuthnot: That's amazing. Yeah. Most of my meetings are with customers, so it's harder for me to skip one. But this is again, the benefits of being internal. 

Karoliina Kettukari: Yeah. Um. Other things? Uh, I think it's been a lot about, uh, summarizing content, um, comparing to documents like very, um, should I say basic, basic, but important personal productivity stuff.

So you can't really touch, uh, by use case if it's useful or not. It depends so much on your personal role and what you're doing, uh, in your day-to-day job. But, um, I think we are on the verge of moving from like small personal use cases. To, to bigger. For example, team related use cases or more complex processes, for example, um, regarding, uh, risk and compliance documents or something.

You can't actually work with Copilot. 

Tom Arbuthnot: Yeah, yeah. The technology is getting better for the multiplayer use cases as well, so things like. Facilitator in meetings versus your personal Copilot experience or agents in teams where multiple people it still, it still feels like we're feeling out what multiplayer AI looks like.

You know, we've had pages, we've now got notebooks, like there's, I still feel like we're in a phase of like throwing things out and being like, do people like working together like this or like this? 

Karoliina Kettukari: Yep, and and it is very interesting point from adoption point of view. We don't even try to educate or inform people about every single new functionality.

Like, no, no one has the time. Yeah. So we are, we try to focus on, uh, showcasing the basic use cases. And then when someone goes to, Hey, what's this notebook? We, we can tell about it. Yeah. Or like, refer to some link, but it's not something we try to like power on a, on a everyday basis. 

Tom Arbuthnot: Ah, that's super interesting actually.

'cause, because also there's a bit of a, I feel like there's a risk there and more with Microsoft than there used to be. They're like, they're experimenting. So actually you could go out on this, you know, do a big pitch on notebook, and then in three months time they're like, ah, that's not the thing. Now this is the thing.

We sort, you know, scheduled, scheduled prompts and Copilot actions and the changes there. So there's some safety to going with the fundamentals and letting, because if someone's come to you to ask by definition, they're a little bit more interested, I guess. 

Karoliina Kettukari: Yeah, yeah. And of course, um. We are working in a very regulated, uh, business areas, so we have to go through quite a lot of processes before we can give the tools to our end users.

Uh, I'm really happy that, for example, Copilot is now like accepted as a formal tool. Mm-hmm. But then when it comes to, for example, interpreter agents. So can we allow users to try that out right away? Do we have to go through yet another process? 

Tom Arbuthnot: Mm. 

Karoliina Kettukari: There will be interesting lines to be drawn, which is a new functionality and which actually is like a new product.

You have to, you have to get an accepted, 

Tom Arbuthnot: yeah. Yeah. And typically the, uh. The compliance audit log conversation tends to lag the features when Microsoft are moving fast as well. So they're like, here's a new thing, like audit log coming soon. It's like, well, that's out for us straight away, you know? Yeah.

And how do you, I know you, you've got an inside track, right? 'cause you're an MVP and you're super involved in the community and with Microsoft, but how do you deal with this flow of constant change in choosing what to communicate and what not to communicate? How do you, how's that work internally? 

Karoliina Kettukari: We try to give things as simple as we can, can to the end users.

So if you are a pro user, we have a lot of documentation. We have a lot of links, we had a lot of trainings of our own and from our partners as well. But for the like normal end user who has 10 minutes in a day to learn something new, we try to. Um, advertise or communicate, do this thing, or This is now important.

Try this prompt, for example. Yeah. So as like, as boring as it may seem, the answer really is try to keep it very, very simple and focus on like one through 1, 2, 3 most important things. 

Tom Arbuthnot: And is that, is that like. Email, is that like, um, I suppose say Yammer, like Viva engaged, video? What modality are you using to get that information to the users?

Karoliina Kettukari: Yeah. Uh, we have a Copilot support team in, in teams actually. Uh, we do have UA engage and, uh, it's quite an active community. We do have an group, group in there as well, but actually the Teams team, um. It was, it was, uh, grounded when we had this like pilot phase going on and then it just grew. 

Tom Arbuthnot: Yeah. 

Karoliina Kettukari: So I would recommend, like as a model work, best practice, I would recommend an open group just as Engage, engage group.

But we, we have both and then of course, a news in our internet. 

Tom Arbuthnot: And have you found any good way for your users to share their use cases with each other? I felt like a lot of organizations are. Scrambling to kind of do like best practice prompts or workflows or use cases. 

Karoliina Kettukari: We do have the same problem actually.

We have had, for example, competitions in our teams channel like say or Best prompt or try this and this, and then you'll get a sticker or, or whatever. And um, some people are eager to share ideas. But what has worked best actually is that we have tried to encourage teams. Like, uh, as single teams in our organization to have like, take 10 minutes in your weekly or in your monthly mm-hmm.

And they're one use case and that's been working really, really well. Because then you know your audience, it's like easy for you to share and then you all know the context. You can discuss, discuss about the prompt or, or the use case. So like team based, uh, use case sharing has been very successful and 

Tom Arbuthnot: that's interesting.

Makes sense. I guess it's hyper relevant and, uh, understandable. Whereas actually if it's the, the, the, the finance team sharing a use case of the HR team, they're like, well, great, we don't, we don't deal with that scenario. It, it, it, it, it could be hard to make the leap to how I might apply that capability to my part of the business.

Karoliina Kettukari: Yeah. The sad part is that then we don't have like the organization wide prompt or use case library, but we try to gather those in our surveys as well. 

Tom Arbuthnot: Yeah. I feel like there's something missing there. I'm not sure what the answer is, but there's something missing for this organizational library, and maybe it'll come to the product at some point, but like, here's, here's the best practice prompts and use cases across the org.

'cause there's, there's so much value there and you know. There are individuals within the organization who are absolutely really pushing the value and really getting great, great use out of it. Yeah. How do you replicate that across the org? 

Karoliina Kettukari: Yeah. 

Tom Arbuthnot: Awesome. Well, thanks so much for sharing so much insight.

It's really nice to hear, again, someone living with it, pushing the journey and kind of mm-hmm. I know you are, uh, you, you are, you are kind of really into everything around it. So I, I, I, I a hundred percent trust your thoughts and opinions on what's working and what's not. So thanks for sharing. 

Karoliina Kettukari: I had to live with my opinions.

Exactly. 

Tom Arbuthnot: Exactly. Awesome. Well, um, I'd love to have you back in sometime. Maybe we give it a bit of time and see how the, the agent adoption's going and then, uh, have that conversation in, uh, in six months or a year or something. 

Karoliina Kettukari: Yes, of course. I'm happy to share my experiences. 

Tom Arbuthnot: Awesome. Thanks so much. 

Karoliina Kettukari: Cool.

Thank you so much.