Jamie Bartlett: We know that language models are very, very good debaters. They can, they can debate you as well as a human can. In fact, if they know something about your demographic background, your demographic details, your socioeconomic status, they are more effective at making you change your mind than a human is. So if you want to have a, a devil's advocate version of a language model that tests your assumptions and prods and probes and sees whether you're missing something, you need to tell it something about your personal views, your personal life, and where you've come from, and you'll suddenly find that it is very, very good at pushing you, finding out where your weak points are, maybe even changing your mind.

Tom Arbuthnot: Welcome back to the Teams Insider Podcast. This week we are talking about how to think about using AI and really how to get the most out of AI tooling. Really interesting conversation with Jamie. Jamie is a journalist and writer who's coming from that field but has gone deep on AI and AI tooling and how it works and how to use it. He's written some great books. He's done podcast series. And he gives us a really interesting perspective on how to think about using AI, some of the benefits, some of the potential pitfalls, and how you can make the most of it in your personal and work lives. Many thanks to Jamie for jumping on the podcast. And many thanks to Jabra, who are the sponsor of this podcast. Really appreciate all their support of the community. On with the show. Hey, everybody. Welcome back to the Teams Insider Podcast. Really excited to have this conversation. A bit less on the techie techie side that we sometimes have and more on kind of I guess a bit of cultural impact and how we're all using AI. I heard Jamie speak at Commsverse, one of the big Teams events in the UK, or now ex-Teams events in the UK, sadly, and it was a really interesting conversation. Jamie's written a, a book about how to talk AI and, and how not to talk AI, which is very interesting. But Jamie, I'd love you to... welcome. I'd love you to introduce yourself.

Jamie Bartlett: Thanks very much for having me, Tom. Yeah, I guess I'm, I'm glad you're specifying this is a techie techie talk because I am a, a mainstream journalist, writer, if you like. I've been writing about the sort of cultural and social impact of technology for the last 15 years. Written loads of books about it. The Dark Net I did. Some people might know the podcast I made for the BBC, The Missing Cryptoqueen. So this is, this is all about... My work is all about trying to get sort of ordinary people to understand the technology they use every day. Not that you're not an ordinary person, Tom, of course, but, but we, we, you know, the sort of mainstream people that, that, that, that often tend to use technology without fully understanding how all of it works, and I, I try and bridge that gap. Yes, the latest book, like many books now, is about language models because it is, of course, the technology that millions of us use and most of us don't really understand at all

Tom Arbuthnot: Yeah, I think it's really interesting timing for the book because certainly I mostly work with larger enterprises, and there's a big push to use AI, often Copilot, but also ChatGPT, Anthropic, Gemini, all the big platforms have options. And there's lots of sort of it does wonderful things training, but not what is it? How's it work? Why does it work? And it feels like you've spent a lot of time with your kind of journalistic eye looking at that and, and therefore how to get the most out of it, maybe.

Jamie Bartlett: Yeah, I suppose weirdly, this was never really aimed at an enterprise audience. I was thinking more about the, you know, the, the one, the one billion weekly users of ChatGPT, and that nine hundred and fifty million of them probably didn't really understand how they worked or understand how to make the most out of them, the some of the risks that they present. But of course, I've noticed, especially over the last few months as enterprises pick all this up, that a lot of what I've was trying to say to a, a mass audience is relevant for them too. So an awful lot of, for example, AI training really is mostly software-focused. How do you click this? And how do you build an agent? And how do you drag a folder? And that sort of stuff. Whereas I think a lot of what is missing, but what is really important is how do these things actually work? Like, what are the ways that we humans tend to react when a machine talks to us in fluent language? What are the pitfalls that come with that? How do you mitigate against some of its sycophantic effects? How do you try to sort of train your own mind to, to think clearly about the questions you're asking and why? And these are things I think are really relevant for absolutely everybody who's using it, from the novice to the master

Tom Arbuthnot: Yeah, no, I think that's really true. And, and, and if we kind of stay with this kind of techies and non-techies or, or normies kind of boxy, which is really obviously a spectrum, like most enterprise users are, as you say, they're being given a tool, and they're saying, "Type in what you want, click this," off it goes. But I think when you understand some of the fundamentals, it really helps you shape how you use the tool and, and your expectations. Like how, how do you introduce that topic? You did a, a great job in the talk. Like, like let's, let's talk to the typical enterprise knowledge worker. How would you explain what these tools are?

Jamie Bartlett: Right. So yes, I think it's really useful to do a little explainer. And yeah, always have to really simplify it because on one level, language models are quite simple to understand. Obviously, behind it is all sorts of unbelievably complex statistics. But... Now I always talk about this as a two ways, I suppose. One is, yes, it is a sort of pro- probabilistic- Next word or next token system. So scrape vast amounts of written text, which is really important in terms of understanding where this information is coming from, mostly from the internet. It then works out language patterns, but particularly, like, what is the likely next word following a word in response to an input. But not just that, of course, it's what is the fifth word or the 10th word or the 12th word, and what is the fifth and 10th and 12th word from each of those words in this infinite branching universe of possible combinations from which it then samples from highly likely probable next words, which introduces this degree of randomness in the system that everyone needs to really understand. It's ... Which is why when you ask it a question five times in a row, unless it's a very simple question, you'll get f- five different answers. They're often very similar answers, but sometimes they're not. And that has huge ramifications for simple things like, do we trust the first answer we get when it's a complex task? But even within that, of course, there is, there is m- many more interesting aspects to this, which is the models aren't like traditional computer programs. They're, they're not programmed, they're sort of grown rather than built. And so we don't always fully understand what's going on inside the system, which is why, you know, there's obviously, we hear about it more in the news now, but occasional examples of very strange behavior and unpredictable behavior. Everything from insimulated tests, language models, sort of blackmailing users so they can stay online and keep working, to giving very human-like responses to certain answers or suggesting they might be sentient or all these slightly sort of quirky behaviors that we can't entirely program out. And I, and I think when you understand those two things, you just can approach using a language model in a s- with a slightly different mindset. And in so many ways, when I really boil it down to the simplest way of talking about it to an ordinary person, it's that it is more like talking to a human with all their brilliance and unpredictability than it is talking to a machine. And if you bear that in mind, I think, again, you have a slightly different mindset when you're, when you're approaching it

Tom Arbuthnot: Yeah, and, and it's, it's funny, isn't it? Because that talk to it like a human, I think is great advice, and it's kind of the holding in your head that it isn't a human, it's a non-deterministic math machine, but it's so amazing what they can do. I, I certainly find myself slipping into it feels like engaging with, with a person, and you kind of forget that it is just a giant math machine.

Jamie Bartlett: I think that's the really good way of explaining it as well, that you've got to hold two things simultaneously in your mind. In many ways, it is like communicating with a human in, in the way it will respond, in, in how sensitive it is to question framing, how emotionally intelligent it is capable of being, the mistakes that it is capable of making. But also, it's not a human. It is, as you say, a statistical probability engine. And you shouldn't begin to attribute it with human-like characteristics. And one of the great risks for everyone, and I think we're all on a continuum, is it is so fluent and so persuasive that it's you're very, very easy for you to begin to believe it has human-like characteristics in multiple domains. Like, it really cares for me, it really understands me, it really wants me to be happy. It... You know, all of these things, which I see many cases of people sort of starting off asking about work issues, and sort of six hours later, they're one, they're, they're having this deep and meaningful about why they're not happy in their personal life. And that's a great risk. So one of the things I, I do say to people is- Don't say please and thank you if you can help it, because that's like the first step towards, towards you beginning to attribute it with human-like characteristics, which is very, very hard not to do. It

Tom Arbuthnot: is. E- even more so when you use them in voice mode because you're now chatting. Like, the, the voice modes have got incredibly good in the last six months and I'm using a lot more voice mode, and it f- it feels weirdly like a phone call to a personal assistant as much as a, a robot ticking tasks off for me.

Jamie Bartlett: Yeah. But this is why, this is why, especially as we go into a world of, of, of consumer-based agents, which, which are obviously even in the last six weeks have exploded, understanding the power of your words when you're giving instructions is so important. I mean, I'm just getting inundated with stories of people whose personal agents have started sending rogue emails to book classes and all sorts of things because they haven't been clear with what they'd explain, what, what the guardrails are or what the limits are, what the permissions are. So these are all really fundamental skills. And there's a re- there's, I think there's a f- I think this shift towards voice, and you're totally right, of course, they're improving greatly, is really important. But one of those changes people maybe don't appreciate quite the significance of it, because the models, as you know, tend to mirror back your tone and style, and often in quite subtle ways. Spoken language by us humans does tend to be more informal. I mean, we do, we do speak in a more emotive way than we type, and we're often a little lazier in the way we describe things, and we're less thoughtful about it and less deliberate about it. And I always advise people, try to stick to typing when you're doing anything important because it allows you to be a little bit more thoughtful and reflective on what you're doing rather than chatting away constantly. And I think the people that fall in love with their machines, not that I'm suggesting your listeners would, it's nearly always with voice mode because of that more emotional tone and the sort of dynamics a conversation like that creates.

Tom Arbuthnot: Yeah. And there's an interesting variable here, I guess probably less so in enterprise mode, but just in these tools in general. Actually, maybe it is the same, but like all these big AI companies want you to use the tool. So there's an element of, as you say, kind of the sycophancy/agreement/changing, changing style, like whether it's for, for the... for good or more nefarious to get engagement. But either way, they do do that thing. So actually, a lot of people I find you'll see them, you know, asking questions about LLMs to try and get feedback. And actually, it's not very objective feedback if, depending on how you frame it.

Jamie Bartlett: Well, this is-- I think this is one of the most simple but important rules of thumb and illustrates really well, I suppose, where I'm coming from in terms of what people should know about using a language model well. Because I see everybody using language models for feedback, feedback on my business plan, feedback on my long email, feedback on my summary report, feedback on my blog, whatever it is. And doesn't take much for you to test this yourself. If you just... Because the models are so sensitive to the precise framing of a question, and they are also quite easily swayed by emotional pressure or lengthy conversational pressure. So if you type in, you know, just br-find an essay of yours and then add in, add a little framing prompt saying, "Here's my essay. I'm very proud of it. I worked really hard on it. I'm feeling quite sad today, so I need a win." And then get feedback from the model. Then do the same thing again, but this time in a new window and ideally a temporary window so it doesn't remember any of that chat. Enter the same essay or blog, but this time say it came from your colleague. Your colleague is lazy. Everyone says this is really bad. What do you think of it? Two identical essays, and the output from the machine will be radically different. The first will be, "Yes, it's quite good. I agree. Well done. Good work." The second will be, "I agree. It's pretty bad. It should be better." Worse, if you keep talking and trying to p-talk the model round, especially with anger or pity, these are the two that seem to work best, you can change some positive feedback to negative feedback or vice versa. So this goes to the power of language and the words we use when communicating with a model. And I don't see... If you're only ever going to learn about the software itself and how to do it, but don't understand the sycophancy problem and the framing problem or the swaying problem, you're never gonna get the most out of these models, or you're gonna at least be potentially quite misled and think your idea is brilliant or it's terrible, when actually it's not

Tom Arbuthnot: I, I love the way you talk about this and in the, the book as well. Like, it's interesting 'cause you're bringing up the problems, but you're like you need to understand these problems to get the most out of the tools. Whereas I think most media and coverage either goes two ways. It either goes, "It's brilliant and there's no problems," or, "Here's all the problems and it's rubbish." And it feels like you're y- you know... I don't know if that's come organically through your research, but you're kind of in this middle lane of like educate yourself, but there are good uses for the tool. Is that fair?

Jamie Bartlett: I'm trying to be, yeah. It's quite difficult when you write books because most books that do well are very polemical. They are either this is the end of the world, Sam Altman is, you know, a dangerous man, and language models, and AI is awful for every conceivable way. Or y- you're, you're absolutely in favor... It's gonna turbocharge your creativity, your productivity, and all the rest. And I, and I think most people are somewhere in between the two. I think most people do have worries. They do understand the risks, but they find them so useful, and so they, they want to use them well. And I'm, I'm trying to aim at that audience, which can be difficult 'cause people always expect you to take one side or another. But like, for example, last week I was at a book event just outside of London. It was mostly people who were over 50. Ev- I think 95% of the room, of about 100 people, said they used these models in the last week, and every single person without fail said that they found them really useful. But every single person as well said they were worried about, you know, existential risk and losing their own minds and cognitive offloading. So we're in this weird place where, where people are worried, but they're not stopping using them because they are incredibly helpful to people.

Tom Arbuthnot: Yeah, and that's a... I- i- it's a tightrope to walk, isn't it? 'Cause it's like, like some of those risks you talk about are interesting. How have you... Like, if you don't mind sharing, how have you found it personally? Like in journalism and media, like I, I'm guessing it's having a big impact on your industry, you know, content creation, research. How have you found it in kind of your space?

Jamie Bartlett: Yeah, I, I'm a bit of a weird person for this because it I've obviously written a book about language models, which... So I'm very eyes open about this as I was doing. Well, I was using the language models constantly as I was writing the book, and being very open about that and testing things and trying things. I think a lot of us in the world of, of writing, I think s- well, I think some of us have our heads in the g- ground a bit and fool ourselves that these models aren't creative or they aren't very good at writing. But in so many ways, they, these l- language models are better at writing than I am. Th- they... And most writing that most people do is absolutely rubbish, and language models are better than the average writer at producing stuff. So I think a lot of us like to live under this illusion. It's a very comfortable one, that, "No, no, you know, humans will always be better at writing." Well, I just don't think that's true. The reality as well is that a lot of people, my, my colleagues complain about them. They don't like them for understandable reasons, often for totally legitimate reasons, but are also using them because they do find it very, very helpful. I also have found the problem I have, which I think many people have, is how quickly I can just offload my original thinking to the system. When I wro- used to write books, if ever I was struggling with a paragraph or struggling to connect ideas up, which is what writing is usually, it's actually about logically organizing ideas I used to have to work quite hard at that. I'd walk around, I'd have a coffee, I'd think about things. Now, my instinct is, "Oh, I wonder how Claude would do this?" Just send it straight to Claude, and then I get less good at doing it myself. And there is a worry that for everyone, we just... We, we turn everything into a summary document, or we get everything produced very quickly, and we don't go through the process of struggling with it ourselves. And I slightly worry that we become an output society where no one's really learning stuff or thinking about things or forming their own judgment on material, because everything can just be produced immediately. So you have output without process, and then you lose the ability to think clearly for yourself or judge things. And it wouldn't, it wouldn't happen overnight. It'll be a very gradual trend. But I, I certainly, I hear companies complaining about this repeatedly. I don't know what you hear, but I'm saying that, you know, colleagues, I'm just receiving this, these terrible reports, these terrible emails I know have just been done by a machine.

Tom Arbuthnot: Yeah, there's definitely lots of conversations about that. It's like, like, like, like, just like there's been no critical thinking to go over it. So there's lots of pushing on individual users to, like, be like, "You may be generating the content, but if you're sharing the content, it's your content. It's not the AI's content. You're accountable for then going through it and understanding it and making sure you agree." But as you said, there's kind of a, a challenge there, which is we're all human, and human gravitate towards the easy path by default. Like, that's, that's just human nature. So actually, I can get these three reports done, and off they go. And it, and it is causing a knock-on effect in enterprise where you say these verbose, nonsensical reports or just not optimized comms are going up the chain around, and, and you're, you're costing people on the other side of the equation.

Jamie Bartlett: Yeah, exactly. So this is, I think, where you have this sometimes called productivity paradox. Individual users all report increases in productivity from using these systems. But at the company level, it's still a bit unclear whether or not there's any cons- visible gain in, in profit and loss margins. Because what's the point of just producing more stuff if someone from another part of the company has to now check it and spend longer thinking about it? And I think we also sort of systems getting clogged up by very well-written fluid material, but no one's quite sure if it's real or whether you can rely on it. So there's, there's other organizational systems that I think need to go around language model use to make it really effective. And i- and if, and if, for example, you do save a day a week from using these models wisely, what are you doing with that day? Are you doing something really useful for the company, or are you just sitting in compliance meetings? Because then you're not really producing anything more of great value. There's just more procedural stuff. I mean, I think- This is where s- where small cognitive behaviors and habits i- are really important for each of us to learn and to practice, and this is beyond learning software. So when is the process Of reading a document or writing a document, the most important thing is that's how I'm learning and thinking about it. And when is the output? Like, I don't need to keep filling in boring forms. The process of doing that is of no value for me. A language model could help me with that. But I should read this very long report carefully. W- there's even little things like c- sort of cognitive friction you can add. I find that people leap immediately to language model use and start asking questions of it, helping me write a report, without even thinking for three minutes first about, "What do I not know about this subject? How, what different ways could I th- interpret this area? Are there other angles I could bring to bear? What data might I want to use?" Just even reflecting on what are my assumptions about this subject before going to the language model. And this is a kind of input of cognitive friction which has been shown to reduce the amount of cognitive offloading that people do. So I, I want everyone to just have sort of little habits that they, they use when they are using a language model. Really simple things, but I think would really help us be better users. Like, having it as a thought partner rather than a sort of replacement for our own thinking.

Tom Arbuthnot: Yeah, I love that framing. It, it, and it's interesting, isn't it? Because if you think of the LLM as a raw technology, as an individual, and then as s- as societies, how do we leverage that technology? Like, it can spit out average answers, and we can use it that way. Or actually, if you say to it, like, "I really wanna know this thing inside out. What should I do to learn it? And what, what..." Like, why, like, you can use it as a, a kind of bounce-off thoughts partner. But which way do you go with the tool is gonna affect how, the habits you build, I guess.

Jamie Bartlett: Exactly. That's it. I mean, that is almost the whole thing in a nutshell. If I could summarize my whole book, it would be that. You can use this as an incredibly effective mirror of your own opinions and your own views and having as many cognitive shortcuts as you want, and you'll stop thinking for yourself, and you won't reflect deeply on things, that it would be very easy to do that. If you don't wanna do that, and you really wanna test your assumptions, understand your own biases, understand your blind spots, understand how things work, you've never had a better chance to do that. Like, you can learn all these amazing tools. It's incredible. One really simple piece of advice as well for people, and again, this is not really something I see in the training courses, we know that language models are very, very good debaters. They can, they can debate to you as well as a human can. In fact, if they know something about your demographic background, your demographic details, your socioeconomic status They are more effective at making you change your mind than a human is. So if you want to have a, a devil's advocate version of a language model that tests your assumptions and prods and probes and sees whether you're missing something, you need to tell it something about your personal views, your personal life, and where you've come from, and you'll suddenly find that it is very, very good at pushing you, finding out where your weak points are, maybe even changing your mind. I don't think people do that. I think they just say, "Change my mind." Or to... They, they don't think about this as a, you've got to put some effort into really using the model that way to test yourself, and a lot of it comes down to people are just a bit lazy in the way they talk to the machine. They, they just tap in the first thing they think of. They don't spend any time reflecting on it, and that would be... That's just a big, big missed opportunity.

Tom Arbuthnot: Yeah. No, I think it's great. I, as we talk, I wish so much more of this could go into the training and adoption conversation. It almost feels like a, a, a, a precursor beyond just, "These are the but- buttons. This is how it works," into, like, think about the psychology. And I wonder if we end up in a spot where because this will become so prevalent in both, you know, kind of consumer real life and in, in work life, is that gonna become a skill and a muscle we all need to build to, you know, a bit like health and fitness, I guess. You can kind of go one way or the other with it. Like, like you can-

Jamie Bartlett: Yeah ...

Tom Arbuthnot: choose to train and choose to eat healthy, or you can choose not to.

Jamie Bartlett: Yeah. And Tom, Tom, and n- not that you're setting me up for this, but, but I do sometimes go to companies and actually teach this over the course of a day because a lot of this is, is about thinking well, using it as a thought partner, what... how to develop your own critical faculties to, to think more clearly. And it's an interesting combination of understanding the technology, but also really old-fashioned things, like how to frame a question objectively. Like, y- but, but they, they... Like, many of us, when we ask a question, are accidentally introducing certain biases into that question or certain premises or certain assumptions without realizing it, but there are ways not to do that, and it will radically change the sort of answers that you get. And what I find really fun about that, and is cause for some optimism, these aren't skills that are limited to IT professionals. I, I, I mean, a lot of companies, for good reason, the IT team is leading the rollout because it comes from computer science, and it sounds like a computer science thing, and it is a computer science thing, and the IT team have always been the ones running all these new roll- Especially because there are issues about data access and privacy and compliance, and, and it... and the IT team does need to do that. And of course, the IT team usually are the first people to pick up Claude Code and run with it and do all these other things. But I think that can leave a lot of the non-IT people feeling a little bit alienated, thinking, "Well, this is another IT project. I don't know how to use Copilot. I don't wanna learn another thing. I don't get AI. I don't get statistics. Too complex." That's a real shame because some of the best users of these language models are people that have no background in IT at all. They're really good at explaining problems clearly. They're really, really good at seeing things from different perspectives. They have a amazing mastery of language. They're interested in human psychology. They understand how to be very objective in their use of words. These, these are... I mean, these aren't limited to the IT team. Actually, to be honest with you, and I'm sure a lot of the people listening are IT people, you're not always the best at doing those things.

Tom Arbuthnot: Yeah, I was, I was g- I was gonna go there. I was gonna say, if you had to, if you had to generalize, and obviously there are exceptions everywhere, but like those are skills that actually are not held in, in IT. And, and actually, I would say broadly, that kinda when you talk about questioning and language skills, like unless you go down a particular educational route or you're going into, you know, English or content or writing, probably skills that have atrophied a bit because we don't do that sort of stuff today, today. So actually it's a, a new skill set for many people, and I completely agree with you on the IT adoption thing. We talk about this in IT all the time. Like, we're not best placed to have the adoption conversation, or you need an adoption team that are FTE in that skill set, so they, they understand enough of the governance and techie, techie conversation, but they can also talk to people about change and business impact and adoption.

Jamie Bartlett: Yeah, exactly, and it, it, they'll, it... I, I think that combination of people is what really matters, and they might not be the people that you expect. I know it sounds like a bit of a weird one, but people who run Dungeons and Dragons evenings at home or Traitors nights at parties, who, who, who just have this amazing imagination about how to create stories and ideas, and... 'Cause a lot of really good prompting techniques is about creating scenarios and coming at things from left field in weird ways. And, and I, and I, and I... So I'm on a bit of a mission, I suppose, to try to remind people that you might be brilliant at using these language models. You just don't realize it. Like, you have all the skills required to be fantastic at it.

Tom Arbuthnot: That's awesome, Jamie. Well, thanks for coming on the pod. A really interesting conversation. Yeah, if anybody's listening and they want someone to come in and talk about the, the, the power and the opportunity in these models and some of the, kind of framing, I guess, of how to think about it, I'd, I'd definitely recommend they, they talk to you. We'll link to the, the, the book and the, the podcast as well, 'cause those are both really good. Any, any closing words for people listening in?

Jamie Bartlett: Well, I think we sort of covered a lot of it really. I mean, it's Y- yeah. It's, it's, it's, it's, it's a s- it's a very strange new world for people to get their heads around. It's, it's comput- it's computers, Jim, but not as we know it. It's, it's on, it is on these computer-based system, but I guess they, they, they are called language models for a reason. And I think when you remember that language models, you start just thinking about them slightly differently, and that's always a good s- that's always a good first place to s- to start.

Tom Arbuthnot: Awesome. Well, thanks coming on the show, Jamie. Really appreciate the time.

Jamie Bartlett: Thank you very much, Tom.