AI and the Workforce: Why Augmentation Is Winning Over Automation

TBR Talks: Decoding Strategies and Ecosystems of the Globe's Top Tech Firms

AI and the Workforce: Why Augmentation Is Winning Over Automation

Play Episode Pause Episode

1x

00:00 / 00:57:30

Apple Podcasts Spotify

RSS Feed

Share

Link

Embed

AI and the Workforce: Why Augmentation Is Winning Over Automation

TBR Talks Host Patrick Heffernan: Welcome to TBR Talks: Decoding Strategies and Ecosystems of the Globe’s Top Tech Firms. Where we talk business model disruption in the broad technology ecosystem from management consultancies to systems integrators, hyperscalers to independent software vendors, telecom operators to network and infrastructure vendors, and chip manufacturers to value-added resellers. We’ll be answering some of the key intelligence questions we’ve heard from executives and business unit leaders among the leading professional IT services and telecom vendors. 

I’m Patrick Heffernan, Principal Analyst, and today we’ll be talking about practically delivering AI solutions to clients with Rob Kopel, Partner, AI Leader at PwC Australia. 

Context for Rob’s AI expertise 

Rob, welcome to TBR Talks. We’re in season five, and we’re very excited to have you. I’m joined today by Principal Analyst, Boz Hristov, and Senior Analyst, Alex Demeule. None of us are in the office together, but we’re very excited to be talking to you. And I know we met in Australia at lunch in Sydney, which for everybody else is just on your side is just simply lunch in Sydney. For us, it was very exciting because I got to go to Sydney and meet with you and your colleagues at PwC. It was a fantastic event, and we had such a great lunch I wanted to have you come on and have a chat with Boz and Alex and talk about AI because you’ve been in this field for so long, despite professionally not being that far into your career. So, I’d love for you to give us a little bit of your own background and then we’ll dive into some of the questions we wanted to ask you about AI.

Rob Kopel, Partner, AI Leader at PwC Australia: Yeah, I’d love to. So as a bit of background, I originally started out in the insurance AI space back before AI was as well built out as it was today. We actually spent a lot of time taking old servers that had been shut down from the decommissioned from the IT department and racking them in our own server rack in the mail room, which frustrated everyone in the office to no extent. And we challenged some actuarial guys to build better AI models than they could build standard actuarial models. And we managed to smash them. So, I think ever since that, probably almost 12 or so years ago now, I’ve been in the AI space. And I think I came to PwC Australia with a bit of a mission to build up our capability and almost startup style, build up the AI opportunities both within the firm and then take those lessons to our clients. And so, for the last six or so years, that’s what I’ve been focused on is building our Australian AI factory and then taking those lessons of how we reinvent ourselves to support how our clients are able to do it too. And I think with the explosion of intelligence we’ve seen, it’s been a really exciting time.

Patrick: That’s amazing that you started your AI journey with perhaps one of the most boring and traditional businesses, insurance and actuarial tables.

Rob: Yeah, I would, I would not disagree with that, but I would say, math nerds can be fun occasionally, so don’t hate too much.

Patrick: Okay, fair enough, fair enough. Well, I mean, it actually has done a lot of things for you and brought you to PwC where you have that sort of startup opportunity. 

Client sentiments around AI and adoption

One thing I want to kind of start off with, I’m curious about this, because again, it was a great, great meeting you in person and chatting over lunch in Sydney. And one of the things that sort of struck me about our lunch and then the presentation that you made during the day, during the Analyst Day, was there’s sort of a sense we have as analysts looking at it from the outside, that clients are sort of, they’re bought into AI. They’re absolutely sort of on board with the disruption that’s coming. They’re certainly fearful of it. They’re not sure they’re making the right investments. They’re not sure they’re going to see that return on those investments. But what I’m really curious is, because you’re working with clients day-to-day, are you seeing any kind of real pushback where there’s sort of a, we actually don’t need to adopt AI, or we don’t believe there’s going to be this great disruption that comes from it, or we don’t believe there’s going to be any value that comes? Do you still see any kind of naysayers, like real sort of, you know, I don’t want to say head in the sand, but you know what I mean?

Rob: Yeah, I don’t think we even, I haven’t encountered it for, maybe I’ll say, 14 or so months at this point. I think it used to be far more common of a trend. I think especially when you’d spend time really deep in with different professions and you ask them, is this fundamentally changing how you’re working or is this just saving you 30 minutes a day? You would hear a lot of maybe it’s saving me 30 minutes a day, I don’t see how this will change. Over the last four months, we’ve seen a step change in that though. There was a point where everyone had summer break, maybe December, where people really could spend the time on their own, playing with the latest and greatest AI tools, where when they really dedicated the time to it, they were able to experience kind of what the potential of these tools actually is. And every single conversation I’ve had since that point, even at the board levels, there’s always one person in the room who’s spent the time with advanced AI tools, and they will be able to communicate how meaningful that was to them and just how different, how big that difference is. And I think that speaks to a bigger trend in the market. 

When I’m talking to clients, we often discuss how fast people are going. You know, what does going fast mean? Is that adoption, you’re giving your staff maybe one, two, three different AI tools, and they’re saving 45 minutes a day. Is that startup style business reinvention? You are potentially pivoting into whole new markets. You’re actually reinventing the way you do processes. But I really try and force myself to zoom out a little bit and try and see the entire playing field. Where would Usain Bolt be if he was moving fast in the AI adoption market, right? What would so fast that it’s actually dangerous look like? Where should I see myself on that scale? And I think when you take that approach, you end up saying almost, if I honestly look at the market, I don’t see that many catastrophic failures of people going too fast. And when I talk to clients, I don’t hear about building things burning. I don’t hear about companies going out of business. And almost to me, that seems a little bit odd. When you naively think about it, if there is such a massive advantage in betting the house on AI that competitive pressures would force you into an all or nothing strategy today, you would expect to see that. And you would expect to see some companies who managed to pull off that strategy, right, wipe the floor with their competitors. But we really don’t see that today. 

So, I think the one thing I do want to raise, and I always raise, is that there is this massive reliability problem that AI has today. And I think that is why we see some people say it’s massively effective, and some people say it’s massively not effective, even though that second category has declined so much. And I think the easiest way I have to illustrate this is that if we went out, if I took you out to a finance firm, any one in Australia, we could go in the building, we go up to the 12th floor, we go and find someone who’s just working randomly at a desk, and we asked them, what task are you actually working on right now? What are the inputs? What is the job? What are the outputs? And we gave that task to AI, I think you would find that over 90% of the time, AI could do that task immediately. And in fact, you would prefer the work that the AI did the majority of the time. But that one third of the time, it’s significantly worse than what the human did. And you simply can’t trust it to always do that. And so, this eventuates, and it comes up through the data in what you see, which is that AI can do 90% of the tasks, but it only gets used for less than a quarter of them at the end of the day. And so, the best companies and the best people I see who are able to see the value and grab the value, they’re able to see this opportunity, this difference here. And they’re able to actually take it. 

To give you one example, as well, there’s a really standout hospitality client I’ve been working with. And they really get this. So, they’ve done two things. They’ve done the table stakes first. They’ve enabled their staff carte blanche access to secure AI tools. And then they’ve also set up a small team, not a massive transformation project, not a huge steering committee. They’re really a hit squad who’s been in their business for decades, who understands it implicitly. And They also understand the reliability problem. They’ve got a direct line to execs and a mandate to essentially go find where AI can be genuinely helpful to them and actually focus on how they can change how the business operates. And that team has been massively successful for them. I think they’ve done more than one project per month. They’ve done, with AI at their backing, with a small team. And that’s going from customer service into marketing and so on. And those aren’t just time-saving measures, but they’re really kind of reshaping the customer’s experience for their clients and reshaping how their business functions. So, I think it’s really important to recognize that there’s a reliability issue here. But if you are able to actually understand and master that, which we see clients do, and I think everyone’s slowly picking up over time, it really sets what I see as clients being able to do that can set them apart. And I think today, most organizations, although they’re slowly getting there, they’re only really doing that give staff AI tools and give them a little bit of training today.

Managing the risk in mature AI strategies

Boz Hristov, TBR Principal Analyst: Rob, this is Boz. This is a great overview of where you see the clients. It just made me think about some of the risks that may be associated with the various speeds of adoption or, you know, going too fast or not moving fast enough. In your experience, I mean, how would you describe clients with the more mature AI strategies manage that risk?

Rob: Yeah, I think, so that risk comes down to this understanding of how well AI can be relied upon in different cases. And generally speaking, when we talk about different ways of applying AI, we are talking about either we are looking at it in an automation sense, or we’re looking at it in an augmentation sense. And the automation sense, you often hear about, we really need to reinvent entire parts of our business, maybe whole processes, more standard business model reinvention. Because in order to get automation gains, you can’t just automate step three in an eight-step process because your bottleneck is now going to be step two, and it’s going to be step four, and five, and step one. You really need to redesign your entire process to support automation and make sure the throughput matches on the input and output sides. That requires standard project management governance. It requires standard product management governance. You’re building a tech solution, essentially, at the end of the day. And that’s where we see the best governance today, is where people have this understanding of reliability and standard product management. So that’s automation. 

On the augmentation side of the world, this requires your staff to have really deep understanding of where they can and can’t rely on AI. And currently, by and large, the best way to build the skill set that we see is firstly through training, but secondly, through putting the impetus on your staff that they have to own their own AI journey, and they have to show that they’re owning that. Because a lot of the time, one of the failure modes we see is that when I go and I chat with a client, they’ll say, hey, well, we’ve got this AI team in the center who’s incredible. But everyone at the coalface is going and saying, okay, we’ve got this incredible AI team for me. They’re going to handle it. I don’t need to do anything. And taking that impetus away from every staff member across the board means that they feel like they can rely on someone else to deal with AI change for them, and they aren’t actually going to get the experience on the tools. They won’t understand the reliability challenges. And then when they do have to use them, it causes issues because they can’t actually interpret whether a response from AI is good or bad. They don’t have that intuition yet. And so, we really strongly believe, obviously, in end-to-end AI governance, proper setup of your frameworks, policies, trainings, how to’s. But intuitionally, I really encourage people to more focus on how well, when they actually go and talk to staff members, do they understand the pros and cons or risks and benefits of AI when they use it? Because that’s really critical on the augmentation side. So, it’s a bit of a combination of those two worlds and the best people are getting both right at the moment, there’s no doubt about it. And if you get only one, you will get less than half the benefit. You really need both combined.

The talent reskilling conversation 

Boz: Yeah, that’s a fair way to put it, augmentation and automation, especially as it pertains to the skills. And that’s one thing that we hear in the market. I mean, everyone is talking about reskilling talent to ensure they have the AI-enabled bench. But us as analysts, we struggle sometimes to understand, you know, how much of that effort is centered on the professionals that are more on the engineering side of the house versus the client engagement? And do those two different, in terms of AI skills, are they different from what the knowledge base has to be? And most importantly, I mean, more importantly, I guess, how do you measure impact from any AI reskilling either on the engineering or on the client engagement side? So, there’s a two-part question that I have, but just maybe think about what we’re hearing in the market as well.

Rob: I’ll do my best to hit both parts. But I think generally speaking, there’s a real trap in the reskilling conversation today, which is this idea that we take people, we give them a reskilling that they need to do, and we treat it as this fixed target of what they actually need to achieve. You know, what skills do they need, great, let’s go train them. But the interface for AI and the interaction mechanism, it keeps changing so rapidly. By the time we’ve actually rolled out something, the tools, the programs have generally moved on. And to give maybe just a bit of insight into that, and maybe the best case study that I’ve seen is this one of software engineering, which we’re talking about, right? And really, I want everyone, I always encourage everyone to look at this because engineers were really, and still are, the first cohort to really go through this existential crisis at pace, right? And I think it tells you a lot about what’s actually going to come for other careers and professions. 

And if you go and you roll time back a little bit, this all started a few years ago with GitHub Copilot. You know, an awesome piece of software that got released that was essentially a smarter autocomplete. It could finish your sentence for you when you’re writing code. And then we improved it a little bit. A few months later, we got this new Codex model, and it could write a few lines of code. It could maybe even write a whole small chunk, like half a paragraph for you. And a few months after that, we get a new tool, right? This one’s called Cursor. It’s incredible. You have this little agent who sits beside you as you work. And it can kind of suggest whole chunks of code. And you can ask it, go and write this little user icon for me and add it to my software. And you can approve one by one, as the chunks crop up, you can say, tick, I’m happy with this chunk and include it. 

And then Cursor improved. The agents improved, right? And they evolved to the point where it was too tedious to go and approve each individual little chunk, and all of a sudden it would go and you’d say, hey, go and implement this whole new page in my application, and it would go out and do that, and I would just review the code at the end of the day. And that was a real game changer in how our software engineers started doing work. Additionally, since then, we’ve had awesome tools like Codex and Claude Code, where you’ll write the entire pull request for our software engineers. They’ll go do a whole ticket for you, and you just review that at the end of the day. And very finally, last month, now we’re getting to this point where I tell the agent what to do. It sends me a video showing the new feature it built. I just go and I highlight with a red marker, on the bit I don’t like in the video, and I send it back to it, and then it goes and continues working. And that entire progression has happened in one, maybe two years. 

And when you notice what’s actually changed is that the human has gone from writing code manually by hand, to reviewing code manually by hand, to reviewing the outcomes that coding actually leads to. And the skill of how they used AI at each one of these stages was really different from each other, right? But the thing that stayed constant throughout was their software engineering judgment. Can they look at the output and know if that output is right? Do they understand the problem well enough to actually direct the AI to do the right bits of work and catch mistakes as they happen? And I think that brings kind of the really important framing to the surface. which is that we are seeing AI create a bit of this kind of K-shaped economy at a job level. And I know econs often talk a lot about kind of this K-shape, but I don’t think enough people talk about it, where essentially when AI is coming to a role, whether it’s software engineering or other types of roles we often encounter, it can go in one of two directions. It either is removing high-skill work and leaving the drudgery, or it’s removing low-skill work, and it’s leaving the judgment. 

The classic example of this is if you take a London cab driver, when GPS first arrived. These drivers have spent years, years earning the knowledge, memorizing every street, every route, every shortcut. And that was their expert skill. When GPS arrived, it destroyed the intrinsic value of that knowledge overnight. The work that they were doing didn’t go away, but it went downhill a lot. There was lower wages, there was less pay, there was less expertise required, it was more commoditized. Anyone with a phone, you know, with Uber could go and do that job. The opposite of that one is accountants, for example, when digital spreadsheets arrived, right? Reduction in manual work, no more manual ledgers, no more manual reconciliation, the drudgery vanished. And the bit that was left, which we’re seeing for software engineers right now, is advisory, right? It’s judgment, it’s interpretation. And that profession got more interesting, more valued, right? And pay actually increased. 

So when I’m talking to clients about reskilling, I often am talking to them not so much in the lens of the question of, here is the exact reskilling about picking up AI software your people need, but it’s firstly, for each role in your business, which arm of the K-shape is it on? Is AI potentially stripping away some of the judgment and leaving the drudgery? Then we would encourage, we want you to go and reskill those people for other roles, right? For other areas they can build their judgment and expertise in, we don’t want to train them to use AI itself because AI is going to be able to use AI. 

There was a great case study recently, I’m sure you saw, of a global retailer who was able to transition all of their customer service staff and start a new line of business, which has been massively profitable with those same customer service staff because they cross-trained them. But in the case that AI is stripping out the drudgery of the job and it’s just leaving the judgment, which I would actually say is a majority case right now, like with software engineers. That’s the best-case scenario. And that’s where we want your staff to actually leverage AI to massively increase their productivity. But we still want to focus really the skilling and the training on their judgment and abilities, because that is the multiplicative effect on AI. And you can see, these are completely different responses from each other. 

And lastly, I guess, on your question of engineering versus client engagement question, I think we’re seeing teams shift a lot, not only in that you can be smaller and you can move a lot faster, but I’m seeing a lot that when our engineers are able to move at these ridiculous paces because they’ve stripped out of the drudgery, their blocker changes a little bit from not only purely on the engineering, can I see what is wrong with the code, but also the blocker becomes the people around them who are feeding them, this is what we actually need to do. These are the business requirements. And so, these staff who are almost cross-functional, who maybe you call them T-shaped staff, they are going massively up in demand and their ability to deliver an end-to-end project or product rapidly is increasing. And so I think what we’re focusing on today at I’ll say PwC and many other businesses as well, is really increasing the breadth of our experts, not just the depth, but we do need them to have that depth, otherwise they won’t have the right amount of judgment to actually review what the AI is producing.

Patrick: Yeah, Rob, I think that’s fascinating, and to me it’s frightening because you- it’s difficult to teach judgment. I mean, I’m not saying you either have it, or you don’t, but frequently it comes through experience and pain. It doesn’t just come, you know, out of a textbook or watching a YouTube video. 

I do want to challenge you on one thing. I want to bring Alex into this conversation, but before we do, you said the first group of people to sort of feel that existential threat from AI was engineers. I actually, I have, and Boz can back me up on this, a very vivid memory of a PwC event in Boston, it’s got to be three years ago now at least, where we saw a marketing application of AI that it was just, it looked like it basically reduced the human component by about, let’s say, 90%. Basically, the AI tool could do so much of the work that went into traditional marketing. And this was like to develop collateral for sales pitches and engagements. And it was just sort of, that to me was an eye-opener in terms of the existential threat to a certain profession. But I’ll take your point, engineering and marketing probably went hand in hand in that existential threat moment, right?

Rob: Yeah, I think so. And I think if you look at the stats and figures, there’s two really massively exposed industries ahead of everything else, which is customer service reps as number one, if you go and you sort by number of jobs that we estimate to see some level of impact. And number two is software engineering at the moment. I think for years, there were all these expectations and market of self-driving cars, what impacts is that going to have on the trucking and moving industry? And that really hasn’t eventuated because of that same kind of risk reliability point we were talking about earlier, right? Where to have something that is moving at 60 miles an hour on the road that weighs tens of thousands of pounds is so incredibly dangerous. You need 99.9999% certainty that it’s going to operate safely, right? And you need to be 100 times safer than the average human driver. But when we’re talking about interactions that often when you are phoning up a business today, it already isn’t a good experience. AI can actually improve that experience on average, and the risk is so so far much lower, right? And so, it’s really these cases of, well, where can you get the productivity benefit within the reliability threshold that AI has there today. And I think that is where we’re seeing big impacts in customer service, where we’re seeing big impacts in software engineering. And I think next off the rank, which we’re already starting to see a little bit, is in financial analysts and services, where AI has been massively improving in its capabilities recently.

Augmentation vs automation and workforce rationalization

Alex Demeule, TBR Senior Analyst: That’s awesome. Great. And hi, Rob, this is Alex Demeule jumping in. And this is- the next question is something that I feel like we have been really kind of talking about this entire conversation so far. We’re talking about automation versus augmentation, and I love the K-shaped economy call out and looking at sort of drudgery versus judgment and sort of the role of AI. But I think all this kind of comes back to sort of the corporate goal for AI. And so obviously one of the things that’s been in the headlines and has been a part of this conversation for so long is just is this a cost efficiency goal or is this a way for me to increase productivity and be able to do more with what I have? And when we talk about augmentation versus automation, I think a lot of the story that I’ve heard from you is just about sort of identifying where augmentation makes sense versus automating for cost efficiency. But is there sort of a through line for how enterprises in the economy today with the sort of geopolitical risks and the cost considerations, is when we net this out and look at software developers, marketing, and going into financial analysts, is there a through line towards wanting to sort of decrease headcount? Or is this a story of reinvestment and job growth, or maybe not job growth, but unlocking new job opportunities over the long term.

Rob: There’s something that is a good question that I may throw back to you on this, Alex, which is that it’s really around- when I talk to business leaders and I talk to staff, at least in Australia today, they’re surprisingly aligned on this when you get down to the facts. I think there’s a story at the moment of, you know, business leaders want to automate everything in existence, and staff are terrified of being replaced in their role. And I don’t think that really holds up today. I’m not saying it won’t hold up in the future, but I don’t think it holds up today. Because when I talk to clients who are really interested in this and when I talk to staff as well, they say the same thing, which is that they really enjoy the augmentation benefits that they get from AI, and they see that augmentation is what they want out of this technology. And I think the reasoning that they often give behind that is really logical, which goes something like full automation is not a competitive advantage. It’s a competitive disadvantage because none of these companies are really training their own multi-billion dollar model. They’re not training the next GPT-7. They are just giving the same text to the same model that their competitors also have access to. And so, if they can fully automate with something with AI that’s off the shelf, so can everyone else, right? Including startups, including competitors that they’ve never even heard of yet. It becomes table stakes, whatever that task is, it’s not an edge for them in the market. 

And I think it was a good example of this early in the year when there was an AI lab, they published just a plugin, a legal plugin. And when you went and you read what was in that legal plugin, it was literally a, I don’t know, it was a 12-page Word document, right? It was just a text file. There was no new model, there was no proprietary technology in that plugin. But when they did that, it wiped, I think, almost 300 billion US dollars off legal stocks in a single day. Not because it’s doing something different, right, but because the market had suddenly, I guess, started pricing that in that anyone could actually do that. And so, I think you get a lot of this demand for, I want augmentation because my competitive advantage isn’t in AI itself. It’s in what my people do with it that my competitors can’t do, right? It’s in my people’s heads. It’s in their institutional knowledge. It’s in their judgment, right, that we’re talking about before. It’s in the relationships that they have. And that’s my competitive moat, essentially. AI is just amplifying whatever is already there. And I think when you talk to staff in the business, they want the same thing. They want augmentation. They don’t want to do the drudgery, right? I haven’t met anyone yet who is really missing the drudgery work, if that makes sense.

Alex: Yeah, that’s a fantastic answer. And I can completely see the case for, you know, access to the capabilities being sort of commoditized. And so that judgment layer and kind of whose hands are on the tools is where it really makes a difference. And that keeps the human in the loop narrative really strong.

Rob: 100%. And I think it’s always interesting to also try and argue the steel man. And I think there is also a lot of discussion going on in the market that is a bit obscure. I’m sure you’ve heard about this phrase of AI washing, which is important to call out, which essentially goes something like if you go and you actually interview executives and you say, hey, how many of you have made headcount cuts in anticipation of AI versus how many of you have made headcount cuts based on the results that you’ve gotten from AI so far? Over two thirds will actually tell you we’re making headcount cuts in anticipation of AI benefits. And if you go and ask all of them after the fact and you say, hey, how many of you were able to actually replace those roles with AI at the moment today, it’s less than 10% will actually say we’ve been able to do that, right? So, I think a lot of what gets written about today as AI-driven workforce rationalization is unfortunately cost cutting in a trench coat, right? It’s AI giving you a nice story that you can tell your board. That sounds innovative rather than we need austerity measures or similar to that.

Alex: Yeah, and do you think that that’s true even in sort of a software development capacity, you know like, obviously the block announcement was one of the big headlines that we saw. And I’ve been in complete agreement with you. I think that a lot of the rationalization that we’ve seen has been sort of AI washed. And we saw a massive hiring spree sort of on that back of COVID. And so, tech companies, especially, are probably in a position of being sort of workforce rich, and so the ability to use AI- but when we were talking about the software development, when I think about one slice of maybe truth in sort of being able to get more for less out of your workforce, it’s got to be in that software development field.

Rob: Yeah, I agree with you that there is a lot of potential there. I think it’s still very hard to quantify at the moment. Maybe to call out a few key pieces that really rattle around my head on this. One is that software engineering is very interesting work in that it’s essentially limitless work. There’s no product, no solution that doesn’t need more adjustments. It doesn’t need some bug fixes. It doesn’t need some feature enhancements. It doesn’t need some updates to the latest iOS version to run on your phone. And because of this, you would naively expect that as people become more productive, there’s still a marginal value to having them on board because there’s just infinite work that comes with software. And you would expect that in markets where there isn’t infinite work, you know, I think the classic example is something like a tax return where even if you can do 1,000 tax returns a year because you’re so productive, you’re still only going to do one tax return a year unless you’re crazy. 

So, I think in software, you would naively expect that you still want to keep your software engineers because there’s just so much more value that you can add with them on board. And I think that is roughly what we actually see in the market, right? Which is that experienced roles don’t necessarily disappear at the moment, but it is these new entry level roles that are kind of disappearing. And it’s those people who don’t have the judgment to actually review the AI outcomes that start disappearing. But firms are hanging on to the software engineers that they have, and they are hiring, but they’re hiring at that kind of expert level end where they can get a 10x increase in productivity from AI, but they aren’t hiring at that kind of entry level end where they can’t get any benefit from AI because they don’t have the experience to review what AI has actually generated. And so, I think there’s a few very interesting effects that are adding up together. But I think this is aligning to what you see when you look at kind of US data, when you look at kind of Australian data. We’re seeing kids out of uni, four to six months out of uni, in software engineering, particularly, and in STEM slightly more broadly, their hiring rates are lower than the darkest days of COVID now, essentially. And so obviously, there’s a massive impact there. And we don’t see that across medical roles. We don’t see that across other roles in Australia. But we see that across software engineering and a little bit more broadly STEM. But for experienced devs, the market is as hot as it’s ever been in Australia, essentially. So, there’s a real concerning adjustment there.

Alex: That’s super helpful and definitely something to keep in mind. I want to hand it off to Boz, who I think is going to follow up with something on AI companies. Go ahead, Boz.

TBR’s new HIRI metric 

Boz: Yeah, I think the topic of like, you know, what you’re saying, how much the workforce reduction is a result of AI today versus anticipation, I think it’s an ongoing discussion. We’ll be starting to look into some metrics of some of the professional services companies as well and trying to better understand, you know, the productivity improvement. What does that mean? Can companies do more revenue, more profitable revenue with fewer people? And how do you measure that impact? And how much do you put that weight on AI versus utilization versus offshore leverage versus pricing, you know, all these other ways to think about it, right? So, I mean, we’re starting to kind of put some pulse on the market a little bit and trying to gauge of that direction, but it’s so, there’s so much happening in the meantime. So, it’s so much, it’s hard to, like you said, to really make it concrete, this is happening because of AI, like a complete, like, you know, an agent or AI specific number to pin it down.

Patrick: Yeah, actually, Boz, let me interrupt you for a quick second. I know where you’re going, but- and I want to interrupt you because Rob asked me this question over lunch in Sydney. What did I think about the idea of the sort of the zero human or the one human company where, sort of everything could be done by agents and you just have the one person. And I absolutely laughed it off and said it wasn’t possible. But now we’ve been working on this metric that Boz came up with to measure exactly what does that productivity gain look like. And I can tell you right now, it does not look like we’re heading towards the one human or the, Rob, what was the term you used for it?

Rob: A zero human company.

Patrick: Yeah. All right. I guess that was it. Yeah. So just want to say, just want to put it out there that we have been doing some further thinking on that and throwing some metrics at it, but definitely it certainly resonates. Boz, I know you had another question on a slightly different topic. You want to go ahead down that road.

Boz: Yeah.

Rob: Maybe before we go to that, sorry, and maybe a question to you, Boz: I’d just love to hear, how have you thought about building that metric? And what do the figures or initial findings kind of look like? I’m just very interested.

Boz: So, I think, so the metric, the way we’re looking at it, and it’s still kind of a little bit of like a preview for you and some of the listeners on the podcast, but it’s what we call human intensity reduction index, essentially. And it’s exactly kind of looking at the change, of the headcount understanding for the output for revenue, right? And trying to understand how much that, you know, is pure demand-driven versus AI. And as I was suggesting earlier, you know, it’s really, there’s a lot of factors right now into it, but early findings are showing the- if you look back in the three-year span when GenAI first came around, we’re starting to see, you know, coming from the index from -5 to -10%, which essentially tells you that was still companies relying heavily on humans, right, on labor arbitrage, traditional labor arbitrage, so now some of them crossing into mid to upper single digits, right, into 5 to 10% that index is going in a positive direction, right? 

So now we’re starting to think about, okay, what does that mean? How does that translate to economic value? And we’re trying to look into one economic value factor is profitability, right? More of a concrete measure of success of any of those kind of investments and initiatives that’s coming and go after. So, some companies are a little bit more conservative on changing and taking a hit on the operating profit because that kind of falls into their culture and just the way they are conditioned to operate. And you can see a very flat margin, which again tells you that at least they’re protecting the margins. Some of them are having a dip in their margins. And I think it’s a reflection of their little bit more of a aggressiveness for innovation and reinvestment back in the business. And as they’re trying to build some of their AI kind of IP asset portfolios that they maybe try to find new ways of monetizing that as they look into the next three to five years. And others are, just doing business as usual, right? And they’re more into a kind of an internal reorganization transformation phase more than actually doing, you know, something meaningful that, you know, it’s, I mean, you can look at the, just the raw numbers like, wow, they actually, the indexes are above 10%. But when you start peeling back, you know, and understand why the headcount reduction has been so much. And again, it’s more about that anticipation and trying to maybe offload some of the unprofitable units and the investitures and whatnot, that it’s less about AI, it’s more about making Wall Street happy for the time being, right? 

So, this is some of the nuances we’re seeing, but a lot is about leadership, about culture, about how those companies are, you know, conditioning their stakeholders. And there’s another factor that we’re starting to get a little bit better understanding is the impact and the influence of the alliances as well, because they operate in ecosystems, right? So, they have to account for the implications of the partners as well along the way, because they can’t just sell AI products in a vacuum, right? Because they have partners that they do the same thing. So, they have to be very careful how they position that with clients as well.

Rob: Understood. And one thing that’s always interesting is when we’re talking about kind of, do you keep employees or do you increase number of employees to try and take advantage of AI or potentially decrease, you know, if employees are more productive, is it then even more valuable to have more? I think when I chat with econs, they often say to me that labor, they traditionally view it as a game of comparative advantage, where as long as AI plus a person is more beneficial than just AI on its own, we should expect increases in hiring. We should expect your marginal profitability per person should increase. And so, you should actually want to hire more people is often the storyline that they talk to. Is that in alignment with, I guess, your initial findings from the index or? Would you disagree with that?

Boz: No, 100%. I think what the biggest test would be is for the, especially professional services companies, because I think they’re having the biggest kind of implication that AI is putting on their operating and the business models that they’ve been working with, you know, for hundreds of years, some of them, right? Is, you know, if they’re able to start driving revenue beyond the mid to upper mid-single digits, and they maybe crack into the low double-digit growth, I think that’s going to provide a good boost of confidence of what the strategy is. And while historically the headcount would have been expected to grow in parallel in a linear progression. I think this time around with the use of AI and the tools that everybody’s been investing, the net headcount growth will be more like in the low to mid to mid-single digits as long as the revenue grows in the upper mid-single, low double digit, right? So yes, the net effect of the AI is headcount will grow, albeit a slower rate than historical trends and rates, but I think there’ll be net positive effect on the headcount and the employment overall.

Rob: Very interesting. I’m keen to see when you release the full report.

Boz: Yeah.

Patrick: Timing is everything, right?

Boz: Timing is everything, yes.

Patrick: It may come out by the time this podcast comes out. We’ll see. We’ll see. 

I do just want to kind of wrap it up. Boz, I’m sorry, Boz, I know you had one more question. Throw that out there and then we’ll and then we’ll wrap things up, Rob. Go ahead, Boz.

What makes an AI lab or a client good to work with 

Boz: No, all good. Yeah. My question was because we’ve been talking about all the implications of AI and, you know, you’ve been in the field and you work with clients, you work with partners, you know, you’re kind of, you’re in the weeds so to speak, you know, in the AI. And just curious, I mean, what’s the, what’s the kind of the favorite AI company profile you like to work with? Not a fair question. I get that part, but just, you don’t have to name names. Just thinking about from a, it’s kind of like, what does a real good AI company look like in Rob’s world that you like to work with.

Rob: I think there’s probably two sides to that question. There’s which of the AI labs do I think build maybe better technology, if that’s the lens, or there’s which of our clients have attitudes that lead to the most success with AI? Do you want me to go into both or one of them?

Boz: Well, I’ll give you the consulting answer. Both, I guess. *laughs*

Rob: Yeah, happy to. I think the labs is a particularly interesting question. And I think more and more, a number of the really technical staff of my team, and myself included, are starting to form opinions based on the personalities of the models that they’re creating. There’s a great example at the moment where if you try and build an AI system from Anthropic versus you try and build an AI system from OpenAI, They have taken very different approaches to how you actually try and build reliable, high-confidence AI systems in that they have given their models different personalities from each other. OpenAI has taken this stance of, and you can go and read their documentation, they’ve taken the stance of, you need to be very prescriptive, is how I’ll put it. You need to be very prescriptive of how you’d like the model to act. And if there’s scenarios you don’t like, you just tell it on a prescriptive basis, don’t do X, right? And that’s really simple to work with, and it’s quite nice in that way. But the Anthropic guys have taken quite a different approach. They’ve taken what you would maybe call in a regulatory sense more principle-based, of I’ll give you the general governing principles, and I want you to go out and do these. And hey, Claude, generally, you are a very astute, hardworking, well-intentioned person. Figure out how you should do it in the right way, essentially. 

And so when we work on these AI systems, it’s getting harder and harder to build one system that works across multiple providers because they are taking kind of fundamentally, almost like Commonwealth law versus more kind of Americanized style of law – like a principled versus rules-based (prescriptive) approach – in how you actually instruct and teach these models. And not to say that either side is wrong or necessarily better or worse than the other, but they’re very different. And I think for individuals like myself, we probably have a bit of a preference towards one side or the other. And so, I’ve got a number of people on my team who really prefer the one way and a number of people who really prefer the other way. And I think we’re seeing more and more of that over time, which is that the differentiation isn’t necessarily in, well, my model can do X, and its score is 3% higher on this benchmark of impossible PhD questions that no one can even understand. But rather, my model is easier to work with in X scenario because the personality is like Y. And that allows it to be very easy to iterate and get reliability in this case scenario. So, I think the labs to me, I think obviously there’s lots of partnerships and relationships there. But more than that, it’s also they’re creating different things now, which is really exciting to me, because for a while it was all converged. They’re really creating different things now. And that is forming a lot of our preferences. 

On the client side of what makes a really awesome client to work with who I’m excited about is that you walk into their boardroom and you start talking about some of these things and they say, hey, I read that report. I read that economic index. I have been playing with this and we’ve actually, we were considering buying this small piece of software and now we’re actually, we just made it in two weeks and we’re not even going to bother going through a four-month-long procurement cycle because we can just make this thing that quickly. And previously, all these things that were, you know, we were putting them on a prioritization matrix and we were going to choose which of these projects we wanted to do. We’ve actually just, instead of doing that, we took the time it would take to have the call to do the prioritization and we just delivered the first project, right? Because that’s how fast we can move when we’re empowered with AI. I think those clients are the most exciting and fun to me because they aren’t held back. They understand the potential and they just want help in actually going out and getting it and achieving it or even accelerating it faster. And when you have an opportunity to work with them, you can really help them push the boundaries and you can get 100x benefit for them rather than just 3x benefit of setting up the initial AI foundations correctly, necessarily. So, I hope that answered both questions.

Boz: Yeah, it’s fascinating just to think about the pace of movement. And I agree, I mean, especially on the client sides, I mean, we look as well and the enterprises, there’s so many of them have been stuck in their own ways, but the ones that really want to do something, they’re not afraid to innovate, they’re not afraid to just to take on that next kind of idea and just embrace it and run with it as well. But the lab side, I didn’t think about it, but something for us to pursue and to go after a little bit more deeper, you know, just as the way you describe it and just think about how the evolution of the labs is going to impact the ecosystem as well.

Rob: Yeah, 100%. And I’m really excited to see where they go because it’s just such different approaches that we’re starting to see now where it was becoming almost commoditized for a while.

Patrick: That’s crazy, Rob, to say it was becoming almost commoditized for a while when this whole field is just still feels so fast moving, so new, so everything is changing all the time. And yet you can still talk about an element of it that was being commoditized recently. It’s crazy.

Rob: Yeah, look, I think it’s- it’s every layer of this field just changes so rapidly. And I think, you know, for a while we started seeing this, oh, the models are just commodities. And then we started getting these agentic harnesses, things like Claude Code and Codex, and then people started saying, okay, well, these things are just commodities. But when you look at these, the one lens I do really enjoy, and from Leopold’s situational awareness paper a few years ago, is almost like you take all these different elements from the model to the harness, to the training it had, to the data it had, to how many millions of hours of GPU compute and billions of dollars were spent on training. You kind of take all these things together, and you just plot them into a single capability line, essentially. So, you have a single metric for capability. And all of these things are essentially just different multipliers on that capability metric. And all of them have so much opportunity for exploration and they all have so much opportunity for improvement. And I think when you talk to people in the labs, the one thing you consistently hear is that just like if I want to increase revenue, I could work on making my business more efficient, or I could work on going out finding new clients, or I could work on global expansion, or I could work on a whole different bunch of areas. And I may find gains in all those areas. I think when you talk to people in the labs, they say we’re finding gains in all the areas, right? There’s 100 different areas and all of them are finding gains at the moment. And I think that is really exciting because the more areas they explore, the more diversity we’ll see in the market of how these different models act and operate. And that means there’s a lot more kind of innovation. And I think it’s a lot more exciting rather than a convergence. I think we’re going to keep seeing a bit more of that divergence.

Patrick: Yeah, that’s a great reminder. I forgot that not that long ago, we were sort of talking about how all the large language models were seemingly becoming so much alike. And you’re right, that opportunity now for innovation, that divergence is a lot stronger now, which convinces me we’re going to have to have this conversation again in six months on season six of TBR Talks and see where things stand. 

Final thoughts

Been really fascinating, Rob. I appreciate this time. I feel like we covered so much ground. You talked a lot at the beginning about reliability and you mentioned sort of the governance and the standard product management. And then this idea that people want augmentation from AI, like the clients that you’re talking to. And I think one of the reasons I was really excited to have you come on the podcast is I think you’re working with clients, with PwC’s clients on sort of a day-to-day basis. You really, you get your hands dirty in the AI at client sites. So, your reflections on what they’re actually asking for is just enormously valuable to us. So, I really appreciate that.

Rob: Of course, always happy to.

Patrick: And then I got to end with one last question, because that’s been sort of the theme of this season of TBR Talks. And if you’ve listened to some of the previous episodes, maybe you’ll know what it is. But right now, my daughter is 22, our youngest is 22. She’s graduating from university in a couple of months. Sort of the whole world is in front of her. And it made me in a previous conversation sort of ask some people who are my age or perhaps a little bit younger, but think about what they wanted to do when they were 22 years old. And one answer was a woman who is in technology, but not doing what she really wanted to do at 22, which was to be an engineer on a Formula One team. So now, of course, have to mention PwC does have a relationship with Formula One. Good for PwC, good for all of us. But anyway, so my daughter has no idea what she wants to do with her life. But at 22, Rob, what was your sort of, what did you think you wanted to do with your life? Because at the time you were messing around with actuarial tables at an insurance company, right? So.

Rob: Yeah, I think I’ve always been quite interested in the spectrum of AI. And I think one of the most interesting things was, and something that most people don’t realize is that if you went 10 years back and you went to the biggest AI conference in the world, no one there, not a single person, if you ask them, will we have anything even close to what we have today in 10 years time? Even if you ask them in 50 years time, will we have this? No one would have said, we’ll have this, right? This field has gone so much faster than anyone in the field ever thought it actually would. There’s this concept in the field called out of distribution, essentially, right? And people often today say, like, if AI doesn’t do exactly what I want at this task it’s never seen before, then it’s out of distribution. You know if AI wasn’t trained on this task and it doesn’t perform well. Like 10 years back, out of distribution meant to everyone in the field, it meant that when you use Google Translate, it didn’t have a literal word where this word in English means this word in French. That’s what it meant. Like, we had tables that went this word in English means this word in French, essentially. And we are now at this point where these models are just so incredibly capable and so much further than anyone thought they would be. And so, I was very excited about trying to get these models to even understand a sentence. That was one of my things I was really excited about, was even getting them to be able to understand the semantic meaning between words well. And it turns out that wasn’t that hard.

Patrick: *laughs*

Rob: So, I think I continue just to be really excited by the sheer amount of progress. I think I’m also very concerned about a lot of progress, if I’m being very frank. But I’ve always been interested in what we can actually achieve in this space, and it’s just gone beyond my comprehension of what I thought we’d ever get to. So, it’s very exciting, but it’s going to be a wild ride the next few years. That’s a guarantee.

Patrick: That’s fantastic and wild and scary. And I think we all definitely share your concerns. I’ll speak for Alex and Boz in saying that we definitely have our concerns about AI. Something we can dive into next time. Rob, thank you so much. Alex and Boz, thank you as well. And it’s really been a pleasure. Thank you, Rob.

Rob: No worries. The pleasure has all been mine. Thank you. Thanks, Patrick. Thanks, Alex. Thanks Boz. Lovely meeting you.

Patrick: Tune in next week for another episode of TBR Talks. 

Don’t forget to send us your key intelligence questions on business strategy, ecosystems, and management consulting through the form in the show notes below. Visit tbri.com to learn how we help tech companies, large and small, answer these questions with the research, data, and analysis that my guests bring to this conversation every week. Once again, I’m your host, Patrick Heffernan, Principal Analyst at TBR. Thanks for joining us and see you next week.

TBR Talks: Decoding Strategies and Ecosystems of the Globe’s Top Tech Firms

Join TBR Principal Analyst Patrick Heffernan weekly for conversations on disruptions in the broader technology ecosystem and answers to key intelligence questions TBR analysts hear from executives and business unit leaders among top IT professional services firms, IT vendors, and telecom vendors and operators.

“TBR Talks” is available on all major podcast platforms. Subscribe today!