Monzo Bank is strategically integrating AI tools into its engineering workflow by employing a data-driven evaluation process to ensure effective adoption, manage risks in a regulated industry, and address emerging bottlenecks like code review.
Mind Map
Click to expand
Click to explore the full interactive mind map • Zoom, pan, and navigate
One thing we've seen is that there's
definitely more pull requests per
engineer. And what was really
interesting to us was that the average
size of a pull request is higher. The
other thing we've noticed is that now
20% of our code that is produced is
produced by AI. And we found that really
interesting. We also seen a resurgence
in people mentioning time for review
being a bottleneck because reviewing
pull requests coming from AI can be much
more complicated and long just to do
because there's just more code. As of
now, we have a new bottleneck and a new
problem to solve.
>> Welcome to another episode of
Engineering Enablement. I'm Laura Taco,
your host this week. My guest today is
Fabian Dei from Monzo Bank, a
Londonbased bank that serves over 13
million customers and is scaling fast.
Fabian oversees multiple platform
engineering teams at Monzo, who own
everything from CI/CD for backend and
mobile, mobile releases, feature flags,
and of course, AI tooling. Before he was
at Monzo, Fabian worked at Spotify on
backstage and developer experience.
Fabian, thank you so much for joining us today.
today.
>> Thank you very much. Thank you for
having me. Longtime listener of the
show, so very pleased to be here.
>> Yeah, well, we're we're pleased to have
you. Could you start by telling the
audience a little bit about Monzo for
those of them that haven't heard about
Monzo before?
>> Sure. So, Monzo is a digital bank in the
UK. We have more than 30 million
customers here and we're really proud of
what we're building, very new. Um and
yeah, we're we're coming to Europe now.
>> Very excited to hear that. Can you share
a little bit about your role? So, you're
responsible for several platform
engineering teams, but um you know, what
does the scope of your role look like?
>> Uh you look after four teams at the
moment, which are looking around what we
call developer experience, but there's
touching many different parts. So, for
example, we've got the mobile platform
which looks after our release and
everything to do with our release apps
and performance. Um we also have two
teams dedicated to developer experience
and teams about AI AI enablement. We
call it augmentation engineering.
>> Ah that's a good that's a good name for
it. AI augmented augmentation
engineering. To start our conversation
Fabian we're going to talk a lot about
AI. We'll talk about datadriven
evaluations of AI tools and talk about
some of the downstream impact that it's
having on your engineering organization
and your business. To start with, Monzo
is a bank. You It's a tech company that
also does banking. We could say you're a
modern bank, but a bank nonetheless. You
have a lot of sensitive information,
personal information. It's a highly
regulated industry. Can you share a
little bit about your approach to AI and
your AI strategy in a regulated industry
like banking where, you know, sometimes
those those things don't seem to go
together so well, innovation and and
regulation, but you've done it in a way
that's really working for you. Can can
you talk a bit about that?
>> I think what's important for us is that
us as being the platform group for all
the engineers at Monzo, we can provide
with provide tools to everyone. Um, and
by centralizing the kind of um making
these tools available, we can pro we can
make sure we have guardrails in place
when it comes to like PII, client data,
all of that. That's like top of mind for
all of us. So we're not going to make
all the tools available and going to
tell everyone you can use it on whatever
you want. Uh we're going to make sure
that it's only applicable to certain
domains to only certain people who've
been doing some training. That doesn't
mean we're not looking at expanding
these tools, but we want to make sure
that we have the things well in order
before expanding. Um, and we're also a
tech company as you said, so we also
want to make sure like we are not
falling behind the curve and we know all
of our competitors are looking at these
tools and we've done the same um
something like a year ago where we
looked at ourselves and wondered should
we just bring all those tools in or
should we wait a little bit? We decided
that the best approach was to get the
tools in and go through a trial with
some evaluation criteria and just be
very deliberate about like what are we
trying to achieve with these tools,
where are we going and which ones are
the one that we want to have. Um it's
also not about just looking at what
works for the others. It's about
understanding what works best for us
because we have a a very specific
context. We're as I you said a regulated
industry. We also have a monor repo. uh
we've we've got a few different language
uh go uh swift cotlin so all of these
things are kind of parameters for how AI
needs to work for us yeah I think that's
such um a an example of how actually
your regulation and the constraints that
you're operating in have forced you to
think very deliberately about every
choice that you're making with AI which
in my experience with companies that I
speak with has led to in a lot of cases
better outcomes because the AI tooling
and evaluations are a lot more targeted
at very specific problems and there's a
lot more change management around kind
of how they're being rolled out which
turns out to be an advantage in the long
term even if it feels like a
disadvantage in the short term because
you hear a lot uh on LinkedIn or in in
other headlines about how fast other
companies are moving but you know to
your point Fabian making sure it's the
right all for your circumstance and you
know every company is a little bit
unique. So I think that's a very good
grain of advice for for those listening.
Let's rewind
two years ago. Let's talk about where
you started with AI. I know it was with
Copilot. Can you talk us through kind of
the very early days of Monzo and AI
augmented engineering? So um I think
like many other companies our journey
with AI started with GitHub turning on
copilot for us and suddenly this landed
in our VS code and that was quite
interesting and nice and a lot of people
starting trying it but there was not a
lot of adoption not everybody was keen
and also they didn't really know what to
do with it um and I think from there we
started being more and more curious and
I think our leadership was really keen
to make sure that we are investing into
bringing these tools in um and we were
we were really empowered in the platform
uh collective to make sure that we can
bring these tools in but also make it in
a very structured way. So that's uh
that's what we did.
>> Let's talk about some of that structure
because datadriven evaluations for AI
tools are it's it's so critical. Can you
share why that was the approach that you
took? You know why was this structured
evaluation with really clear criteria so
important for your AI strategy?
>> We are like very data driven at Monzo
and any decision we make we need to make
sure that it is the right one and what
is best other than data to make this
decision. So we wanted to make sure that
even before we started any trial we had
a evolution criteria in mind so that we
could compare all the different tools
that we're going to try. So we've tried
three tools since then, cursor, wind
surf, and clo code. And we've applied a
set of evolution criteria for all of
these. Um, and this has been extremely
helpful because at the end of the trial,
we've got our evolution criteria, we've
got our values, our numbers, and we
could decide whether we want to continue
with it or not. And um, it was very easy
to convince anyone that could question
or that wanted to say, "Oh, this tool is
great. I've used it on my personal time.
We should definitely use it." And then
we look at the data and we can say, "Oh,
no, look at that. there's some things
that don't match up with what you're thinking.
thinking.
>> So having that evolution criteria has
been extremely helpful for us
>> and some of the criteria we've looked at
that was really helpful. So, um, we've
got a list of like 10 to 15 criterias
around adoption, uh, the weekly active
and the monthly active users, retention,
which is super important for me cuz I
see a lot of people wanted to try the
tool, then they'll be counted as part as
weekly active users, but then they're
just going to drop the tool cuz it's not
really helpful. So, having the retention
uh, criteria was really helpful.
We've been looking at the number of
lines of code written by the AI and the
percentage of code written by the AI.
We've been looking at the acceptance
rate of suggestions, the satisfaction of
our engineers. That has been really
really important. Um, we've looked at
the different use cases that it covers
and whether it's good at certain things
rather than others. So bug fixing,
documentation, large migration,
refactoring, developing new features
based on templates, all of these things.
We wanted to make sure we had detailed
feedback on all of these. Um, we also
looked at the company health and also
the just the cost of the tool because
some of these tools can be quite
expensive if you use them quite a lot.
So we want to make sure that there's a
good return on investment and we
understand where we're spending our
money. Yeah, thank you so much for
sharing that with with me and with the
audience out there. I know that
datadriven evaluations and understanding
how should I even be approaching making
it making a purchasing decision is
something so many leaders are in the
thick of right now really kind of
struggling to find an approach. So I
think your guidance will be really
helpful for them. I want to get into
specifics on some of these things. So
first let's start with the tools themselves.
themselves.
you so you're part of the platform
collective. You mentioned that you're
you're triing cloud code cursor and
windsurf. Who picked those tools? Was
that something the platform engineering
team or your platform engineering
collective was responsible for? Could
developers nominate tools for trials?
Like how did you land on those being the
ones that you were triing?
>> It's an interesting mix of both. So
we're listening to what people are
suggesting. We're looking at the market
and we're talking to some of our friends
and at other companies and trying to
understand where this is going.
Obviously, the market is moving really
quickly and I remember we started a
trial and suddenly there was another
tool that everybody wanted to try. Um,
but we also wanted to make sure we
sequence the trials not to have too much
overlap as to otherwise it could bias
some of our data.
So it's it's really a mix of what our
engineers and leaders have been asking
about uh but also us going more deeply
to try to understand if this is
something that could work for us in our
context. Uh we also want to make sure we
can cut for the largest use case. Uh so
we've looked at tools that could help
more specifically iOS engineer for
example. um which would be great because
they don't always have the best tooling
in their day-to-day life. But now might
not be the best time for us to invest in
a tool that can only serve 10% of our
engineering force. Uh when we look at
tools like cloud code, the great thing
with it is that it can integrate as many
different workflows and ids and that
makes it very easy to um like adopt and
start using. So there's also a question
of which tool and we've had to reject
some proposal of people wanting to do a
trial just because we believe that it's
not the best investment of our time.
We're not we don't have an unlimited
budget and a very large team that works
on like bringing these tools in, but we
are much more interested about making
sure we can bring tools that can help
the majority of our engineers and we can
invest in the tool because I think it's
one thing to bring a tool in, it's
another thing to make sure that the tool
is useful and understand all of the
context at Monzo. So we've in invested a
lot into tooling and code files and
workflows and and all of the markdown
files that really guide the models.
>> You know some organizations take the
approach that a developer just gets a
stipend and they can choose whatever
tool they want in which case you know
your iOS engineers could be picking
something different from your you know
your go engineers based on whatever you
know whatever preferences they have
there. their own workflows and and use
cases that they that they need to to
have covered. At Monzo though, because
you are, you know, operating at scale
and scaling, you're in a highly
regulated industry. When you talk about
bringing in a tool, you're not talking
about just developers being approved to
use it. You're really talking about like
an organizational
decision to support this tool, train
people, enable them how to use it. as
you said all the MD files and all the um
you know context gathering for this tool
to really integrate it in the workflow.
Um am I getting that right?
>> Yes, that's absolutely correct. Um
another example of that is how we use
MCP like it be it would be very easy for
us to just enable all of the MCP
integrations and get models to access
our BigQuery data and our notion data
and our Jira data and whatever tool you
want. But some of these tools might have
some PII or there might be things that
are confidential that we don't want to
share. So our approach by default is not
to enable that. But we're creating an
abstraction on top of all of these what
we call knowledge stores and we are just
allow listing the different things that
we want the modules to be able to
access. Uh so something that we've done
for example is we've got a lot of like
reference documentation as part of our
onboarding uh paved path or golden paths
however you want to call them and we
want to make sure that the um the models
know how to access them because it's by
ensuring that model can access the Monzo
context it's going to make it 10 times
better at doing our day-to-day job and
>> I think the first reaction of a lot of
engineers we've tried these tools is
like, oh, it's pretty good, but I had to
tweak the code to make it the Monzo way.
Um, and that's because the tool doesn't
know about it. So, it's not going to be
magic magically picking things up. And
because we are working on a monor repo,
there's quite a lot of context if you
were to push the entire monor repo
there. So for us having markdown files
that can guide models and having an MCP
that can open some like um rag workflow
for the models that's been extremely
important and that's been very
successful as well.
>> Yeah. So this is really about
transforming the ways that software gets
made at Monzo not just about an
individual developer having you know
three different AI tools that they can
use however they want for their own
individual tasks. This is really about
changing the way that your organization
works, which is why data is so important
and why you need the validation of the
use case and of the impact so that when
you do scale it, first of all, it's an
investment of money, but also time,
resources, training, um that you're
really making a solid decision.
>> Yeah, definitely. So important to make
sure that it's not just bring the tool
in and forget about it. it has to be
continuous um involvement and it's also
not just about one tool like if we can
work on tools and that's why MCB I think
is really great is it can interact with
all the different models and all the
different tools so that that's really a
great value for us
>> I want to get back to some of the
metrics that you mentioned before about
actually tactically how you are
evaluating these tools and get into some
specifics so you mentioned a couple
things like adoption rate, percentage of
code written with AI, percentage of PRs
written with AI, satisfaction, company
health. What are you actually using to
measure these things both during the
trial period but also beyond the trial
period? Are you using system data? Are
you using self-reported metrics? Like
what have you had to to put together in
order to get this to get a field of view
on the impact of these AI tools at
Monzo? some of these criteria that we
get the data straight from the tool
providers. So things like active users,
lines of code return, sometimes
acceptance rate as well, that's the kind
of thing that they like to put in front
of us and credit usage and cost as well.
That's very easy for us to gather. For
the other one, it's mainly uh survey
data. Um so we we send that to all the
users. We have the list of users. We we
send a short survey and we've used the
same template for all the different
tools again just to make sure we we're
doing this the right way. Yeah, it's
been it's been pretty good because we
can then dig into some details area like
I talk about satisfaction and we have
one satisfaction but we also dig into
the different use case and try to say um
how good is tool X for refactoring how
good is tool X for large migration how
did you find tool X at explaining code
so all of that help us dig into the
details and then um we're running
quarterly developer experience uh
surveys as And we've started adding some
like more AI taillet questions to that.
Example, one thing that was really
interesting is that we still only have
half of our engineers who feel confident
to use AI engineering tools on a
day-to-day bas basis. So it means it
shows us that we still have a long way
to go.
>> So even if the adoption rate and we can
see people are using it maybe weekly or
monthly, that's not the same as they
they feel confident, they feel like it's
really helping them, they trust the
output. Um and then from there you can
make strategic decisions about investing in
in
training enablement targeting other use
cases doing other things. So I can I
imagine that data is really helpful for
that continued continued support of the
tool and and helping people stay on
boarded as users.
>> Exactly. That's like it really points to
us where do we need to invest like do we
still need to invest in upskilling
people? Do we need to invest in more
tools? I think at the moment we're in a
position where we know that we have good
tools in but we just need the engineers
to feel more comfortable about it.
>> Yeah. So you mentioned uh doing
evaluations of a couple tools, cloud
code, cursor, windsurf for an individual
developer. Do they have access to
multiple trial tools at the same time or
do you segment who gets access to which
tools? So you're kind of comparing
different cohorts to different cohorts
or does does is there ever the the
experience of one developer has access
to a bunch of tools and then they pick
their favorite?
>> Yeah, that's a really great question.
And um what we've tried to do is first
sequence how we bring these tools in
just because we want to avoid uh like
overloading people with new things that
already the pace of change in AI is is
very high and we want to avoid people
feeling drowned or even having a fear of
missing out. That's also something we're
very conscious about. It's not because
we bring these tools in that everybody
needs to drop what they're doing and
jump on these things. We've had a good
learning story when we did these trials
cuz the the first one we did we
>> gave the tool to as many engineers as
possible and it was almost free for
anyone to use. Uh but we quickly
realized that there was a lot of people
who were interested but didn't have the
time to invest. couldn't give us any
good feedback and it felt a little bit
disappointing is that in that we spent
so much time trying to enable 400 450
engineers and only 100
150 would use it and then 50 would give
us feedback. Um so we decided then to
change our approach in future trials to
just start with a small cohort of people
that have a very welldefined use case.
So they might be working on a migration.
They might just be very interested. Um
we have a group called the AI
engineering champions which are just in
different parts of Monzo uh looking at
different part of the product. Having
couple of engineers who are very keen on
leveraging these tools but also be the
relay for the rest of their collective
to present their learnings. That was
very important to have them engage so
they can relay the information.
>> And starting with these group of like
smaller number of people who can really
push the boundaries of these tools
>> made us realize that some tools might
look great on the surface but when you
dig into actually using it on real
project for real use case there are some
limitations. Um so having this co this
cohort helping us evaluate and give us
better kind of feedback or more more
qualitative feedback was really helpful
for the subsequent trials that we've run.
run.
>> How are you getting that
information from those champions then
scaled across the rest of the
organization because I imagine that's
also like that advocacy part is part of
that responsibility of being an AI
champion. Yeah, it's some it's something
we haven't yet fully invested in. Um
because in our the way we look at
things, we believe that first we wanted
to bring some tools and and try them and
get our hands dirty. The second phase is
to invest and double down on these tools
by building all the tooling around it,
all the safety nets around regulation,
the guardrails that we want to have. And
then the last phase is we want to start
upskilling people. And we're just
getting there. But part of our strategy
is to communicate very often whether
it's like slack channels or all hands
announcements around what are the things
we're doing and what are the success
stories as well. I think it's really
motivating for people to hear that one
of their colleague tried the tool for a
particular use case and it really
worked. And what I see is that there's
there's a lot of people who are not part
of the champions but who just
>> did it just because they feel that it
was really productive and they just want
to share it with the rest of the
organization. And I find that really
great. But I also know that it will not
be will not be able to scale that model
for everyone. So we'll definitely bring
some more structured learning looking at
like prompting engineering for example.
I think that's something really
important. It's not really yet part of
our um like on boarding tutorials at
Monzo. So, it's not something we're
going to look at in the next six months
for sure.
>> Yeah. A lot of exploration. It sounds
like a lot of experimentation happening
trying to figure out where the high
leverage use cases are and then you know
nailing it really well and then scaling
it across your organization seems to be
your approach.
>> Definitely. And there's some good we've
got some good data around that because
when we look at AI usage and also
spending we see that we have a minority
of users who are the biggest spenders um
and the curve it start with a high peak
of like the top 10 users will spend
>> 50% or more of the of the money and then
it really decrease and then there's a
long tail
>> and what we want to see is that for that
curve to become a little bit more
linear. Uh, so yes, there always going
to be high spenders, people that are
using it more, but we want that critical
mass of the kind of 25 to 75 to be
spending a lot more and just being able
to be more comfortable with these tools.
And the great thing is we have that data already.
already.
>> Yeah. How and how are you getting that
data you're getting directly from the
tools themselves, right? The telemetry
has sort of matured a lot in the last 12
months when it comes to what you can get
from the tools. And I think usage and
cost spending data is one of the areas
that there's a lot more visibility into
than 12 months ago for example.
>> Yeah, definitely. And we also conducted
um a bunch of one-to-one interviews with
our most uh costly users or I'll call
them power users.
>> That's much nicer. It's really really
interesting to know what they're doing
because they're like very advanced in
the curve of what you can do with
engineering uh with AI in engineering.
There are also some people who just said
you know I I was just trying to compare
the different models. So yes there might
have been a lot of like spending for
that time but I think it's not going to
be always like that.
>> It was an experiment for a purpose right.
right.
>> Yeah but that person has so much
learning that we actually shared what
they've learned to the rest of the
company and I think that's really going
to help everyone.
>> I want to talk a little bit about the
spending and budget aspect. I mean,
Monzo is a bank, so it's probably not
unusual for you all to be thinking about
pounds and cents and and other things.
You know, cost control, I think, is
something that is on a lot of people's
minds. Maybe they're not actively doing
anything about it, but 2026 is not going
to see, I think, AI costs reduce, right?
We should be expecting to spend more.
Can you talk about how the cost piece of
it factors into your evaluation
criteria? like are you you have a budget
in mind for AI tool spend per developer?
Like how are you approaching bringing in
all of these tools, saving room for
experiment, but also not blowing the top
off of of the budget that you have for R&D?
R&D?
>> Yeah. Um that's something where we
started like a year ago. We had no idea
and we looked at what can what would be
sensible, how much would be would be
comfortable spending and we started with
that. But very quickly realized that
number was not really helpful. Um I
think what helped us to land on a figure
was to actually put the tool in the
hands of our engineers and seeing are
they satisfied with it and how much does
it cost. Um and we've landed on a figure
of um $1,000 per engineer per year at
the moment. And that's something we're
comfortable spending and we know we're
not there yet but we're close to that.
Um and that's something that has been
very transparent from the beginning up
to our our CTO for example we were
discussing with them and saying like
would you be comfortable if we spend
that much and uh they said yes and I
think we we had some rational behind it
it's not just a random number in the air
but we are thinking that a typical user
would use that many requests per day and
all these things
>> then agents are coming that's yet
another unknown and similar I think with
agents we're going to be spending more
but we'll take a similar approach of
saying we expect to see these gains from
agents and these gains will translate to
these different cost but I think until
we have real data it's difficult to project
project
>> and just for some kind of benchmarking
generally what we're seeing in the
conversations we're having with
engineering leaders as well as the a
poll we did during a recent webinar
about budgeting, we had about 200 plus
folks respond, how much money are you
spending per engineer? We had the
majority of people falling into that
500, a thousand and then, you know, ta
tailing off as we got above a thousand.
But that's also my general
general generic guidance is sticking
around that $1,000 mark per engineer per
year. You know, GitHub copilot
enterprise license is about $500 per
developer per year. And you know, I
think like Monzo, a lot of companies and
many companies that I talked to, I would
say most of them are taking a
multi-bender approach where there's a
lot of different use cases and tools are
specialized still. We don't have one
tool that's going to cover every single
use case. And so it's important to keep
keep uh room for experimentation,
challengers to incumbent tools, kind of
keeping options open, keeping options
open, especially as this ecosystem is
evolving so fast. Um just so fast it's
hard to keep keep track.
>> Yeah. Very tricky to know what's going
to happen in 3 months.
>> Absolutely. And one of the things about
cost which leads me into my next
question is about return on investment.
So we can look and say, okay, we're
comfortable spending up to $1,000 per
developer per year, but we also want to
see the results coming back to the
business. And so Fabian, I wanted to ask
you about, you know, at Monzo kind of
generally speaking, what are some of the
results that you're seeing from using AI
right now in terms of, you know,
throughput, speed, quality, those kinds
of engineering performance metrics? What
has been the the change or the shift
with AI, augmented engineering?
collected some data very recently on
that. So I'm very happy to be able to
talk about them. Um one thing we've seen
is that there's definitely more pull
requests per engineer. Not not 100% more
but something around the 10 10 to 20%
more PRs per engineers. And what was
really interesting to us was that the
average size of the pull request is
higher. Um so it's like 20% more um
larger pull requests in terms of size.
Uh which I think was really really
interesting. The other thing we've
noticed is that now 20% of our code that
is produced according to um quantit
qualitative data is produced by AI. Um,
and that we found that really
interesting. And I think on our latest
developer experience survey, we also
seen a resurgence in people complaining
or mentioning time for review being a bottleneck.
bottleneck.
>> Okay. Because reviewing AI or pull
requests coming from AI can be much more
I mean complicated and long just to to
do because there's just more code. Uh so
we've we found that now we have a new
bottleneck and a new new problem to solve.
solve.
>> Yeah. So you're having not not just more
PRs per engineer but also the size of
the PRs per engineer is growing. And so
that's you know kind of two two things
um that lead to an increased bottleneck
in in code review. Are you using AI
tools for code review? Right now
>> we're using it but we're using it as
optin. Um it's interesting because at
some point we tried um having it like a
blanket thing on all of our pro requests
we would have an AI review and we we did
some survey again after two weeks and
realized that a lot of people thought it
was like not a great investment was
really good at certain things but also
created created a lot of noise so we did
decided to make it optin.
>> Talking about bottlenecks we mentioned
code review being a bottleneck. One of
the things you mentioned as well was
developers trying to get their changes
into a dev environment for them to test
and now we have more code more
frequently. This was an existing
bottleneck that you had and AI is just
exacerbating the problem making it
worse. But you came up with a solution
which is these preview environments.
You're calling them tendencies. Can you
talk a little bit about how that project
came to be and you know how it's helping
developers get more valuable work done?
Yeah, that's been a very interesting
journey that started roughly at the same
time when we started to adopt AI. So the
bottleneck was already there and we saw
that through our developer experience
surveys and we knew it was a problem and
we spent a lot of time thinking about
what would be the right solution because
at Monzo at the moment we have a dev
environment and a production
environment. We don't have anything in between.
between.
>> Mhm. And then we could have just said oh
let's create another environment like a
staging integration environment or
something like that but that would just
move the bottleneck somewhere else I feel
feel
>> um and we really went back to our first
principles and trying to understand what
are the engineering try to achieve
basically they want to test their code
in an environment
um but also don't want to go through the
process of to emerge and wait for the
deploy and and and do all of that so
they would just straight deploy without
reviews or anything which would create a
lot of instability in our dev
environment. So what we did there was um
as you said created preview environments
which allows us to route some of our
request through our dev environment to
this tenency which would be um per user
and then the the user would deploy their
service to that tenency and then we
would route the traffic straight to that
service. And that allows us to scale
really really well because now we have
one per user and our dev environment is
a lot less stretched. And some of the
side effect we saw around that was we
had a lot of flaky tests for example
because we were running tests on that
environment that was constantly changing
and that state was mutating a lot and
and so we've seen a lot of improvements
on that area already and we're very glad
that we're building that now and we've
got that available because with AI being
here and soon AI agents being able to
test their changes on these environments
that's going to be fantastic. So you
know AI can exacerbate and really
highlight some of the existing
bottlenecks as you said you had this
bottleneck even before AI came and
thankfully you know you have such
platform principles at Monzo thinking
about scaling this and addressing the
bottleneck which is also helping you
accelerate AI. This is now groundwork
preparation for agents being able to
create and then test and verify their
own changes in these sort of ephemeral
environments and you already have the
scaling built in. So that's a huge uh a
huge accelerant multiplier for you all
by having that really good plumbing and
that platform uh platform mindset to
start with that will be very cool to see
in the next 12 months how this all unfolds.
unfolds.
>> Yeah, exactly. That's very exciting
future. I wanted to ask you as we start
to wrap up here is not just developers
but also other folks are using AI um and
we're sort of expanding the definition
maybe of like who maybe not who you know
who is a developer but you know where
where the work gets done who where it
gets handed off. You mentioned before
about product managers and designers now
having access to more access to AI tools
in order to get a bit further along in
prototyping and validation before that
work would maybe be transitioned to a
developer team. Can you talk a little
bit about that approach because it's um
I think it's a really great illustration
of using AI not just to speed up
individual coding but to really change
the way that software gets made at Monzo
in general. It's been a fantastic
thing to observe this change because it
came organically from our designers, our
product managers, and other disciplines
really at Monzo who just reach out to us
saying, "Hey, I know you're um in charge
of tool X or tool Y. Could you help us
like get on board because we have these
great things we want to achieve." Um,
and we've seen some fantastic results
where all of our designers have been
able to try the tool and prototype
things directly. And the feedback we got
from them is fantastic because they say
that the the prototype they create is
much more much better quality and it's
much much faster to get it out. Um but
in interestingly and similarly to what
we've discussed before, we need to make
sure that we give the tool the or we
give the LLMs the right tools to work
well at Monzo. So for our designers, we
have a design system with with building
blocks for what does a button look like,
what are the different tools in terms of
spacing look like and everything and all
of that is available to the tools. So
when they create the prototypes, all of
that is pretty much instantly created.
Um, and that that that was fantastic to
to see and to observe. But I think the
best aspect for me as an engineer is to
see that we're blurring the line between
an engineer and other discipline and
where do we create the value of like
these products. Um, I think the pass
from an ID to a prototype to something
that can be passed on to end engineer
has widely changed in the past few
months and I think it's going to speed
up the way we can uh develop and ship
features at Monzo for sure.
>> Yeah, that's exciting to hear. It's
exciting to hear. Two things to ask you
as we close out our conversation. first
for companies that are maybe feeling
stuck. They're making these tools
available and they're noticing like you
did in early days, adoption is just not
what they expected. The the value or
impact is isn't what they expected. What
is your advice to companies who are
trying to use AI as a way to really
change the way work gets done, move the
needle for the business, and not just be
a fun, enjoyable thing for engineers to
to do? What advice do you have for them?
Yeah, I'd say there's multiple advice
because it's quite a complex thing. I'd
say don't don't limit yourself to just
making these tools available. There's
going to be a lot of things that comes
with it. You you'll need to share
examples that work. You need to build a
foundation to make sure people can
leverage it and it it is efficient for
their context. You need to share the
prompts that you've used successfully.
You need to share success stories and
you need to build communities as well. I
think it's very important to be able to
have relays in different part of your
organization who understand what you're
building and can relay your message when
you're saying that there's a new tool
available, you've built an MCP for them
or there's something else that that is
coming, new features, new models that
are really good at. Um, and I think
related to that, bringing a tool in your
company is is quite an investment. It's
not something that you just bring it,
put it on the shelf, and people can just
self-s serve. You're going to have to
invest in it to make sure that it work.
So, don't try to bring all the tools at
once. I don't think that's going to work
well. Make sure you're very like make
sure you have a evaluation criteras or um
um
you you have ways to understand what's
going to work for you, what's not going
to work and and be, you know, just happy
to say no to certain tools if it doesn't
meet the threshold that you've defined.
Um invest where you think it's best. Um
and yeah, good luck because uh it keeps
changing. So it's a it's a tricky domain
to be in for sure, but super exciting.
>> Yeah, really exciting. Things are
changing really fast. When it comes to
AI and the tooling ecosystem, what do
you think will be the biggest changes
that we'll see in the next 12 months?
>> I think for me there's something around
the agents and how engineer are going to
use and leverage agents in the future.
We definitely can picture a word where
as an engineer you're actually leading a
team of agents that are specialized to
do very different task. So you might
have your unit test engineer or your
unit test model or tool or you and your
refactoring one and you have your
architecture one and you might have your
Monzo X domain expert one which knows
all about finance and all these things.
And for me the big challenge and where I
haven't seen any tools around that yet
that does it greatly is the
orchestration of all of that. Um I
believe that's definitely something
that's going to be very powerful because
these tools are going to be here. Um and
they are going to be even better at what
they're doing. But for engineers or
developers to leverage that to their
maximum, I think we'll need something in
between where you won't have to give all
of that context and say, "Oh, look at
that file here and this is the
instruction for unit test." So all of
that kind of manual linking that
engineers still have to do sometimes. I
think there's an orchestration tool here
that can really help with with that and
>> make it very easy for everyone.
I bet there is a platform engineering
team out there working on tooling like
this that will I don't know open source
it or create it make it available
somehow but I think you're right it's
that orchestration layer we just want to
declare what we want and get it and I
think at the heart that's really that's
really the role of platform engineering
right and developer productivity
engineering devx engineering is
connecting the dots you know
centralizing the the complexity so that
we can abstract it away from the
developer in in a sensible way. But uh
that will be really interesting. Um
wonderful Fabian, thanks so much for
sharing your your guidance on datadriven
AI tooling evaluations, how that's
looked for Monzo, some of the the some
of the impact that you're seeing
already, and then some advice to
engineering leaders. I think it's been
really helpful for those listening out
there. And thank you very much for for
joining us.
Click on any text or timestamp to jump to that moment in the video
Share:
Most transcripts ready in under 5 seconds
One-Click Copy125+ LanguagesSearch ContentJump to Timestamps
Paste YouTube URL
Enter any YouTube video link to get the full transcript
Transcript Extraction Form
Most transcripts ready in under 5 seconds
Get Our Chrome Extension
Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.
