0:03 Good morning. Well,
0:03 Well,
0:05 >> that's great. I love this crowd. Well,
0:08 welcome to reinvent in Las Vegas. I'm
0:11 excited to be here with you today. Uh,
0:14 this is my 13th reinvent. I attended
0:17 first six like many of you in this room
0:20 as a customer of AWS and this is my
0:23 seventh one as a builder here at AWS.
0:27 Now, in my role at AWS, I wear dual hat.
0:30 I get to lead and build and solve
0:32 business problem using technology and AI
0:34 and I also get to work with seuite and
0:36 executives of some of our largest
0:39 customers like yourselves. How many of
0:40 you in the room are first time at
0:44 reinvent? Show a hand. Well, wow. Many
0:46 welcome. Any veterans here? Five plus years.
0:48 years.
0:50 Excellent. Well, I hope you have a
0:52 fantastic week uh because it's an
0:54 amazing time to be a senior leader in
0:57 the organization. What a privilege that
0:59 we all have to become the agent of
1:03 change and lead our organizations beyond
1:06 just automation to high agency. So in
1:08 this session over next 40 minutes or so
1:11 I'm going to talk about what do leaders
1:14 need to do to lead in this new era of
1:16 high agency. What are the mental model
1:18 shifts that we require as leadership
1:21 team? How do we reinvision our business
1:24 processes? and then go slightly deeper
1:25 into some of the technical and
1:28 architectural capability that help bring
1:31 this vision to life. So let's dive in.
1:34 Now I'm sure many of you have seen lot
1:37 of things being labeled as agents. But
1:39 when you peel back that wrapper
1:43 underneath that you find many of these
1:47 things. You have just good old scripted
1:49 automation that takes repetitive
1:52 predictable task and in a fixed set of
1:56 workflows codes it as you move up that
1:58 autonomy chain you have generative AI
2:00 assistance and the chat bots that
2:03 provide useful information has access to
2:07 data can respond to queries uh can
2:08 actually summarize the document and
2:10 information and has an ability to
2:14 respond but very limited ability to act.
2:17 As you move up that autonomy chain, we
2:19 start to see higher order business task
2:22 and the goal that these agents can do.
2:24 The goal and taskbased agents are
2:28 optimized for completing a specific task
2:30 by working with humans.
2:32 This is where we start to see lot higher
2:36 value. And then at pretty high up is the
2:38 highest level of autonomy of agentic
2:41 system where multiple agents work
2:43 together, take a much higher order
2:46 complex and sometimes ambiguous task and
2:49 are able to break that down into steps
2:51 and pursue that. So this is a good
2:54 mental model to think about when your
2:56 teams come and talk to you about
2:59 automation or AI agents. Now this is not
3:00 to say that one is better than the
3:02 other. As senior leaders, it is
3:05 important that we understand what tool
3:08 to apply to solve which problem. So the
3:10 question that I really find useful when
3:13 I'm talking to my team is to what are
3:16 the behaviors or attributes that makes
3:19 something an agentic system. So let's
3:21 talk about what are some of those behaviors.
3:23 behaviors.
3:26 Agentic systems are goal- driven. As I
3:28 talked about, they take a higher order
3:31 goal or an intent and have an ability to
3:34 break that into multiple chunk chunks of
3:36 tasks to go and complete. They are
3:38 resourceful. They have an ability to
3:41 access the tools, the data, the context,
3:43 the hierarchy and the roles inside of
3:46 your organization to take those action.
3:48 They remember, they have a memory so
3:51 that you don't start again when you have
3:53 a new transaction. They remember that
3:55 there was an issue with the billing last
3:57 time I tried to do something. They learn
4:00 and adapt. They get better with the
4:04 feedback and they use escalation not
4:06 just as a mechanism when something is
4:08 wrong but as an inbuilt trust mechanism
4:11 to invol human in the loop. So it's good
4:14 to keep this framing in mind when you
4:17 think about agentic system.
4:20 Now you might ask but we've been on this
4:22 journey for a couple of years now. So
4:25 why now? What has changed? And there are
4:27 two fundamental shifts that are
4:30 happening very very rapidly that we as
4:33 senior leaders need to pay attention to.
4:35 The first thing that is happening is
4:38 that the unit of task or the length of
4:42 task that AI can do is doubling roughly
4:45 every seven months or so. So you now can
4:48 take lot more complex task and give
4:51 agents to go and accomplish those task.
4:55 Now as senior leaders we know that just
4:57 having a technology or tool that is very
4:59 intelligent and highly capable but is
5:02 extremely expensive and unaffordable is
5:05 not very useful is it? So what is also
5:07 happening on the other end is that the
5:10 unit price or the cost to access this
5:12 intelligence is continuously dropping as well.
5:14 well.
5:17 MMLU is a standard benchmark massive
5:19 multil- language understand massive
5:21 multil- language task understanding and
5:24 that is used to benchmark language
5:26 models and just to give you the context
5:30 an MML of 43 is similar to a struggling
5:33 high schooler they mostly are guessing
5:36 answers but they are very confident
5:38 uh I have one at home so I know how that
5:41 feels like I'm sure many of you do too u
5:45 but MML of 83 is like a PhD level
5:47 expert. They understand the nuance. They
5:50 are able to deal with ambiguity. And so
5:53 we can see that MMLU and the unit price
5:56 to access both of this is dropping. So
5:58 we are in the sweet spot where the
6:00 capability are increasing and the unit
6:03 price and the cost is dropping. And this
6:07 is why we are starting to see real world
6:09 applications of agents.
6:12 DUA is a data security company. I'm sure
6:14 many of you in this room use them. Now,
6:15 when you're dealing with a cyber
6:18 security incident, a ransomware attack,
6:21 you don't have time to go through like
6:23 thousands of pieces of logs and
6:24 information manually to sift through
6:26 that when you're in middle of the
6:29 crisis. And so, Dua worked with AWS to
6:32 build Drew AI. These are agents that
6:34 don't just respond, but they take
6:37 action. They look at proactively the
6:40 logs and autonomously detect the system
6:42 issues and then they take corrective
6:44 action. They move the data around. They
6:47 update the policies. They move the data
6:49 to a storage that is lot more cost
6:51 effective based on the data retention
6:54 and the policies that you set driving
6:57 lot higher value.
7:00 But to take advantage of this requires a
7:03 change in how we lead.
7:05 If you think about historically
7:07 leadership has been mainly about
7:10 optimizing for determinism.
7:12 We are all rewarded for lowering and
7:15 reducing deviation having repeatable
7:19 predictable processes that do the same
7:21 thing over and over again by reducing
7:24 the error rate. And now this works
7:28 today. But the non-determinism, the
7:31 ability to improvise, the ability to
7:34 adapt, the ability to take different
7:36 action based on the context and changing
7:39 requirement with agent is a feature, not
7:42 a bug. Now, I'm sure this might make
7:44 some of us nervous, especially from
7:46 regulated industries. So, this is not
7:49 about just stepping into the wild west.
7:52 This is about us as leaders being aware
7:54 that the non-determinism is what gives
7:57 agents the power and how do we safely
7:59 harness this by setting up the right
8:03 guardrails. And the mental model that I
8:05 find useful is to think about how we
8:07 work with some of our high agency teammates.
8:09 teammates.
8:11 You don't call your high agency teammate
8:13 every Monday and say, "Hey, I'm going to
8:16 sit down with you to give you specific
8:18 tasks that you're going to execute every
8:20 single day and then you're going to come
8:22 and at the end of the day report to me
8:24 what you did, right? We actually give
8:28 them a strategic higher order intent. We
8:30 give them the boundaries within which
8:32 they can operate. We empower them with
8:34 the right resources and access that they
8:36 need to get the job done. And then we
8:38 trust them to escalate to us and involve
8:41 us as needed.
8:43 And so to lead in this high agency
8:46 requires a shift in the leadership model
8:49 because we are no longer managing automation.
8:50 automation.
8:55 We are managing thousands of agents at
8:58 scale and that intelligent autonomy and
9:01 managing that at scale requires shift
9:04 into four key areas of leadership.
9:06 Now please keep in mind that the mental
9:08 model that I'm going to share are just
9:11 mental models. They are not recipes or
9:13 strict operating model. So take them
9:17 with that lens. And four areas that I'm
9:19 going to cover starting with governance.
9:22 Now traditionally whether it is in
9:24 software or in compliance we have
9:28 gatebased governance. system moves,
9:31 comes to a complete stop, waits for
9:33 someone to verify a series of things via
9:36 a checklist that has been defined and
9:39 maybe audited couple of times a year.
9:41 They approve it, then the system moves
9:43 again, stops again at the next toll
9:47 gate. Now, this works today, but when
9:48 you have hundreds and thousands of
9:51 agents, you can't put these toll boots
9:53 in front of them consistently. That will
9:56 just slow it down.
9:59 Instead, the mental model that is useful
10:01 is to think about providing that
10:03 strategic intent and then having a
10:07 policy engine that can govern this at
10:09 scale. And as I was thinking about,
10:13 well, what governs like this, the the
10:15 thing that came to my mind is that if
10:16 you look at how board of directors
10:19 actually work with the CEO, once again,
10:22 they don't go to CEO and say, "I'm going
10:24 to tell you exactly how to run your
10:26 company every single day." Board and CEO
10:28 typically align on a strategic direction
10:32 and the vision. Periodically they go and
10:34 audit and calibrate but there is also
10:38 specific guardrails around what requires
10:40 board approval and the the management
10:43 team knows when to engage the board. So
10:45 it's a useful mental model to think
10:49 about governance this way.
10:52 The second shift is in risk.
10:55 Now today our riskmanagement controls
10:58 are pretty much like a factory floor. On
11:01 a factory floor you have fixed threshold
11:02 right? Even today in many of our
11:04 organizations we have fixed threshold
11:07 regardless of the exposure of the risk.
11:09 So if a purchase order is more than $2
11:12 million it requires approval of a vice
11:15 president. If it is beyond $10 million
11:18 it must go to the CFO. Now these
11:22 controls for risk break down when you're
11:25 dealing with autonomy at scale. The
11:27 mental model that I find useful is to
11:30 think about managing the risk like a
11:33 trading floor. And on a trading floor,
11:36 the risk is actually managed lot more aggressively.
11:37 aggressively.
11:39 But it is managed through real-time
11:41 visibility and control. Traders have a
11:44 portfolio that they can uh play the
11:47 trades in. Not every move is managed but
11:49 if the portfolio or a particular
11:51 transaction violates the policy then
11:53 there are circuit breakers that stops
11:55 the trading. Let's say the portfolio
11:58 drops by 20%, the circuit breaker kicks
12:00 in and stops the trading. And so we need
12:03 to think about agents this way where
12:07 agents have to earn their liquidity by
12:09 operating within the risk controls and
12:12 the boundaries that we design and have
12:14 it with real-time visibility and
12:16 automation. And it's also not about
12:19 managing every single agentic
12:21 transaction but also monitoring the
12:23 systematic drift in a multi- aent
12:25 system. When that happens, you want the
12:29 circuit breakers to kick in.
12:30 One of the most fundamental change
12:32 though is going to be in the
12:35 organization structure. And again as
12:37 leaders we all know this. Every single
12:39 time there has been a major
12:42 technological change that has required
12:45 us to reimagine and rethink how we organize.
12:46 organize.
12:49 Before cloud we had application and
12:50 development teams and then
12:52 infrastructure and technology operation
12:56 team. When cloud came these boundaries
12:58 got blurred. you started to get full
13:00 stack engineering teams that can operate
13:03 at all layers of the stack. In fact,
13:06 with agents, we are seeing this inside
13:10 of our own teams where boundaries beyond
13:12 just engineering, including the business
13:14 process roles, product management,
13:17 engineering, TPMS are all getting
13:20 blurred because you have an ability to
13:22 execute and work with agents throughout
13:25 the entire value chain.
13:28 Historically our orc structures are
13:30 vertically optimized.
13:33 Now this structure provides us stability
13:36 but it is not designed to move fast. If
13:39 a customer has an issue that gets handed
13:41 off sequentially between the department
13:46 from customer support to supply chain to
13:49 distribution to return and refund and it
13:52 slows it down. But the mental model to
13:55 use in terms of how you organize for
13:58 agents is like an immune system.
14:01 How does our body's immune system work?
14:03 If there is a threat or if there is a
14:05 problem, our white blood cells don't
14:08 wait for a memo from the brain, right?
14:09 They don't call a meeting with the
14:12 lungs. They actually swarm that problem
14:15 and then figure out how to solve that.
14:17 So the crossf functional teams that are
14:20 organized around a business workflow
14:22 regardless of the current departmental
14:24 structure is how you really get outcome
14:27 from agents. The other thing that
14:29 happens in this kind of arc structure is
14:33 that system learns constantly. So it's
14:35 sort of like creating antibodies, right?
14:37 So when you have this responsive swarm,
14:41 it gets better with period of time.
14:43 The last piece
14:45 and so before before I go there really
14:47 what we need to think about is that
14:50 agents should not be limited by your org
14:52 chart. They should be limited by the
14:55 objective that they are pursuing. That
14:57 brings us to the last part of this
14:59 leadership mental model and that is
15:01 around culture.
15:03 Historically we have all optimized the
15:06 culture for precision. It's sort of like
15:08 well you got to almost hit a bullseye
15:11 every single time. That's what we reward
15:14 a perfect execution.
15:16 But what happens in a research lab-like
15:20 culture? That culture is open to new
15:22 discovery. Sometimes good, sometimes
15:26 bad. But if those mistakes happen, they
15:30 are used as a learning mechanism. They
15:33 are documented. They are shared widely.
15:36 And the new discovery and adaptation is
15:38 once again a feature not a bug in this
15:40 kind of a culture. And so that's the
15:43 shift that we need because what we are
15:46 trying to scale is not obedience. We are
15:47 actually trying to scale the
15:49 intelligence throughout our organization.
15:51 organization.
15:53 And so the four key leadership mental
15:56 model shifts that I talked about,
15:59 governance like board of directors that
16:01 provide strategic direction, the
16:02 continuous calibration and the
16:05 guardrail, risk management like a
16:08 trading floor where you have real-time
16:10 visibility and circuit breakers and then
16:12 agents earn their liquidity and
16:14 authority to operate by consistently
16:16 showing us that they're operating within
16:18 those risk controls. the organization
16:21 structure that is like an immune system
16:23 that is a lot more fluid and responsive
16:26 and then finally the culture that is
16:28 like a research lab culture that is open
16:31 to new discoveries. If you want to read
16:33 more you can scan that QR code and and
16:35 and dive deeper into some of these
16:37 mental models.
16:39 Now I want to talk about the business
16:42 process and I'm going to pick a business
16:44 process that I'm pretty sure most of us
16:46 in this room are familiar with uh which
16:48 is accounts payable.
16:50 Now traditionally the goal of an
16:52 accounts payable business process has
16:55 been to pay invoices on time and
16:58 accurately. And I want all of us to time
17:01 travel to preppp days. Right? So just
17:03 stay with me here. In the preppp days,
17:05 what happened was that an account
17:07 payable system had a fixed goal to pay
17:10 every invoice on time and uh and
17:12 accurately. But the workflows were
17:15 vertical. So procurement will process
17:17 the PO receiving will confirm the
17:20 receipt of the goods. Uh treasury will
17:22 make the payment and AP will post the
17:25 invoice and so on. The SOPs were all
17:27 vertically inside of each of the
17:29 department and the function and then
17:31 there was sequential handoff that
17:34 happened. Now in this kind of a system
17:38 if there was any error or an issue it
17:40 will manually get kicked back into the
17:42 previous process. Somebody will have to
17:46 call that department to fix that issue.
17:48 How many of you in the room have been
17:50 part of an ERP implementation
17:55 or or run an ERP? Okay, several of you.
17:58 Now, what happened during ERP is that we
17:59 could not look at these vertical
18:02 processes. We had to re-engineer an AP
18:05 process horizontally. We had to connect
18:09 the dots between procurement, receiving,
18:12 treasury, AP and so on.
18:15 that that forced us to talk across
18:18 department and function to relay out our
18:20 business process. But that there was
18:22 still limitation because the goal was
18:23 still fixed which was to pay the
18:25 invoices on time and accurately and if
18:28 there was an issue anytime during the
18:30 process it will get kicked back and then
18:32 manually you'll have to handle the exception.
18:34 exception.
18:38 Now what if in an agentic system that I
18:41 described earlier we take a lot higher
18:44 magnitude goal. What if the goal of AP
18:47 is not just to pay invoice on time but
18:50 to optimize our cash flow and the system
18:53 has the freedom to operate fluidly
18:56 within the boundaries that we set. So a
18:58 particular invoice will go through all
19:01 of the steps, but then a second invoice
19:04 comes in and the agents will know that
19:07 this invoice is in euro and I have a
19:09 60-day payment term on this particular
19:12 invoice. That means I'm going to call a
19:14 forex trading agent and say, can I
19:17 invest this for 59 days and actually
19:20 improve my cash flow? or a third invoice
19:23 comes in and it recognizes that last
19:25 four times I approved this particular
19:28 vendor payment. It had to go into
19:30 dispute because we did not like the
19:32 quality of the deliverable that the
19:34 vendor had. And so this time I'm going
19:37 to proactively bring human in the loop
19:39 and say even though approval processes
19:41 have confirmed I'm going to involve the
19:42 human and say hey last four times we
19:44 didn't like what happened here. Do you
19:46 still want me to pay? And so that's a
19:50 lot higher order task but also a lot
19:52 more fluid workflows that are happening
19:55 here. Today though as I was talking
19:57 about we are already seeing goal-based
20:00 agents being in play. This is where you
20:03 have a planning agent or a planner that
20:05 takes this higher order task breaks it
20:08 into manageable chunk and then you have
20:10 orchestrator that works across
20:12 individual agents inside of each of the
20:14 function like procurement, receiving,
20:17 AP, treasury and goods to get to the
20:21 payment and and reduce the cycle time.
20:23 In fact, we are seeing this in Amazon as
20:26 well. I'll give you an example of an
20:29 orchestrator within Amazon. In our
20:32 warehouses, we have over a million
20:35 robots that actually help us safely and
20:39 quickly move the packages around. And we
20:40 have this orchestrator
20:43 called Deep Fleet. Sort of think about
20:46 this as a brain of this robotic fleet
20:50 that manages these 1 million robots
20:52 distributed all over the world across
20:54 our warehouses. And by constantly
20:57 optimizing that we are able to reduce
20:59 the travel time by 10%.
21:03 Now 10% may not sound a lot but at our
21:06 scale with million robots and multiple
21:07 hundreds hundreds of warehouses and
21:09 distribution center that means that we
21:11 are able to get the packages faster to
21:14 you. And so that's an example of an
21:17 orchestrator operating at scale
21:20 optimizing the business outcome.
21:22 Now earlier I talked about the behaviors
21:25 of an agentic system. Let's double
21:27 click. We we talked about the behaviors.
21:29 We talked about the leadership mental
21:31 model as well as the business process
21:34 changes. Let's double click and say what
21:37 are the technical capabilities that
21:40 bring these behaviors to life. And there
21:42 are three things that agents need. And
21:44 again, if you want to geek out a little
21:46 bit more in architectural implication,
21:48 you can scan this QR code and and read
21:50 an article that dives a lot deeper into
21:52 the architectural implication in your
21:55 environment. But really, agents need
21:57 three things. They need intelligence,
22:00 they need context, and they need trust.
22:03 And and if we were to think of agent as
22:07 sort of human body, here is how that
22:10 looks like. Uh the intelligence is sort
22:13 of like the brain of the agentic system
22:15 and your models including your thinking
22:17 model for chain of thought reasoning
22:20 reflection they provide the brain an
22:22 ability to take the intent and break it
22:26 into task. But just having a brain is
22:28 sort of like having a brain in a jar
22:31 right it cannot take any action. So you
22:34 then have the access the context which
22:37 provides agent an ability to access the
22:39 right data. It provides an ability to
22:41 actually take action. It provides an
22:44 ability to access the tools to go and
22:47 make things happen. And that's sort of
22:49 like having the the hands to go and do
22:51 the work.
22:54 But without trust none of this matters.
22:57 And so the final layer of this is the
22:59 trust. It is sort of like having an ID
23:03 badge. What is the agent's identity? Who
23:05 are they operating on behalf? What are
23:08 their authorizations to execute and do
23:10 this task? And then what are the
23:11 guardrails that they are going to
23:14 operate within to make sure that they
23:16 are doing this safely. And so let's
23:19 double click on each one of them and say
23:21 what are some of the implication in
23:23 terms of technical choices that we have
23:25 to make. So the first thing is that
23:30 agents need intelligence. Now in a in an
23:33 ideal world, we would want the most
23:36 intelligent agent at the lowest possible
23:40 price at the fastest possible speed,
23:42 right? We want to optimize on all three
23:45 of them. But in real life, based on your
23:47 use case, based on the problem that
23:49 you're trying to solve, you might not
23:51 need all three of these in equal
23:55 measure. In fact, there is a trade-off.
23:57 So, if you're working with an agent that
24:00 is legal agent, you might say, "Hey, you
24:02 know what? I cannot compromise on
24:05 accuracy and intelligence here. But I'm
24:08 okay if it takes half a day or few more
24:10 hours or even a day to give me the
24:12 response. So, I'm going to optimize for
24:15 intelligence, but I'm okay to trade off
24:18 on speed of response. On the other hand,
24:20 if you're dealing with a customer
24:23 service agent, you want fast, quick
24:25 responses because your customers are not
24:28 going to wait for 10 minutes for agent
24:30 to come back with a 100% accurate
24:33 answer. And so this is where you
24:36 optimize for speed, but then you can
24:39 trade off on intelligence or the cost.
24:42 And this is why when we are thinking of
24:45 enabling the intelligence in the agentic
24:49 system, the choice of models, having
24:51 access to different types of model is
24:55 very important because every model has a
24:58 different strength. Some models are good
25:00 with reasoning, others are very good
25:04 with quick responses. Some do text based
25:07 task really well. Others do images and
25:09 mathematical task really well. And so
25:12 this is why our approach with Amazon
25:15 bedrock has been to make the broadest
25:17 and the widest set of models available
25:19 including open source model, Amazon's
25:21 own model with Noah as well as a lot of
25:24 third party models and we see this
25:27 application inside of Amazon businesses
25:30 across our company. Take example of
25:34 pharmacy. Now at Amazon Pharmacy, our
25:36 goal is to reduce the time it takes to
25:38 fill your prescription and do that
25:39 accurately. And sometimes the
25:41 prescriptions are handwritten notes and
25:44 this is where we use models and
25:47 intelligence to reduce the time by 90%
25:51 while reducing the error rate by 50%.
25:53 On the other hand, uh on Amazon ad site,
25:56 we want to provide the ability for
25:59 brands who want to advertise to take an
26:01 image of their product and actually
26:03 touch it up and make it better and and
26:05 make it a lot more compelling when they
26:08 list it. And this is where we optimize
26:10 for ability to generate compelling
26:14 images on the prime video side. Uh any
26:17 any football fans in the house? All
26:20 right, some of you. uh if if you're like
26:21 me, you always join the game a little
26:23 bit late and then you're like what did I
26:25 miss and so you have the feature of
26:28 rapid recap which is where model is
26:29 going and saying here are the key things
26:32 or key plays that happened and let me
26:35 catch you up as you come in or nextgen
26:38 stats which actually provides
26:41 lot more probability and information to
26:43 the viewer bringing them closer to the
26:46 action. All of these use cases require
26:48 different behaviors and different models.
26:51 models.
26:55 Take example of our customer Senta.
26:57 Now farming if you think about it is a
27:00 is one of the most complex multivaried
27:02 problem right a farmer has to take into
27:05 account so many different things. What
27:07 is the weather condition? What seed to
27:11 plant? When do I plant it? Uh what
27:13 pesticide to use? How much should I
27:16 spray? Do I fertilize or not? When do I
27:17 fertilize it? Now, these are just the
27:20 factors that are in their control. But
27:22 they are all dependent on so many other
27:25 external factors. The soil condition,
27:28 the moisture, the weather pattern, the
27:30 best activity in that particular area.
27:34 And this is why Sententa worked with AWS
27:36 to develop cropwise AI, a series of
27:39 agents that take information from
27:41 multiple different data sources
27:43 including soil condition, historical
27:47 yield, the seed quality, but also 80,000
27:50 different growth stages of crop to
27:53 provide specific action plan to farmers
27:57 on what they need to do when, including
27:58 here's the prediction of what might
28:00 happen next week and this is what you
28:03 should be doing right now increasing the
28:07 yield 5%. Now again that is sustainable
28:10 but it is also providing a direct
28:12 business outcome making it easier for
28:15 the farmers. So we talked about
28:17 intelligence but as I said just having
28:19 intelligence is sort of like having a
28:22 brain but not an ability to act. So
28:26 let's talk about context. Now context is
28:29 sometimes misunderstood as this is all
28:32 about just data. But as I'll show you,
28:34 context is a lot more than just your
28:37 existing data. In fact, there's a
28:38 tongue-in-cheek uh statement and by the
28:40 way, this is tongue-in-cheek. So take it
28:42 with that from a senior scientist at
28:45 Amazon that talks about the fact that
28:47 like if you think about this, anything
28:51 out of an LLM is a hallucination. It is
28:52 our job because it's basically
28:55 predicting the next token and uh and and
28:59 the word but it is our job to provide it
29:01 the right context to make sure that that
29:04 hallucination is relevant to what we are
29:07 trying to solve for. And this is why
29:10 there is a growing importance of context
29:12 engineering. When we think about just
29:15 optimizing the prompt, you're just
29:18 providing a one-time input with text.
29:20 Now sometimes you give one shot, you
29:22 give multiple example but it is still
29:26 back and forth with context in your
29:27 organization especially when you're
29:30 targeting complex workflow you need
29:33 agents to understand the role the
29:36 hierarchy the data the system the tools
29:39 and this is why we are seeing emergence
29:41 of context engineering roles in this
29:44 competency in many of the organizations.
29:46 So to provide this context, let's talk
29:48 about what are some of the technical
29:50 capabilities that you need. The first
29:53 thing is that agents need to understand
29:57 the relationships in your data. And take
30:00 example of a relationship looks like,
30:04 right? Gartner evaluates magic quadrant
30:07 and magic quadrant evaluates technology
30:10 vendors, right? That's the relationship
30:12 between different objects and domain
30:14 inside of your data. If you're a
30:17 retailer, if you're a media company, if
30:20 you are a bank, you have many objects
30:22 whether it is customer transaction,
30:24 their trade, their viewership behavior
30:26 where you need to establish this
30:29 relationship so that agents understand
30:32 how to navigate your workflows. And this
30:35 is why knowledge graph
30:38 becomes important in your data strategy.
30:40 The second thing that agents need is
30:43 that they need to understand semantics
30:45 which is the adjacency because
30:47 especially around adaptation and
30:50 understanding the ambiguity, you're not
30:53 always going to have an exact term that
30:55 an agent is going to go and search and
30:57 look for. This is why they need to
31:00 understand what is closer to what other
31:03 thing. So in this example, if we were to
31:04 envision this in a three-dimensional
31:07 space, a cat and dogs both are pets. So
31:10 they're closer to each other and Gartner
31:11 Magic Quadrant and the leading cloud
31:14 providers are are sort of far away from
31:17 cats and dog. But if you actually go
31:20 into second dimension, cats and dog are
31:23 not the same family and so they are
31:25 farther apart and the Gartner magic
31:27 quadrant is a lot closer to the leading
31:29 cloud provider. And this is why the
31:32 semantic understanding of your data is
31:36 what AI and agents need. And this is
31:38 where vector databases come into play
31:40 because vector is the language that
31:43 agents speak. It provides that you need
31:45 to provide them relationship. You need
31:47 to provide them the semantics.
31:50 But all of this won't matter if you
31:53 don't have the memory. And when it comes
31:57 to memory, I want to focus on the bottom
31:59 right. The four things on the bottom
32:03 right. Think about priming. Now priming
32:06 is sort of like uh mindset. It's giving
32:09 a role to your agent that you are a
32:12 financial service uh agent. That means
32:13 you're staying in character. You
32:15 understand what the role you are
32:19 playing. You then have procedural memory
32:22 which provides the agent the how. Right?
32:25 This is our email system. This is our
32:28 ERP. And so they have the ability to
32:32 remember who I am, what I'm allowed to
32:34 do and where it is. Then you have
32:37 semantic. Semantic is sort of like your
32:39 word knowledge. So the memory needs to
32:42 understand not just all of the knowledge
32:44 that is available in the word but also
32:47 understand here is my Q3 sales strategy
32:50 and then finally the episodic memory so
32:53 that agents remember what happened last
32:55 time. So hey, you know, this particular
32:57 invoice as I shared in the AP example
33:00 had issues four times in previous run.
33:03 So I better remember to address this
33:06 next time when I'm encountering this
33:08 problem because if you don't provide
33:11 memory to the agent, it is sort of like
33:13 having a goldfish, right? It forgets
33:15 everything and then you don't get the
33:18 outcome. And that creates not only poor
33:20 customer experience, it also destroys
33:22 some of those behaviors that I talked
33:24 about earlier that it needs to get
33:26 better. It needs to remember, it needs
33:29 to constantly optimize and improve. So
33:31 we talked about in the context the
33:34 relationship and knowledge graph,
33:37 semantics, the vector databases, four
33:39 different types of memory that the agent
33:42 needs. But lot of us know especially in
33:45 large enterprises that lot of our
33:47 content is not even in databases. It is
33:51 actually in documents
33:53 that look like this. Right? And they are
33:56 designed typically for all of us as
33:58 humans to understand. We know when we
34:00 look at this memo to say is a large
34:02 bolded test text that looks like a
34:05 title. There are section with numbering
34:06 and the bullets that means I'm looking
34:09 at a list. There's a paragraph and this
34:12 break. We know how to read this, but
34:16 agents don't. And this is where we need
34:19 to focus on taking the content that is
34:21 in lot of these documents and convert
34:25 that into MD machine readable files that
34:27 are friendlier to agent. Here's a
34:30 wonderful example from Stripe and you
34:32 can read more about it in terms of what
34:34 they have published but which is taking
34:37 our SOPs, our documents, our workflows,
34:41 our orc charts and making into machine
34:44 readable agentfriendly documentation
34:46 because it's the combination of
34:48 structured data memory as well as
34:50 content which is going to give the
34:53 ability for agents to have this context
34:56 and then the final piece of this context
34:58 is ability to access the tools and
35:00 actually take action.
35:04 Now before MCP model context protocol,
35:06 those of you who have dealt with lot of
35:09 APIs know this that every time when you
35:11 don't have a standard protocol, you're
35:13 trying to connect to a system, you're
35:16 building a point-to-point integration
35:18 and that is very very difficult to
35:22 scale. What MCP has done is it's sort of
35:26 like a USBC port for giving agent an
35:29 ability to access lot of your tools and
35:32 it allows you to have a standard
35:34 interface across your systems, your
35:37 tools, your data so that agents have the
35:39 context and an ability to access the
35:42 tool and the information that they need.
35:45 So the combination of these pieces, the
35:48 relationship, semantics, memory, better
35:52 content and MCP or any other protocol
35:54 that actually allows it to access to the
35:58 tool allows us to move from data just
36:00 being an asset which we have heard many
36:03 many times to actually having knowledge
36:05 as a capability. Because when we are
36:08 leading in intelligent autonomy, we need
36:11 an ability for humans and agent to share
36:14 the knowledge and be able to share back
36:17 that with each other as well. And we see
36:19 this that when you provide the context,
36:23 the outcomes are amazing. Rufus is
36:27 Amazon's shopping assistant. Uh and it
36:29 provides personalized contextual advice.
36:31 So if you're you know buying something
36:34 and say does it need a AAA battery or
36:37 double A battery or hey I bought this
36:39 other thing uh in the past does it
36:41 actually work with the product that I've
36:45 already bought. This year alone over 250
36:48 million shoppers have used Rufus
36:51 throughout the year. And those who use
36:53 Rufus are 60% more likely to complete a
36:56 purchase. And that happens because Rufus
36:59 understands and maintains the context
37:01 and provides the useful information to
37:03 the customer.
37:07 The last piece is the trust because an
37:09 intelligent agent that has an ability
37:11 and the context but the one that we
37:15 cannot trust will never scale in an
37:18 enterprise. And one way we can be sure
37:21 about the trust is by setting guardrails.
37:22 guardrails.
37:25 And as part of Amazon Bedrock, we have
37:28 we have Amazon Bedrock guardrails which
37:31 allow you to set specific guardrails
37:34 depending on the industry, your company
37:36 policy. For example, uh before I joined
37:39 AWS, I was a CTO for a global media and
37:41 entertainment company. And that meant
37:45 that we could not use certain term uh
37:47 during prime time in our show. Now
37:51 imagine having to apply this rule agent
37:53 by agent throughout the enterprise.
37:56 Well, that will not scale, right? But
37:59 Amazon Bedrock guardrail allows you to
38:02 mention the denied topics to say, "Hey,
38:04 never provide financial advice. Do not
38:06 mention competitor." And then that
38:09 applies to every agentic workflow and
38:12 every model that you're using from
38:14 bedrock. So you can be confident that
38:16 these guardrails are consistently applied.
38:17 applied.
38:18 Think about the sensitive information
38:21 filter, the PII. You don't want that to
38:22 be exposed. You can define some of those
38:25 rules in the guardrails. Contextual
38:27 grounding checks, which provides
38:29 information to all of your agents that
38:31 are at a policy level to say, you know
38:34 what, our return policy is 90 days. You
38:37 define it here and every time there's an
38:38 interaction, it can and if there's a
38:40 conflict, guardrail will make sure that
38:43 that is grounded in the policy that you
38:44 defined there.
38:47 But one particular area that I want to
38:49 highlight in the trust is automated
38:52 reasoning checks. Now automated
38:55 reasoning checks is a search for a
38:58 mathematical proof that a function, a
39:02 program, an agent did something that it
39:04 was supposed to do. Now you're doing
39:07 this mathematically.
39:10 And I'll give you an example of how this
39:13 actually works. So let's say we want to
39:16 say how do we know that Pythagorian
39:18 theorem is correct?
39:21 Well, one way to do this is to try and
39:23 draw every symbol, every single
39:26 variation of a triangle and then go and
39:27 manually measure everything to make sure
39:30 that the theorem is correct. But that is
39:32 not practical, right? That is not how we
39:35 can prove the Pythagorean theorem. The
39:37 way we actually establish this is by
39:40 having a mathematical proof that this is
39:42 correct without having to manually go
39:45 and draw and measure every single
39:47 variation of a triangle. That is exactly
39:50 how automated reasoning works. And as
39:52 part of Amazon Bedrock guardrail,
39:55 automated reasoning checks are available
39:58 that reduce the hallucination by over 99%.
39:59 99%.
40:02 And even before agentic AI, we've been
40:04 using automated reasoning for provable
40:07 security across AWS for a number of
40:11 years. So S3's public storage and
40:14 blocking that or VPC access analyzer
40:16 where you can say, hey, does my database
40:18 have access to internet? Well, how do
40:20 you actually prove that mathematically
40:22 so that you have approvable security? We
40:23 do this through automated reasoning
40:26 check. And that's one of those tools
40:30 that allow us to trust the agent and
40:31 make sure that we are able to
40:35 effectively scale them in our companies.
40:38 I'll share an example from Amazon text
40:41 and compliance. Now when you're dealing
40:43 with text rules and compliance all over
40:45 the world, uh you have different
40:47 policies, different documents, and again
40:49 it's very difficult to do this manually.
40:52 And so our text and compliance team
40:56 worked with agents to look at 600
40:58 different companies around the world
40:59 including the text policies and the
41:02 rules to do the benchmarking. Now that's
41:05 an example where the trust is very
41:08 important and you have the provable
41:11 security to establish that trust. Now
41:15 after looking at all of this when you
41:18 start to move agents and operate
41:21 multiple agents in production it is
41:23 still hard.
41:26 It requires you to have better runtime
41:28 so that you can isolate each of the
41:31 runtime. As I talked about it needs
41:33 memory so that it remembers different
41:35 types of memory that I mentioned
41:37 earlier. uh you need to provide an
41:40 identity to that agent so that you know
41:43 who is that acting on behalf of and what
41:46 it is allowed to do. Uh it needs access
41:49 to different tools. It needs a gateway
41:52 that it can access information from and
41:54 then it needs observability so that you
41:57 can validate and audit and feel
41:59 confident that it is doing what you want
42:03 it to do. Now all of this requires a set
42:07 of solid primitives. And if you as
42:10 senior leaders, if you go back to like
42:13 just basic components of primitives or
42:15 the building blocks, if you had compute
42:18 and storage, we pretty much can build
42:20 many different application. You can
42:22 create a website, you can create an ERP,
42:25 you can create a CRM with just compute
42:28 and storage. We believe that inference
42:31 is going to become another building
42:33 block of all of the applications that
42:36 are going to come out. And this is why
42:40 with agent core and Amazon bedrock, we
42:41 provide these building blocks and
42:43 primitives so that all of the things
42:47 that I talked about from identity to
42:50 memory to providing access to the tool
42:53 across any model across any protocol,
42:55 not just the models that are on bedrock.
42:58 Agent core allows you to do that so that
43:00 you can provide agent memory. You can
43:02 manage the agent identity. You can
43:04 provide a runtime that you know is
43:07 secure and is isolated.
43:09 You can provide the gateway that manages
43:11 the connection with the other tools. And
43:14 this is why we are pretty excited about
43:16 what we can build using these primitives.
43:18 primitives.
43:19 Now, one of the questions we often get
43:22 from leaders is what are some good
43:25 places to get started? Now, as I talked
43:27 about earlier, while this is not automation,
43:29 automation,
43:31 it is also not full autonomy. It they
43:34 are still early days. And so, let's keep
43:37 in mind that these are still early days.
43:39 And we make sure that we don't think of
43:42 agents as a magic wand. In fact, if you
43:44 have a predictable
43:47 uh workflow with fixed steps uh very
43:50 limited tool use, then just basic
43:53 automation, some Genai assistant works.
43:56 Agents are really good when you want
43:58 dynamic tool selection, when you want to
44:00 take advantage of that adaptability and
44:03 learning, where you want to do pattern
44:05 recognition and matching, where you are
44:07 looking at things like latency and the
44:09 cost implication based on the business
44:12 problem that you're trying to solve. And
44:14 the three specific areas where we see
44:16 the biggest value is in software
44:19 development, customer support and
44:21 customer care, and the knowledge work.
44:23 especially when you are handling
44:26 exception loop type processes. Now, how
44:29 many of you uh in the room do not have
44:35 Well, that was a trick question. Uh
44:37 pretty much all of us have technical
44:39 debt and that's a wonderful example
44:42 where agents can actually help us move
44:45 fast because paying off technical debt
44:47 locks in lot of our engineering
44:50 resources into doing something that must
44:53 be done but doesn't always necessarily
44:55 give better business outcome or
44:57 features. It is sort of like managing
45:00 the risk. And here's an example from
45:03 Thompson Reuters. They had a lot of net
45:06 legacy code that they had to modernize
45:09 and they use AWS transform which is our
45:12 agentic AI based modernization agent and
45:14 they were able to reduce and move that
45:17 modernization forex faster that meant
45:20 that their product and engineering teams
45:22 can actually solve the business problems
45:24 rather than just paying off the debt.
45:26 And so this is a wonderful example where
45:29 agents are already helping us drive
45:31 better outcomes. So I would I would I
45:34 would look at your businesses and find
45:36 areas like that where you can start to
45:39 apply the agent. Now none of this
45:41 matters if we don't prepare our
45:45 organizations with skill and training.
45:48 And this is why over last 25 years we
45:51 have invested heavily into the training
45:53 and certification
45:56 to build the competency across not only
45:58 the companies but around the countries
46:00 across the world. so that we have the
46:02 right talent as leaders to take
46:05 advantage of this. Lot of this training
46:07 is available for free. So I encourage
46:09 not only all of you but also encourage
46:12 your teams to take advantage of it. One
46:14 of my favorite things here is the AWS AI
46:17 league. What I've found leading product
46:19 and engineering teams for a long time is
46:23 that gamification is always a great way
46:25 to get people highly engaged. It's a low
46:27 friction way. You're making it fun.
46:29 people get to learn something and that's
46:31 what AWS AI league does where you can
46:34 actually host a competitive fun league
46:37 inside of your own company. We also, I
46:38 believe, announced yesterday that
46:40 there's going to be a championship
46:43 league with $50,000 price uh where teams
46:46 can compete into. And finally, you're
46:49 going to need experts and we are here to
46:52 help. Uh there is guidance from AWS
46:55 experts, many of the folks from our
46:56 executive and residence team and other
46:59 experts throughout the company. We have
47:02 made over hund00 million investment in
47:05 AWS generative AI innovation center
47:07 where we bring machine learning and AI
47:10 experts from all over the company to
47:13 work alongside you to help you solve a
47:15 business problem. and then AWS
47:17 marketplace where you can quickly get
47:19 access to some of the pre-built agents
47:22 and tools that you can start to deploy
47:24 today in your organization.
47:28 It is really a fascinating and fantastic
47:31 future that is ahead of you and I'm
47:34 really excited to see what you all build
47:36 together. I hope you have a fantastic
47:38 reinvent and rest of the week. Thank you
