0:03 I'd like to share with you what the
0:06 process of building a gent of AI
0:08 software application feels like let's
0:10 take a look here's what the life cycle
0:13 of a gent VI project to build a software
0:15 application feels like we would start
0:17 off by scoping a project to decide what
0:21 do we want this software to do so for
0:23 example say you decide you want to build
0:25 a restaurant reputation monitoring
0:27 system The Next Step would be to
0:31 actually try to implement it and given
0:34 the ease of building AI applications
0:37 using gen of AI which you may have seen
0:39 in the optional video that came before
0:42 this one very often you be able to build
0:45 a prototype quite quickly and then plan
0:48 to over time improve this software
0:50 prototype for some applications I've
0:52 worked on we would build the initial
0:55 prototype in one or two days and that
0:57 initial prototype frankly isn't that
1:00 good initially but building it quickly
1:02 lets us then take it into internal
1:05 evaluation where we might have our own
1:08 internal team write different restaurant
1:10 reviews and test the system to see how
1:13 often it is giving a correct response
1:15 and sometimes the internal evaluation
1:18 will turn out some examples where it
1:21 doesn't give the right results in this
1:24 case with my pastor was code it outputs
1:27 this as a positive sentiment and you
1:29 know sometimes C Pastor is delicious but
1:32 this sounds like a negative sentiment to
1:35 me and based on problems that we
1:38 discover internally will then go back to
1:41 continue to improve the system as you
1:43 saw last week writing prompts is a
1:45 highly aerv process where you have to
1:47 try something see if it works and then
1:50 improve it and building a generative AI
1:52 software application also tends to be a
1:54 very iterative process after a
1:57 sufficient internal evaluation to give
1:59 you confidence that the systems working
2:02 well enough then we would deploy it out
2:05 in the wild and continue to monitor its
2:09 performance and it would not surprise me
2:11 if you deploy something and initially
2:15 external uses also generates input that
2:17 causes your system to make some mistakes
2:20 for example maybe a user right my Ramen
2:23 tastes like Ramen is this good or bad
2:25 well if you're not familiar with Ramen
2:28 or Japanese Cuisine you may not know is
2:31 this a good thing a bad thing and if
2:33 your system rates this as a positive
2:36 sentiment but it turns out that if
2:38 you're ordering miso Ramen on the menu
2:40 you probably don't want it to taste like
2:43 donot Ramen which tastes more like pork
2:46 based soup broth and when you find
2:48 incorrect responses like this other in
2:50 the while you might decide to go back to
2:53 internal evaluation for example to
2:55 systematically understand if your system
2:58 is say underperforming on certain types
3:00 of Cuisine or or you might decide to go
3:03 back to take these learnings to improve
3:05 the prompt or improve the system further
3:07 assuming you decide that these types of
3:08 errors are
3:11 unacceptable so it turns out that
3:13 building generative AI software is a
3:15 highly empirical and by that I mean
3:17 highly experimental process meaning that
3:19 we repeatedly try something and then
3:21 find and fix
3:23 mistakes we've already seen how
3:27 prompting itself is a highly empirical
3:30 process where you would have an idea try
3:32 the prompt see the El response then
3:34 maybe update your idea in the prompt and
3:37 go again but other than updating the
3:38 prompts there are other tools that we'll
3:40 talk about this week for improving the
3:44 performance of your genter AI system one
3:47 to that we talk about later this week is
3:50 rag or retrieval augmented generation
3:51 that gives the large language model
3:55 access external data sources we'll also
3:56 talk later this week about a technique
3:58 called fine-tuning that allows you to
4:02 adapt to Lar language model to your task
4:04 and then finally pre-training models
4:06 which refers to training a lar language
4:08 model from scratch don't worry about it
4:09 if you don't know what any of these
4:12 terms mean we'll go through each of them
4:14 in depth later this week but they're all
4:16 key techniques that in addition to
4:19 prompting gives you different ways to
4:22 improve the performance of your genf AI
4:24 systems performance just to walk through
4:26 a second example of the life cycle
4:29 regenerative project let's look at what
4:32 building a system to take food orders
4:35 might look like say you decide to scope
4:39 a food order customer service chatbot to
4:42 take orders what you would do is start
4:44 by building the system and quickly throw
4:47 together a chatbot to take food orders
4:50 then because we don't know how well this
4:52 is doing internally you might let your
4:55 internal team try it out and place
4:57 different orders and see how well it
5:00 does and sometimes they'll generate good
5:02 responses like do you have pickles on a
5:05 cheeseburger and also if you want some
5:08 and sometimes it will give an unexpected
5:11 poor response such as if you do have
5:14 mushrooms on your Burgers but for some
5:16 reason the chat bought says I'm sorry we
5:18 don't have mushrooms similar to what we
5:20 saw for the restaurant reputation
5:22 monitoring system it would be by
5:24 discovering mistakes like these that
5:28 helps you to improve the system and
5:30 after you're sufficient confidence that
5:34 this is safe to deploy externally you
5:37 can then deploy it and let customers
5:40 place real orders and monitor the large
5:43 language model's responses to make sure
5:45 that if it still says anything it isn't
5:47 quite supposed to that you can continue
5:49 to improve his performance having built
5:51 a number of gen of AI projects I've
5:54 often been surprised and delighted by
5:56 the strange and wonderful things that
5:58 the users will try to do with your
6:01 system for example if a user asks how
6:03 many calories are there in your burger
6:05 initially the system may not know but if
6:08 you discover this you can then update
6:11 the system using perhaps a technique
6:13 called rag that I mentioned just now and
6:15 it will go into depth later this week to
6:18 allow your software application to give
6:21 the correct answer so that's what
6:24 building a gent VI software application
6:26 feels like and if you work at a company
6:29 with a few or lots of software
6:31 Developers is and if you ever come up
6:33 with a cool idea for a G of AI
6:34 application that your company could
6:37 build this hopefully gives you a sense
6:39 of what that process of getting it built
6:42 might be like now one of the worries I
6:44 sometimes hear about is is it really
6:47 expensive to use these large language
6:49 models hosted by companies on the
6:51 internet it turns out that the use of
6:53 these large language models is probably
6:56 cheaper than many people think in the
6:58 next video I'd like to share with you
7:00 some intuitions about how expensive it
7:03 is or isn't to actually use these large
7:06 language models let's go on to the next video
