0:03 many of the LMS we've been using have
0:05 been previously trained or we say
0:08 pre-trained by some company Often by a
0:11 big tech company when should you
0:13 pre-train your own model this turns out
0:16 to be so expensive that when in doubt I
0:19 would say probably don't do it but let's
0:22 take a deeper look many teams have been
0:24 pre-training general purpose LMS by
0:27 learning from text on the internet these
0:29 efforts to train very large language
0:31 models May cost tens of millions of
0:33 dollars need a large dedicated
0:36 engineering team take many months and a
0:38 huge amount of data many teams have been
0:40 open sourcing such models and that's
0:42 been a fantastic contribution to the AI
0:45 Community if you have the resources to
0:48 pre-trade models and maybe even open
0:50 source them please by all means make
0:52 that contribution to AI I think that
0:54 could be fantastic but for building a
0:57 specific application given the time and
0:59 expense of pre-training a model from
1:02 scratch I think of this as often an
1:05 option of L result it could help if you
1:08 have a highly specialized domain and a
1:11 lot of data for example Bloomberg is a
1:14 company that offers software as well as
1:17 media articles centered around Financial
1:20 Services because of its access to a huge
1:23 amount of TX on finance it trained
1:26 Bloomberg GPT which is Bloomberg's
1:29 custombuilt large language model purpose
1:32 built for financial applications and
1:35 Bloomberg reported that compared to
1:37 general purpose LS that had learned
1:39 mainly from internet data this model
1:42 does quite a bit better on processing
1:45 Financial Texs for many practical
1:47 applications unless you have a huge
1:49 amount of resources and a huge amount of
1:52 data it may be more practical to start
1:54 with an OM that someone else had
1:56 pre-trained say a general purpose LM
1:58 that's learned from a lot of internet
2:00 data and that someone has opened source
2:02 and then to fine-tune that to your own
2:05 data and that will often give pretty
2:07 decent performance but in a much more
2:11 economic way now I am sincerely very
2:13 grateful to the teams that have been
2:14 putting a lot of resources into
2:18 pre-training LMS on a lot of Text data
2:20 on the internet and then open- sourcing
2:22 them and in fact this gives us many
2:25 different LMS that we could choose from
2:28 to use in the next video we'll actually
2:30 take a look at the issue of
2:33 what size omm do you want to use and of
2:34 all the different Elms out there how do
2:36 you think about choosing among different
2:38 ones let's go take a look at that in the
