Pre-training large language models (LLMs) from scratch is prohibitively expensive and resource-intensive for most applications, making it more practical to fine-tune existing, pre-trained models.
Mind Map
Cliquer pour agrandir
Cliquez pour explorer la carte mentale interactive complète
many of the LMS we've been using have
been previously trained or we say
pre-trained by some company Often by a
big tech company when should you
pre-train your own model this turns out
to be so expensive that when in doubt I
would say probably don't do it but let's
take a deeper look many teams have been
pre-training general purpose LMS by
learning from text on the internet these
efforts to train very large language
models May cost tens of millions of
dollars need a large dedicated
engineering team take many months and a
huge amount of data many teams have been
open sourcing such models and that's
been a fantastic contribution to the AI
Community if you have the resources to
pre-trade models and maybe even open
source them please by all means make
that contribution to AI I think that
could be fantastic but for building a
specific application given the time and
expense of pre-training a model from
scratch I think of this as often an
option of L result it could help if you
have a highly specialized domain and a
lot of data for example Bloomberg is a
company that offers software as well as
media articles centered around Financial
Services because of its access to a huge
amount of TX on finance it trained
Bloomberg GPT which is Bloomberg's
custombuilt large language model purpose
built for financial applications and
Bloomberg reported that compared to
general purpose LS that had learned
mainly from internet data this model
does quite a bit better on processing
Financial Texs for many practical
applications unless you have a huge
amount of resources and a huge amount of
data it may be more practical to start
with an OM that someone else had
pre-trained say a general purpose LM
that's learned from a lot of internet
data and that someone has opened source
and then to fine-tune that to your own
data and that will often give pretty
decent performance but in a much more
economic way now I am sincerely very
grateful to the teams that have been
putting a lot of resources into
pre-training LMS on a lot of Text data
on the internet and then open- sourcing
them and in fact this gives us many
different LMS that we could choose from
to use in the next video we'll actually
take a look at the issue of
what size omm do you want to use and of
all the different Elms out there how do
you think about choosing among different
ones let's go take a look at that in the
Cliquez sur n'importe quel texte ou horodatage pour accéder directement à ce moment de la vidéo
Partager :
La plupart des transcriptions sont prêtes en moins de 5 secondes
Copie en un clicPlus de 125 languesRechercher dans le contenuAller aux horodatages
Collez une URL YouTube
Entrez le lien de n'importe quelle vidéo YouTube pour obtenir la transcription complète
Formulaire d'extraction de transcription
La plupart des transcriptions sont prêtes en moins de 5 secondes
Installez notre extension Chrome
Obtenez les transcriptions instantanément sans quitter YouTube. Installez notre extension Chrome et accédez en un clic à la transcription de n'importe quelle vidéo directement depuis la page de lecture.
