This content explains how to use the ImageDataGenerator API in TensorFlow for data augmentation and loading, demonstrating its application in training a convolutional neural network model and preparing it for deployment.
Mind Map
Expand करने के लिए click करें
पूरा interactive mind map देखने के लिए click करें
Now before we talk about deploying the
model to Google cloud
I want to cover image data generator API.
In part 3 of the video, we built the
model part 2 and 3. Actually we used the
data augmentation layer in Keras to
train our model.
Now I received some command in that
video saying that why don't you use
image data generator and based on your
feedback and demand I am going to cover
image data generator API in this video.
So what we are doing in this session is
actually we are building, we are training
the model once again, which we did in
part two and three. Same thing but using
image data generator API. Now this API
allows you to do data augmentation
and data loading. It is quite convenient.
So we'll see how it works in detail and
in the next video we will do Google
cloud deployment.
So again just to summarize, I have this
notebook open where we use
data augmentation and resize rescale
explicitly as the layers
in our model. Now I will do the same
thing
using
more convenient approach and if you
Google
tensorflow image data generator say
image data generator, you find a
documentation where you can create this
class object where you can specify
all your augmentation as an argument. For
example, you want to increase brightness.
You know you can specify this brightness
range horizon to flip vertical flip
rescale.
You can do so many things.
So let me just go ahead and create
a variable
of image data generator so I've already
imported it. I'm going to create a
variable of image data generator and I
will specify different parameters for
example, the first parameter will be
rescaling.
So I want to rescale.
I would say rescale
every image has RGB value from 0 to 255
and I want to divide that by 255. In our
previous notebook what we were doing was
using this
this specific layer for rescaling and
now you're doing the same thing
here
using this particular argument.
The second one I will do is horizontal
flip.
I will say this
and
a rotation range.
I will use it.
Let's say 10 you know by 10 degree I
want to rotate my
images.
You can use more transformation, but I
just want to keep things simple. That's
why just using
these menu transformation and then
the second thing, once your
generator is created- so let me assign a
variable to it.
Now you need to
load the images directory from the
directory on the disk.
So here there is a method called flow
from directory.
The flow from directory it's a very very
convenient method and let me just show
you how you can use it. So train data set
train data generator
flow from directory.
Here you specify directory path.
Now
our directory if you look at it
we have just single directory with three
classes.
We need to think about
train splitting the images into
training and test and validation data
set as well. Now one thing, I don't like
about tensorflow API is
this particular method
allows you to
divide your data set into
training and validation data set. So let
me see where is it subset.
Yeah.
So you can specify
subset and you can say my subset is
either training and validation but I
need test as well. So if
tensorflow guys are watching this video
please add
taste as well. You know here because
otherwise it becomes inconvenient. So
since the support is not there, I'm going
to use a tool called
split folders. So I have already
installed the tool on my computer.
So let me just show you i previously ran
it.
So you just do
pip install split folders and that will
install this tool and
I will show you what I'm doing with
this tool. So here
when you run this command
you can say
split my plant village folder into train
test and validation. I want to split this
particular folder
this folder into three data sets train
test and
validation and the ratio that I want to
keep is seventy percent is training
one person ten percent is validation
twenty percent this test
and my output folder name is data set. So
when I run this command
it will become very clear to you what
this is doing. So here's it created a new
folder called data set.
Now see beautiful it has test train
validation, three different folders and
every folder has now these individual
classes. Now if you look at the images in
this folder there are 200 items
in test
because
there are total thousand items initially.
If you look at my original directory
here
early blight has thousand correct
and in my test I said twenty percent so
twenty percent of thousand is how much
two hundred, correct? So here if you go to
test see 200 images
then if you do
training
700 because 70 we said 70 percent are
training images and 1070 percent is 700
so now I have
folders split into these three
categories and I can just say
this folder is data set train.
So that will be my training generator. So
let me assign a
variable to this training generator
and I will specify couple of more
parameters. So my target size is what
each of my images is 256 by 256,
correct?
And just to
because I know I'm going to use this a
lot
so just storing it in a
makes sense.
Then the batch size will be 32
32 base size is pretty much standard
and the
class mode is sparse.
Why is pass because the labels I want
the labels to be 0 1 and 2.
See here
class mode.
So class mode can be
categorical if it is category curl it
will generate
one hot encoded
I don't want hot encoded vector. I want
simple sparse vector you know
0 1 2
and I'll just show you how this works
Let me do one thing.
I will mention save to directory as well
because I want to show you how the
augmented images look like.
I will create a new directory
called augmented images.
So here let me just
make a new directory- augmented images. So
you can see
here. See augmented images
and
it will save the augmented images into
that directory. Now you found 15006
images and it created a training
generator.
Now if you don't know
how generator works
in Python
I would suggest that you go to youtube's
click code basics
generator click on this video code
basics generator. You'll get an
understanding on how generator works. But
just to give you a brief idea,
Generator on fly it is like training
data set. You know tf data set that we
studied before, it will dynamically load
images from directory it will do
transformation because it's image data
generator. All this transformation and it
will return those images in a for loop.
So generator is something you can use it
in a
for loop.
Let's just run that so for
you can do for something in
train generator
now that something is basically image
batch
and label back just like tf data set.
So you will get
this image batch will be 32 images.
This will be 32 labels which will tell
you whether the plant is healthy or late
blight or lip light you know one of the
three values 0 1 2
and
this loop will just keep on going so you
need to
just break because
you know it's a generator it just keeps
on generating. So I will
maybe
you know first
print a shape
just to show you.
So every image batch 32 images
x and y is for each image is 256 256 3
is for RGB. Now when you run this you
will notice one thing that in augmented
images it created see 32 images total
and these are augmented. So some of them
are rotated you see this one is probably
vertical horizontal flip.
this is also horizontal flip I think
so that rotated there is a flip
and
so on.
Okay so rescale
rotation range, horizontal flip. You can
specify more augmentation
as per your convenience
but each image basically let's see if I
have to print each image
it will be a numpy array three
dimensional.. three dimensional array.
And see this is the most important thing.
Now each value is
between zero and one it is not between 0
and 255
because of this argument.
Now to do the same thing here in our
previous notebook.
Remember we had to
do this
but now
that thing is taken care of.
And when you run it again by the way see
this image directory will keep on
growing. Now you have 64 items so make.
Sure, I just
had this directory just to show you, but
ideally you should not have it. You
should delete it.
O,kay it will just work on fly on fly. It
will just keep on generating it. So I
will control execute this again. ctrl
enter if you have that directory and if
you have that specified here the
directory will keep on growing which
might be bad for your hard disk.
All right.
Now I will do the same thing for
validation
data set. The directory name is different
here.
And you will do the same thing for
test data generator as well and again
the directory name is different. So just
a simple copy paste I created three
generator train validation and
test.
Now you can directly build a model. So
I'm just going to copy paste some code
because we are using the
same model architecture convolutional
neural network, network architecture that
we used in a previous notebook.
The only difference here is I have put
some input layer here
and remaining things are
kind of same
and you control
execute. You will
print
model summary.
Okay because all parameters are
trainable, because we are not using any
transfer learning here.
So you can see the model summary,
model compile
same thing. I'm just doing copy paste
from the previous notebook
and then
you do
model.fit. Now here something changes.
So in model dot fit
you will specify your train generator
and in step
per epoch.
So you just specified step per epoc
otherwise train generator just keeps on
generating new images
and usually the guideline is tap per
epoch is basically how many batches you
are running.
So one batch size is what 32 okay 32 is
one best size and total number of images
let's say in train data set is x
So x divided by 32 is the value that you
want to put here so how many images are
there
n our train data set. Well,
1506.
So now
we have total
47 steps
for training.
Then the batch size is 32
validation data
comes through
validation
generator
and then val there are validation steps.
By the way validation steps
would be
same thing
y divided by 32 where y is number of
images in validation data set.
Validation has two one five images
okay around
six steps. I want here
and then verbose is one and i'm going to
try 20 epochs. I tried this previously.
I'm not going to lie but this is
something
you
decide
based on some experimentation.
Okay.
So
then you will run
your
training and I'm not gonna run it right?
Now because it's it's gonna take time so
i'm just going to show you the
the notebook which I ran before
you know which I used it before.
So I ran this
and I got
around
97 percent
accuracy and my validation accuracy is
95 and
you want to always test your model on
test data set before deploying it
because test data set is something model
has not seen so far.
When you run that you get 97 accuracy
which is pretty good
and then you will plot
you know your usual. The same thing
that we did in the in the part two and
three, which is plotting training and
validation accuracy that that code
doesn't change. So I'm not going to go
over it again you know
and then you can run prediction on
sample images
again. It's a generator. So previously in
a data set we used to do something like
take but here you just
use it live in a for loop. It's an
iterator generator, okay?
And you ran the prediction
the all these functions are same with
little change that's why I'm not going
over it but I ran it on couple of images
and I found that okay most of the
predictions were true if the actual
image is orally applied my predicted
image was also early blight and so on.
And then I saved the model in h5 format.
Okay. Now what is h5 format.
So here
again in Google if you say Tensorflow
save model
you will find
save model documentation and this is
this format called hdf file. Now this is
the old format right now they they use
usual you know they save it to directly
but what I like about this format is
the entire model gets saved into one
file.
Previously
when
we saved our models
you all know
that our models are saved like this in
it was like directory this is my one
saved model. This is my second. But every
model is a directory
but what I just did is I saved it in
this one file called potatoes.i
and that's the file I will be using
deploy to Google Cloud. In Google cloud I
will deploy this file
and we will make prediction
using the
Google Cloud function that is running on
that in the cloud.
All right! So I hope you like this video
if you are loving this series so far
please share it with your friends and
always practice along with me.
Only practice can help you if you just
watch the video, it's not gonna be useful.
So as I'm
coding on youtube, you make a pause you
write code, you practice with me and that
way you can learn effectively. In the
next video we'll deploy this h5 format
model to Google Cloud.
Video के उस moment पर जाने के लिए कोई भी text या timestamp click करें
Share करें:
ज्यादातर transcripts 5 सेकंड से कम में तैयार
एक Click में Copy125+ भाषाएंContent Search करेंTimestamps पर जाएं
YouTube URL Paste करें
कोई भी YouTube video link डालें और पूरा transcript पाएं
Transcript निकालें
ज्यादातर transcripts 5 सेकंड से कम में तैयार
हमारा Chrome Extension लें
YouTube छोड़े बिना transcript तुरंत पाएं। हमारा Chrome extension install करें और watch page पर ही किसी भी video का transcript one-click में access करें।
