YouTube Transcript:
Deep learning project end to end | Potato Disease Classification - 6 : ImageDataGenerator API

Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.

Video Transcript

Video Summary

Summary

Core Theme

This content explains how to use the ImageDataGenerator API in TensorFlow for data augmentation and loading, demonstrating its application in training a convolutional neural network model and preparing it for deployment.

Mind Map

Click to expand

Click to explore the full interactive mind map • Zoom, pan, and navigate

Now before we talk about deploying the

model to Google cloud

I want to cover image data generator API.

In part 3 of the video, we built the

model part 2 and 3. Actually we used the

data augmentation layer in Keras to

train our model.

Now I received some command in that

video saying that why don't you use

image data generator and based on your

feedback and demand I am going to cover

image data generator API in this video.

So what we are doing in this session is

actually we are building, we are training

the model once again, which we did in

part two and three. Same thing but using

image data generator API. Now this API

allows you to do data augmentation

and data loading. It is quite convenient.

So we'll see how it works in detail and

in the next video we will do Google

cloud deployment.

So again just to summarize, I have this

notebook open where we use

data augmentation and resize rescale

explicitly as the layers

in our model. Now I will do the same

thing

using

more convenient approach and if you

Google

tensorflow image data generator say

image data generator, you find a

documentation where you can create this

class object where you can specify

all your augmentation as an argument. For

example, you want to increase brightness.

You know you can specify this brightness

range horizon to flip vertical flip

rescale.

You can do so many things.

So let me just go ahead and create

a variable

of image data generator so I've already

imported it. I'm going to create a

variable of image data generator and I

will specify different parameters for

example, the first parameter will be

rescaling.

So I want to rescale.

I would say rescale

every image has RGB value from 0 to 255

and I want to divide that by 255. In our

previous notebook what we were doing was

using this

this specific layer for rescaling and

now you're doing the same thing

here

using this particular argument.

The second one I will do is horizontal

flip.

I will say this

and

a rotation range.

I will use it.

Let's say 10 you know by 10 degree I

want to rotate my

images.

You can use more transformation, but I

just want to keep things simple. That's

why just using

these menu transformation and then

the second thing, once your

generator is created- so let me assign a

variable to it.

Now you need to

load the images directory from the

directory on the disk.

So here there is a method called flow

from directory.

The flow from directory it's a very very

convenient method and let me just show

you how you can use it. So train data set

train data generator

flow from directory.

Here you specify directory path.

Now

our directory if you look at it

we have just single directory with three

classes.

We need to think about

train splitting the images into

training and test and validation data

set as well. Now one thing, I don't like

about tensorflow API is

this particular method

allows you to

divide your data set into

training and validation data set. So let

me see where is it subset.

Yeah.

So you can specify

subset and you can say my subset is

either training and validation but I

need test as well. So if

tensorflow guys are watching this video

please add

taste as well. You know here because

otherwise it becomes inconvenient. So

since the support is not there, I'm going

to use a tool called

split folders. So I have already

installed the tool on my computer.

So let me just show you i previously ran

it.

So you just do

pip install split folders and that will

install this tool and

I will show you what I'm doing with

this tool. So here

when you run this command

you can say

split my plant village folder into train

test and validation. I want to split this

particular folder

this folder into three data sets train

test and

validation and the ratio that I want to

keep is seventy percent is training

one person ten percent is validation

twenty percent this test

and my output folder name is data set. So

when I run this command

it will become very clear to you what

this is doing. So here's it created a new

folder called data set.

Now see beautiful it has test train

validation, three different folders and

every folder has now these individual

classes. Now if you look at the images in

this folder there are 200 items

in test

because

there are total thousand items initially.

If you look at my original directory

here

early blight has thousand correct

and in my test I said twenty percent so

twenty percent of thousand is how much

two hundred, correct? So here if you go to

test see 200 images

then if you do

training

700 because 70 we said 70 percent are

training images and 1070 percent is 700

so now I have

folders split into these three

categories and I can just say

this folder is data set train.

So that will be my training generator. So

let me assign a

variable to this training generator

and I will specify couple of more

parameters. So my target size is what

each of my images is 256 by 256,

correct?

And just to

because I know I'm going to use this a

lot

so just storing it in a

makes sense.

Then the batch size will be 32

32 base size is pretty much standard

and the

class mode is sparse.

Why is pass because the labels I want

the labels to be 0 1 and 2.

See here

class mode.

So class mode can be

categorical if it is category curl it

will generate

one hot encoded

I don't want hot encoded vector. I want

simple sparse vector you know

0 1 2

and I'll just show you how this works

Let me do one thing.

I will mention save to directory as well

because I want to show you how the

augmented images look like.

I will create a new directory

called augmented images.

So here let me just

make a new directory- augmented images. So

you can see

here. See augmented images

and

it will save the augmented images into

that directory. Now you found 15006

images and it created a training

generator.

Now if you don't know

how generator works

in Python

I would suggest that you go to youtube's

click code basics

generator click on this video code

basics generator. You'll get an

understanding on how generator works. But

just to give you a brief idea,

Generator on fly it is like training

data set. You know tf data set that we

studied before, it will dynamically load

images from directory it will do

transformation because it's image data

generator. All this transformation and it

will return those images in a for loop.

So generator is something you can use it

in a

for loop.

Let's just run that so for

you can do for something in

train generator

now that something is basically image

batch

and label back just like tf data set.

So you will get

this image batch will be 32 images.

This will be 32 labels which will tell

you whether the plant is healthy or late

blight or lip light you know one of the

three values 0 1 2

and

this loop will just keep on going so you

need to

just break because

you know it's a generator it just keeps

on generating. So I will

maybe

you know first

print a shape

just to show you.

So every image batch 32 images

x and y is for each image is 256 256 3

is for RGB. Now when you run this you

will notice one thing that in augmented

images it created see 32 images total

and these are augmented. So some of them

are rotated you see this one is probably

vertical horizontal flip.

this is also horizontal flip I think

so that rotated there is a flip

and

so on.

Okay so rescale

rotation range, horizontal flip. You can

specify more augmentation

as per your convenience

but each image basically let's see if I

have to print each image

it will be a numpy array three

dimensional.. three dimensional array.

And see this is the most important thing.

Now each value is

between zero and one it is not between 0

and 255

because of this argument.

Now to do the same thing here in our

previous notebook.

Remember we had to

do this

but now

that thing is taken care of.

And when you run it again by the way see

this image directory will keep on

growing. Now you have 64 items so make.

Sure, I just

had this directory just to show you, but

ideally you should not have it. You

should delete it.

O,kay it will just work on fly on fly. It

will just keep on generating it. So I

will control execute this again. ctrl

enter if you have that directory and if

you have that specified here the

directory will keep on growing which

might be bad for your hard disk.

All right.

Now I will do the same thing for

validation

data set. The directory name is different

here.

And you will do the same thing for

test data generator as well and again

the directory name is different. So just

a simple copy paste I created three

generator train validation and

test.

Now you can directly build a model. So

I'm just going to copy paste some code

because we are using the

same model architecture convolutional

neural network, network architecture that

we used in a previous notebook.

The only difference here is I have put

some input layer here

and remaining things are

kind of same

and you control

execute. You will

model summary.

Okay because all parameters are

trainable, because we are not using any

transfer learning here.

So you can see the model summary,

model compile

same thing. I'm just doing copy paste

from the previous notebook

and then

you do

model.fit. Now here something changes.

So in model dot fit

you will specify your train generator

and in step

per epoch.

So you just specified step per epoc

otherwise train generator just keeps on

generating new images

and usually the guideline is tap per

epoch is basically how many batches you

are running.

So one batch size is what 32 okay 32 is

one best size and total number of images

let's say in train data set is x

So x divided by 32 is the value that you

want to put here so how many images are

there

n our train data set. Well,

1506.

So now

we have total

47 steps

for training.

Then the batch size is 32

validation data

comes through

validation

generator

and then val there are validation steps.

By the way validation steps

would be

same thing

y divided by 32 where y is number of

images in validation data set.

Validation has two one five images

okay around

six steps. I want here

and then verbose is one and i'm going to

try 20 epochs. I tried this previously.

I'm not going to lie but this is

something

you

decide

based on some experimentation.

Okay.

then you will run

your

training and I'm not gonna run it right?

Now because it's it's gonna take time so

i'm just going to show you the

the notebook which I ran before

you know which I used it before.

So I ran this

and I got

around

97 percent

accuracy and my validation accuracy is

95 and

you want to always test your model on

test data set before deploying it

because test data set is something model

has not seen so far.

When you run that you get 97 accuracy

which is pretty good

and then you will plot

you know your usual. The same thing

that we did in the in the part two and

three, which is plotting training and

validation accuracy that that code

doesn't change. So I'm not going to go

over it again you know

and then you can run prediction on

sample images

again. It's a generator. So previously in

a data set we used to do something like

take but here you just

use it live in a for loop. It's an

iterator generator, okay?

And you ran the prediction

the all these functions are same with

little change that's why I'm not going

over it but I ran it on couple of images

and I found that okay most of the

predictions were true if the actual

image is orally applied my predicted

image was also early blight and so on.

And then I saved the model in h5 format.

Okay. Now what is h5 format.

So here

again in Google if you say Tensorflow

save model

you will find

save model documentation and this is

this format called hdf file. Now this is

the old format right now they they use

usual you know they save it to directly

but what I like about this format is

the entire model gets saved into one

file.

Previously

when

we saved our models

you all know

that our models are saved like this in

it was like directory this is my one

saved model. This is my second. But every

model is a directory

but what I just did is I saved it in

this one file called potatoes.i

and that's the file I will be using

deploy to Google Cloud. In Google cloud I

will deploy this file

and we will make prediction

using the

Google Cloud function that is running on

that in the cloud.

All right! So I hope you like this video

if you are loving this series so far

please share it with your friends and

always practice along with me.

Only practice can help you if you just

watch the video, it's not gonna be useful.

So as I'm

coding on youtube, you make a pause you

write code, you practice with me and that

way you can learn effectively. In the

next video we'll deploy this h5 format

model to Google Cloud.

Click on any text or timestamp to jump to that moment in the video

Most transcripts ready in under 5 seconds

One-Click Copy125+ LanguagesSearch ContentJump to Timestamps

Paste YouTube URL

Enter any YouTube video link to get the full transcript

Most transcripts ready in under 5 seconds

Get Our Chrome Extension

Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.

Add to Chrome — Free

Works with YouTube, Coursera, Udemy and more educational platforms

Get Instant Transcripts: Just Edit the Domain in Your Address Bar!

YouTube

←

→

↻

https://www.youtube.com/watch?v=UF8uR6Z6KLc

YoutubeToText

←

→

↻

https://youtubetotext.net/watch?v=UF8uR6Z6KLc

YouTube TranscriptPreparing your results…

YouTube Transcript:Deep learning project end to end | Potato Disease Classification - 6 : ImageDataGenerator API