Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Different values of accuracy in model.fit and model.evaluate on the same generator

I obtain a different accuracy on the same generator by using the model.fit and model.evaluate in tensorflow. This is my sample code

import tensorflow as tf
from nChannelGenerator import nChannelGenerator
from databaseSplitter import automaticPath
from keras.preprocessing.image import ImageDataGenerator

nClasses = 5

height = 200
width = 200
batch_size = 32


    model_1 = tf.keras.Sequential([
    tf.keras.layers.Rescaling(scale = 1./255, input_shape=[None, None, 3]),
    tf.keras.layers.Conv2D(16, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu'),
    tf.keras.layers.MaxPooling2D(),
    tf.keras.layers.GlobalMaxPooling2D(),
    tf.keras.layers.Dense(nClasses, activation = "softmax"),
])

model_1.compile(optimizer=tf.keras.optimizers.Adam(learning_rate = 0.001), metrics=["accuracy"], loss = "categorical_crossentropy")

datagen = ImageDataGenerator(rescale=1.)
data = datagen.flow_from_directory("database_prova")
model_2.fit(data, epochs = 5)
print("This is the evaluation step")
model_2.evaluate(data)

Then this is the output

Found 1302 images belonging to 5 classes.
Epoch 1/5
41/41 [==============================] - 6s 84ms/step - loss: 1.5366 - accuracy: 0.2888
Epoch 2/5
41/41 [==============================] - 3s 64ms/step - loss: 1.3420 - accuracy: 0.4117
Epoch 3/5
41/41 [==============================] - 3s 63ms/step - loss: 0.9565 - accuracy: 0.5799
Epoch 4/5
41/41 [==============================] - 3s 64ms/step - loss: 0.7061 - accuracy: 0.6920
Epoch 5/5
41/41 [==============================] - 3s 64ms/step - loss: 0.6357 - accuracy: 0.7143
This is the evaluation step
41/41 [==============================] - 1s 26ms/step - loss: 0.5574 - accuracy: 0.7957

Both the accuracies and the losses are different between the last training step and the evaluation. I have no batch normalization here as you can see, I then expected the same values. I have seen a previous response, but I cannot figure out where is the problem.

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

>Solution :

The accuracy reported during the last training epoch is the accumulated accuracy during the epoch. Because you’re doing mini batch gradient descent, the weights of the model used for the first batch are different than for the last batch during training.

If you were validating at each epoch, and then doing one more evaluation, you would see the same number.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading