Emergency vs Non-Emergency Vehicle Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Emergency vs Non-Emergency Vehicle Classification\n",
    "**Source:** Competition page - [JanataHack: Computer Vision Hackathon](https://datahack.analyticsvidhya.com/contest/janatahack-computer-vision-hackathon/#ProblemStatement)\n",
    "\n",
    "Fatalities due to traffic delays of emergency vehicles such as ambulance & fire brigade is a huge problem. In daily life, we often see that emergency vehicles face difficulty in passing through traffic. So differentiating a vehicle into an emergency and non emergency category can be an important component in traffic monitoring as well as self drive car systems as reaching on time to their destination is critical for these services.\n",
    "\n",
    "In this problem, you will be working on classifying vehicle images as either belonging to the emergency vehicle or non-emergency vehicle category. For the same, you are provided with the train and the test dataset. Emergency vehicles usually includes police cars, ambulance and fire brigades.\n",
    "\n",
    "#### Evaluation Metric\n",
    "\n",
    "The evaluation metric for this competition is Accuracy.\n",
    "\n",
    "\n",
    "#### Public and Private split\n",
    "\n",
    "Note that the test data is further randomly divided into Public (40%) and Private (60%) data. Your initial responses will be checked and scored on the Public data.\n",
    "\n",
    "#### Sample Image\n",
    "\n",
    "<img src=\"sample_images/1.png\" width=\"75%\">\n",
    "\n",
    "#### Result\n",
    "Validation Set Accuracy: 0.9453<br>\n",
    "Public Leaderboard Score: 0.9618055556<br>\n",
    "Private Leaderboard Score: 0.9593301435<br>\n",
    "\n",
    "#### A supplementing blogpost to this notebook can be found at [Emergency vs Non-Emergency Vehicle Classification](https://dilbertai.com/2020/06/22/emergency-vs-non-emergency-vehicle-classification/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import shutil\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare data directories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "source_dir = 'train/images'\n",
    "training_file = pd.read_csv('train/train.csv', header=0)\n",
    "\n",
    "print(training_file.columns)\n",
    "training_file['label'] = training_file['emergency_or_not'].apply(lambda l: 'emergency' if l == 1 else 'not_emergency')\n",
    "training_file.head()\n",
    "\n",
    "train, val = train_test_split(training_file, test_size=0.3, random_state=42, shuffle=True)\n",
    "print(train.label.value_counts())\n",
    "print(val.label.value_counts())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_file = pd.read_csv('test.csv', header=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create required directories\n",
    "os.makedirs('model_images',exist_ok=True)\n",
    "os.makedirs('model_images/training',exist_ok=True)\n",
    "os.makedirs('model_images/training/emergency',exist_ok=True)\n",
    "os.makedirs('model_images/training/not_emergency',exist_ok=True)\n",
    "os.makedirs('model_images/validation',exist_ok=True)\n",
    "os.makedirs('model_images/validation/emergency',exist_ok=True)\n",
    "os.makedirs('model_images/validation/not_emergency',exist_ok=True)\n",
    "os.makedirs('model_images/testing',exist_ok=True)\n",
    "os.makedirs('model_images/testing/images',exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Move data to created directories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# copy files to created training directories\n",
    "for _, img in train.iterrows():\n",
    "    if img.label == 'emergency':\n",
    "        shutil.copyfile(os.path.join(source_dir, img.image_names), os.path.join('model_images/training/emergency', img.image_names))\n",
    "    else:\n",
    "        shutil.copyfile(os.path.join(source_dir, img.image_names), os.path.join('model_images/training/not_emergency', img.image_names))\n",
    "\n",
    "# copy files to created validation directories\n",
    "for _, img in val.iterrows():\n",
    "    if img.label == 'emergency':\n",
    "        shutil.copyfile(os.path.join(source_dir, img.image_names), os.path.join('model_images/validation/emergency', img.image_names))\n",
    "    else:\n",
    "        shutil.copyfile(os.path.join(source_dir, img.image_names), os.path.join('model_images/validation/not_emergency', img.image_names))\n",
    "\n",
    "# copy files to created testing directories\n",
    "for _, img in test_file.iterrows():\n",
    "    shutil.copyfile(os.path.join(source_dir, img.image_names), os.path.join('model_images/testing/images', img.image_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train Classification Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 1152 images belonging to 2 classes.\n",
      "Found 494 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "validation_datagen = ImageDataGenerator(rescale=1./255)\n",
    "validation_generator = validation_datagen.flow_from_directory(\n",
    "    'model_images/validation/', \n",
    "    target_size = (224, 224), \n",
    "    batch_size=32, \n",
    "    class_mode='binary')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 706 images belonging to 1 classes.\n"
     ]
    }
   ],
   "source": [
    "# https://stackoverflow.com/questions/52270177/how-to-use-predict-generator-on-new-images-keras/55991598#55991598\n",
    "test_datagen = ImageDataGenerator(rescale=1./255)\n",
    "test_generator = test_datagen.flow_from_directory(\n",
    "    'model_images/testing/', \n",
    "    target_size = (224, 224),\n",
    "    batch_size=32, \n",
    "    shuffle = False,\n",
    "    class_mode='binary')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 1152 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "train_datagen_aug = ImageDataGenerator(rescale=1./255, zoom_range=0.3, rotation_range=50,\n",
    "                                   width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, \n",
    "                                   horizontal_flip=True, fill_mode='nearest')\n",
    "train_generator_aug = train_datagen_aug.flow_from_directory(\n",
    "    directory='model_images/training/', \n",
    "    shuffle=True,\n",
    "    target_size = (224, 224), \n",
    "    batch_size=32, \n",
    "    class_mode='binary')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VGG 16"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "checkpoint = tf.keras.callbacks.ModelCheckpoint('checkpoints/vgg16_with_aug/', monitor='val_loss', verbose=1, save_best_only=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_17\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "vgg16 (Model)                (None, 512)               14714688  \n",
      "_________________________________________________________________\n",
      "dense_41 (Dense)             (None, 2)                 1026      \n",
      "=================================================================\n",
      "Total params: 14,715,714\n",
      "Trainable params: 14,715,714\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "pretrained_model = tf.keras.applications.VGG16(include_top=False, weights='imagenet', pooling='max')\n",
    "pretrained_model.trainable = True\n",
    "\n",
    "model = tf.keras.models.Sequential([\n",
    "    tf.keras.layers.InputLayer(input_shape=(224,224,3)),\n",
    "    pretrained_model,\n",
    "    tf.keras.layers.Dense(2, activation='softmax')\n",
    "])\n",
    "\n",
    "model.compile(loss='sparse_categorical_crossentropy',\n",
    "              optimizer=tf.keras.optimizers.Adam(lr=1e-5),\n",
    "              metrics=['sparse_categorical_accuracy'])\n",
    "\n",
    "model.summary()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.8231 - sparse_categorical_accuracy: 0.6198 \n",
      "Epoch 00001: val_loss improved from inf to 0.59945, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 625s 17s/step - loss: 0.8231 - sparse_categorical_accuracy: 0.6198 - val_loss: 0.5995 - val_sparse_categorical_accuracy: 0.6700\n",
      "Epoch 2/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.4501 - sparse_categorical_accuracy: 0.7960 \n",
      "Epoch 00002: val_loss improved from 0.59945 to 0.39558, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 478s 13s/step - loss: 0.4501 - sparse_categorical_accuracy: 0.7960 - val_loss: 0.3956 - val_sparse_categorical_accuracy: 0.8360\n",
      "Epoch 3/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.3798 - sparse_categorical_accuracy: 0.8290 \n",
      "Epoch 00003: val_loss improved from 0.39558 to 0.39353, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 734s 20s/step - loss: 0.3798 - sparse_categorical_accuracy: 0.8290 - val_loss: 0.3935 - val_sparse_categorical_accuracy: 0.8239\n",
      "Epoch 4/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.3115 - sparse_categorical_accuracy: 0.8681 \n",
      "Epoch 00004: val_loss improved from 0.39353 to 0.25180, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 677s 19s/step - loss: 0.3115 - sparse_categorical_accuracy: 0.8681 - val_loss: 0.2518 - val_sparse_categorical_accuracy: 0.9028\n",
      "Epoch 5/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.2681 - sparse_categorical_accuracy: 0.8819 \n",
      "Epoch 00005: val_loss did not improve from 0.25180\n",
      "36/36 [==============================] - 726s 20s/step - loss: 0.2681 - sparse_categorical_accuracy: 0.8819 - val_loss: 0.3144 - val_sparse_categorical_accuracy: 0.8644\n",
      "Epoch 6/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.2536 - sparse_categorical_accuracy: 0.9002 \n",
      "Epoch 00006: val_loss improved from 0.25180 to 0.21376, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 738s 21s/step - loss: 0.2536 - sparse_categorical_accuracy: 0.9002 - val_loss: 0.2138 - val_sparse_categorical_accuracy: 0.9231\n",
      "Epoch 7/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.2437 - sparse_categorical_accuracy: 0.8924 \n",
      "Epoch 00007: val_loss did not improve from 0.21376\n",
      "36/36 [==============================] - 465s 13s/step - loss: 0.2437 - sparse_categorical_accuracy: 0.8924 - val_loss: 0.2215 - val_sparse_categorical_accuracy: 0.9130\n",
      "Epoch 8/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.2053 - sparse_categorical_accuracy: 0.9219 \n",
      "Epoch 00008: val_loss improved from 0.21376 to 0.19764, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 472s 13s/step - loss: 0.2053 - sparse_categorical_accuracy: 0.9219 - val_loss: 0.1976 - val_sparse_categorical_accuracy: 0.9251\n",
      "Epoch 9/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1943 - sparse_categorical_accuracy: 0.9123 \n",
      "Epoch 00009: val_loss did not improve from 0.19764\n",
      "36/36 [==============================] - 464s 13s/step - loss: 0.1943 - sparse_categorical_accuracy: 0.9123 - val_loss: 0.2802 - val_sparse_categorical_accuracy: 0.8907\n",
      "Epoch 10/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1748 - sparse_categorical_accuracy: 0.9253 \n",
      "Epoch 00010: val_loss did not improve from 0.19764\n",
      "36/36 [==============================] - 463s 13s/step - loss: 0.1748 - sparse_categorical_accuracy: 0.9253 - val_loss: 0.2070 - val_sparse_categorical_accuracy: 0.9170\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(train_generator_aug, steps_per_epoch=36, epochs=10, validation_data=validation_generator, validation_steps=16, verbose=1, callbacks=[checkpoint])              "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23/23 [==============================] - 59s 3s/step\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.load_model('checkpoints/vgg16_with_aug/')\n",
    "pred = model.predict(test_generator, steps=len(test_generator), verbose=1)\n",
    "predicted_classes = 1 - np.argmax(pred, axis=1)\n",
    "\n",
    "filenames = [_.split('/')[1] for _ in test_generator.filenames]\n",
    "results = pd.DataFrame({\"image_names\":filenames,\"emergency_or_not\":predicted_classes})\n",
    "\n",
    "results.to_csv('submissions/vgg16_with_aug.csv', index=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### continue training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1904 - sparse_categorical_accuracy: 0.9297 \n",
      "Epoch 00001: val_loss improved from inf to 0.19549, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 449s 12s/step - loss: 0.1904 - sparse_categorical_accuracy: 0.9297 - val_loss: 0.1955 - val_sparse_categorical_accuracy: 0.9190\n",
      "Epoch 2/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1748 - sparse_categorical_accuracy: 0.9297 \n",
      "Epoch 00002: val_loss improved from 0.19549 to 0.19088, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 496s 14s/step - loss: 0.1748 - sparse_categorical_accuracy: 0.9297 - val_loss: 0.1909 - val_sparse_categorical_accuracy: 0.9251\n",
      "Epoch 3/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1766 - sparse_categorical_accuracy: 0.9280 \n",
      "Epoch 00003: val_loss improved from 0.19088 to 0.18679, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 475s 13s/step - loss: 0.1766 - sparse_categorical_accuracy: 0.9280 - val_loss: 0.1868 - val_sparse_categorical_accuracy: 0.9251\n",
      "Epoch 4/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1698 - sparse_categorical_accuracy: 0.9332 \n",
      "Epoch 00004: val_loss did not improve from 0.18679\n",
      "36/36 [==============================] - 710s 20s/step - loss: 0.1698 - sparse_categorical_accuracy: 0.9332 - val_loss: 0.2052 - val_sparse_categorical_accuracy: 0.9291\n",
      "Epoch 5/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1416 - sparse_categorical_accuracy: 0.9462 \n",
      "Epoch 00005: val_loss improved from 0.18679 to 0.17628, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 463s 13s/step - loss: 0.1416 - sparse_categorical_accuracy: 0.9462 - val_loss: 0.1763 - val_sparse_categorical_accuracy: 0.9251\n",
      "Epoch 6/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1281 - sparse_categorical_accuracy: 0.9479 \n",
      "Epoch 00006: val_loss did not improve from 0.17628\n",
      "36/36 [==============================] - 454s 13s/step - loss: 0.1281 - sparse_categorical_accuracy: 0.9479 - val_loss: 0.1802 - val_sparse_categorical_accuracy: 0.9352\n",
      "Epoch 7/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1200 - sparse_categorical_accuracy: 0.9566 \n",
      "Epoch 00007: val_loss did not improve from 0.17628\n",
      "36/36 [==============================] - 597s 17s/step - loss: 0.1200 - sparse_categorical_accuracy: 0.9566 - val_loss: 0.1821 - val_sparse_categorical_accuracy: 0.9372\n",
      "Epoch 8/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1077 - sparse_categorical_accuracy: 0.9661 \n",
      "Epoch 00008: val_loss improved from 0.17628 to 0.16964, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 575s 16s/step - loss: 0.1077 - sparse_categorical_accuracy: 0.9661 - val_loss: 0.1696 - val_sparse_categorical_accuracy: 0.9393\n",
      "Epoch 9/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1036 - sparse_categorical_accuracy: 0.9653 \n",
      "Epoch 00009: val_loss did not improve from 0.16964\n",
      "36/36 [==============================] - 565s 16s/step - loss: 0.1036 - sparse_categorical_accuracy: 0.9653 - val_loss: 0.1751 - val_sparse_categorical_accuracy: 0.9372\n",
      "Epoch 10/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.1086 - sparse_categorical_accuracy: 0.9618 \n",
      "Epoch 00010: val_loss improved from 0.16964 to 0.16258, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 552s 15s/step - loss: 0.1086 - sparse_categorical_accuracy: 0.9618 - val_loss: 0.1626 - val_sparse_categorical_accuracy: 0.9433\n"
     ]
    }
   ],
   "source": [
    "checkpoint = tf.keras.callbacks.ModelCheckpoint('checkpoints/vgg16_with_aug/', monitor='val_loss', verbose=1, save_best_only=True)\n",
    "model = tf.keras.models.load_model('checkpoints/vgg16_with_aug/')\n",
    "history = model.fit(train_generator_aug, steps_per_epoch=36, epochs=10, validation_data=validation_generator, validation_steps=16, verbose=1, callbacks=[checkpoint])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23/23 [==============================] - 80s 3s/step\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.load_model('checkpoints/vgg16_with_aug/')\n",
    "pred = model.predict(test_generator, steps=len(test_generator), verbose=1)\n",
    "predicted_classes = 1 - np.argmax(pred, axis=1)\n",
    "\n",
    "filenames = [_.split('/')[1] for _ in test_generator.filenames]\n",
    "results = pd.DataFrame({\"image_names\":filenames,\"emergency_or_not\":predicted_classes})\n",
    "\n",
    "results.to_csv('submissions/vgg16_with_aug_20_epoch.csv', index=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0873 - sparse_categorical_accuracy: 0.9670 \n",
      "Epoch 00001: val_loss improved from inf to 0.21804, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 584s 16s/step - loss: 0.0873 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.2180 - val_sparse_categorical_accuracy: 0.9170\n",
      "Epoch 2/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0892 - sparse_categorical_accuracy: 0.9731 \n",
      "Epoch 00002: val_loss improved from 0.21804 to 0.20807, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 1335s 37s/step - loss: 0.0892 - sparse_categorical_accuracy: 0.9731 - val_loss: 0.2081 - val_sparse_categorical_accuracy: 0.9170\n",
      "Epoch 3/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0883 - sparse_categorical_accuracy: 0.9635 \n",
      "Epoch 00003: val_loss improved from 0.20807 to 0.17300, saving model to checkpoints/vgg16_with_aug/\n",
      "INFO:tensorflow:Assets written to: checkpoints/vgg16_with_aug/assets\n",
      "36/36 [==============================] - 541s 15s/step - loss: 0.0883 - sparse_categorical_accuracy: 0.9635 - val_loss: 0.1730 - val_sparse_categorical_accuracy: 0.9453\n",
      "Epoch 4/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0760 - sparse_categorical_accuracy: 0.9792 \n",
      "Epoch 00004: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 483s 13s/step - loss: 0.0760 - sparse_categorical_accuracy: 0.9792 - val_loss: 0.1983 - val_sparse_categorical_accuracy: 0.9271\n",
      "Epoch 5/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0840 - sparse_categorical_accuracy: 0.9670 \n",
      "Epoch 00005: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 455s 13s/step - loss: 0.0840 - sparse_categorical_accuracy: 0.9670 - val_loss: 0.1843 - val_sparse_categorical_accuracy: 0.9291\n",
      "Epoch 6/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0650 - sparse_categorical_accuracy: 0.9774 \n",
      "Epoch 00006: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 432s 12s/step - loss: 0.0650 - sparse_categorical_accuracy: 0.9774 - val_loss: 0.2443 - val_sparse_categorical_accuracy: 0.9231\n",
      "Epoch 7/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0571 - sparse_categorical_accuracy: 0.9826 \n",
      "Epoch 00007: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 430s 12s/step - loss: 0.0571 - sparse_categorical_accuracy: 0.9826 - val_loss: 0.1882 - val_sparse_categorical_accuracy: 0.9372\n",
      "Epoch 8/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0555 - sparse_categorical_accuracy: 0.9818 \n",
      "Epoch 00008: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 429s 12s/step - loss: 0.0555 - sparse_categorical_accuracy: 0.9818 - val_loss: 0.1821 - val_sparse_categorical_accuracy: 0.9291\n",
      "Epoch 9/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0497 - sparse_categorical_accuracy: 0.9844 \n",
      "Epoch 00009: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 450s 12s/step - loss: 0.0497 - sparse_categorical_accuracy: 0.9844 - val_loss: 0.2045 - val_sparse_categorical_accuracy: 0.9312\n",
      "Epoch 10/10\n",
      "36/36 [==============================] - ETA: 0s - loss: 0.0361 - sparse_categorical_accuracy: 0.9905 \n",
      "Epoch 00010: val_loss did not improve from 0.17300\n",
      "36/36 [==============================] - 513s 14s/step - loss: 0.0361 - sparse_categorical_accuracy: 0.9905 - val_loss: 0.2087 - val_sparse_categorical_accuracy: 0.9413\n"
     ]
    }
   ],
   "source": [
    "checkpoint = tf.keras.callbacks.ModelCheckpoint('checkpoints/vgg16_with_aug/', monitor='val_loss', verbose=1, save_best_only=True)\n",
    "model = tf.keras.models.load_model('checkpoints/vgg16_with_aug/')\n",
    "history = model.fit(train_generator_aug, steps_per_epoch=36, epochs=10, validation_data=validation_generator, validation_steps=16, verbose=1, callbacks=[checkpoint])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "23/23 [==============================] - 94s 4s/step\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.load_model('checkpoints/vgg16_with_aug/')\n",
    "pred = model.predict(test_generator, steps=len(test_generator), verbose=1)\n",
    "predicted_classes = 1 - np.argmax(pred, axis=1)\n",
    "\n",
    "filenames = [_.split('/')[1] for _ in test_generator.filenames]\n",
    "results = pd.DataFrame({\"image_names\":filenames,\"emergency_or_not\":predicted_classes})\n",
    "\n",
    "results.to_csv('submissions/vgg16_with_aug_30_epoch.csv', index=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}