Neural_Nets_Gesture_Recognition_Pre_Fina2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gesture Recognition\n",
    "In this group project, you are going to build a 3D Conv model that will be able to predict the 5 gestures correctly. Please import the following libraries to get started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import os\n",
    "from scipy.misc import imread, imresize\n",
    "import datetime\n",
    "import os\n",
    "import cv2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We set the random seed so that the results don't vary drastically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "np.random.seed(30)\n",
    "import random as rn\n",
    "rn.seed(30)\n",
    "from keras import backend as K\n",
    "import tensorflow as tf\n",
    "tf.set_random_seed(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this block, you read the folder names for training and validation. We will set the initial `batch_size` here. Note that we will experimentally set the batch size in such a way that you are able to use the GPU in full capacity. Keep increasing the batch size until the machine throws an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_doc = np.random.permutation(open('./Project_data/train.csv').readlines())\n",
    "val_doc = np.random.permutation(open('./Project_data/val.csv').readlines())\n",
    "batch_size = 10 #experiment with the batch size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Note*** in our implementaiton true **batch_size** is always 2 fold as we agument our data. if if batch size is 10, true batch size at the run time is 20, as twice the number of data set loaded are added to the batch\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generator\n",
    "This is one of the most important part of the code. The overall structure of the generator is broken down into modules. In the generator, we are going to preprocess the images as we have images of 2 different dimensions as well as create a batch of video frames. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters initialization\n",
    "nb_rows = 120   # X dimension of the image\n",
    "nb_cols = 120   # Y dimesnion of the image\n",
    "nb_frames = 30  # lenght of the video frames\n",
    "nb_channel = 3 # numbe rof channels in images 3 for color(RGB) and 1 for Gray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper function to generate a random affine transform on the iamge\n",
    "def get_random_affine():\n",
    "    dx, dy = np.random.randint(-1.7, 1.8, 2)\n",
    "    M = np.float32([[1, 0, dx], [0, 1, dy]])\n",
    "    return M"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper function to normalise the data\n",
    "def normalize_data(data):\n",
    "    return data/127.5-1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper function to initialize all the batch image data and labels\n",
    "def init_batch_data(batch_size):\n",
    "    batch_data = np.zeros((batch_size, nb_frames, nb_rows, nb_cols, nb_channel)) \n",
    "    batch_labels = np.zeros((batch_size,5)) # batch_labels is the one hot representation of the output\n",
    "    return batch_data, batch_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_load_batch_images(source_path, folder_list, batch_num, batch_size, t):\n",
    "    \n",
    "    batch_data,batch_labels = init_batch_data(batch_size)\n",
    "    # We will also build a agumented batch data\n",
    "    batch_data_aug,batch_labels_aug = init_batch_data(batch_size)\n",
    "\n",
    "    #create a list of image numbers you want to use for a particular video\n",
    "    img_idx = [x for x in range(0, nb_frames)] \n",
    "\n",
    "    for folder in range(batch_size): # iterate over the batch_size\n",
    "        # read all the images in the folder\n",
    "        imgs = sorted(os.listdir(source_path+'/'+ t[folder + (batch_num*batch_size)].split(';')[0])) \n",
    "        # Generate a random affine to be used in image transformation for buidling agumented data set\n",
    "        M = get_random_affine()\n",
    "        \n",
    "        #  Iterate over the frames/images of a folder to read them in\n",
    "        for idx, item in enumerate(img_idx): \n",
    "            image = cv2.imread(source_path+'/'+ t[folder + (batch_num*batch_size)].strip().split(';')[0]+'/'+imgs[item], cv2.IMREAD_COLOR)\n",
    "            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "            \n",
    "            #crop the images and resize them. Note that the images are of 2 different shape \n",
    "            #and the conv3D will throw error if the inputs in a batch have different shapes   \n",
    "            resized = cv2.resize(image, (nb_rows,nb_cols), interpolation = cv2.INTER_AREA)\n",
    "            batch_data[folder,idx] = (resized)\n",
    "            batch_data_aug[folder,idx] = (cv2.warpAffine(resized, M, (resized.shape[0], resized.shape[1])))      \n",
    "\n",
    "        batch_labels[folder, int(t[folder + (batch_num*batch_size)].strip().split(';')[2])] = 1\n",
    "        batch_labels_aug[folder, int(t[folder + (batch_num*batch_size)].strip().split(';')[2])] = 1\n",
    "    \n",
    "    batch_data = np.append(batch_data, batch_data_aug, axis = 0) \n",
    "    batch_labels = np.append(batch_labels, batch_labels_aug, axis = 0) \n",
    "    return batch_data, batch_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def val_load_batch_images(source_path, folder_list, batch_num, batch_size, t):\n",
    "    \n",
    "    batch_data,batch_labels = init_batch_data(batch_size)\n",
    "    # We will also build a agumented batch data\n",
    "    batch_data_aug,batch_labels_aug = init_batch_data(batch_size)\n",
    "\n",
    "    #create a list of image numbers you want to use for a particular video\n",
    "    img_idx = [x for x in range(0, nb_frames)] \n",
    "\n",
    "    for folder in range(batch_size): # iterate over the batch_size\n",
    "        # read all the images in the folder\n",
    "        imgs = sorted(os.listdir(source_path+'/'+ t[folder + (batch_num*batch_size)].split(';')[0])) \n",
    "        # Generate a random affine to be used in image transformation for buidling agumented data set\n",
    "        # M = get_random_affine()\n",
    "        \n",
    "        #  Iterate over the frames/images of a folder to read them in\n",
    "        for idx, item in enumerate(img_idx): \n",
    "            image = cv2.imread(source_path+'/'+ t[folder + (batch_num*batch_size)].strip().split(';')[0]+'/'+imgs[item], cv2.IMREAD_COLOR)\n",
    "            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "            \n",
    "            #crop the images and resize them. Note that the images are of 2 different shape \n",
    "            #and the conv3D will throw error if the inputs in a batch have different shapes   \n",
    "            resized = cv2.resize(image, (nb_rows,nb_cols), interpolation = cv2.INTER_AREA)\n",
    "            batch_data[folder,idx] = (resized)\n",
    "            #batch_data_aug[folder,idx] = (cv2.warpAffine(resized, M, (resized.shape[0], resized.shape[1])))      \n",
    "\n",
    "        batch_labels[folder, int(t[folder + (batch_num*batch_size)].strip().split(';')[2])] = 1\n",
    "        #batch_labels_aug[folder, int(t[folder + (batch_num*batch_size)].strip().split(';')[2])] = 1\n",
    "    \n",
    "    #batch_data = np.append(batch_data, batch_data_aug, axis = 0) \n",
    "    #batch_labels = np.append(batch_labels, batch_labels_aug, axis = 0) \n",
    "    return batch_data, batch_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_generator(source_path, folder_list, batch_size):\n",
    "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
    "    while True:\n",
    "        t = np.random.permutation(folder_list)\n",
    "        num_batches = len(folder_list)//batch_size # calculate the number of batches\n",
    "        for batch in range(num_batches): # we iterate over the number of batches\n",
    "            # you yield the batch_data and the batch_labels, remember what does yield do\n",
    "            yield train_load_batch_images(source_path, folder_list, batch, batch_size, t) \n",
    "        \n",
    "        # Code for the remaining data points which are left after full batches\n",
    "        if (len(folder_list) != batch_size*num_batches):\n",
    "            batch_size = len(folder_list) - (batch_size*num_batches)\n",
    "            yield train_load_batch_images(source_path, folder_list, num_batches, batch_size, t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def val_generator(source_path, folder_list, batch_size):\n",
    "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
    "    while True:\n",
    "        t = np.random.permutation(folder_list)\n",
    "        num_batches = len(folder_list)//batch_size # calculate the number of batches\n",
    "        for batch in range(num_batches): # we iterate over the number of batches\n",
    "            # you yield the batch_data and the batch_labels, remember what does yield do\n",
    "            yield val_load_batch_images(source_path, folder_list, batch, batch_size, t) \n",
    "        \n",
    "        # Code for the remaining data points which are left after full batches\n",
    "        if (len(folder_list) != batch_size*num_batches):\n",
    "            batch_size = len(folder_list) - (batch_size*num_batches)\n",
    "            yield val_load_batch_images(source_path, folder_list, num_batches, batch_size, t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note here that a video is represented above in the generator as (number of images, height, width, number of channels). Take this into consideration while creating the model architecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# training sequences = 663\n",
      "# validation sequences = 100\n",
      "# epochs = 10\n"
     ]
    }
   ],
   "source": [
    "curr_dt_time = datetime.datetime.now()\n",
    "train_path = './Project_data/train'\n",
    "val_path = './Project_data/val'\n",
    "num_train_sequences = len(train_doc)\n",
    "print('# training sequences =', num_train_sequences)\n",
    "num_val_sequences = len(val_doc)\n",
    "print('# validation sequences =', num_val_sequences)\n",
    "num_epochs = 10 # choose the number of epochs\n",
    "print ('# epochs =', num_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model #1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential, Model\n",
    "from keras.layers import Dense, GRU, Flatten, TimeDistributed, Flatten, BatchNormalization, Activation, Dropout\n",
    "from keras.layers.convolutional import Conv3D, MaxPooling3D\n",
    "from keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n",
    "from keras.losses import categorical_crossentropy\n",
    "from keras.optimizers import Adam\n",
    "\n",
    "nb_filters = [8,16,32,64]\n",
    "#nb_dense = [1000, 500, 5]\n",
    "nb_dense = [256, 128, 5]\n",
    "\n",
    "# Input\n",
    "input_shape=(nb_frames,nb_rows,nb_cols,nb_channel)\n",
    "\n",
    "# Define model\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv3D(nb_filters[0], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 input_shape=input_shape,\n",
    "                 padding='same'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Activation('relu'))\n",
    "\n",
    "model.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model.add(Conv3D(nb_filters[1], \n",
    "                 kernel_size=(3,3,3), \n",
    "                 padding='same'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Activation('relu'))\n",
    "\n",
    "model.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model.add(Conv3D(nb_filters[2], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Activation('relu'))\n",
    "\n",
    "model.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "model.add(Conv3D(nb_filters[3], \n",
    "                 kernel_size=(1,3,3), \n",
    "                 padding='same'))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dropout(0.25))\n",
    "\n",
    "model.add(MaxPooling3D(pool_size=(2,2,2)))\n",
    "\n",
    "#Flatten Layers\n",
    "model.add(Flatten())\n",
    "\n",
    "model.add(Dense(nb_dense[0], activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(nb_dense[1], activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "#softmax layer\n",
    "model.add(Dense(nb_dense[2], activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have written the model, the next step is to `compile` the model. When you print the `summary` of the model, you'll see the total number of parameters you have to train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv3d_1 (Conv3D)            (None, 30, 120, 120, 8)   656       \n",
      "_________________________________________________________________\n",
      "batch_normalization_1 (Batch (None, 30, 120, 120, 8)   32        \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 30, 120, 120, 8)   0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_1 (MaxPooling3 (None, 15, 60, 60, 8)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_2 (Conv3D)            (None, 15, 60, 60, 16)    3472      \n",
      "_________________________________________________________________\n",
      "batch_normalization_2 (Batch (None, 15, 60, 60, 16)    64        \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 15, 60, 60, 16)    0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_2 (MaxPooling3 (None, 7, 30, 30, 16)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_3 (Conv3D)            (None, 7, 30, 30, 32)     4640      \n",
      "_________________________________________________________________\n",
      "batch_normalization_3 (Batch (None, 7, 30, 30, 32)     128       \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 7, 30, 30, 32)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_3 (MaxPooling3 (None, 3, 15, 15, 32)     0         \n",
      "_________________________________________________________________\n",
      "conv3d_4 (Conv3D)            (None, 3, 15, 15, 64)     18496     \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 3, 15, 15, 64)     0         \n",
      "_________________________________________________________________\n",
      "max_pooling3d_4 (MaxPooling3 (None, 1, 7, 7, 64)       0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 256)               803072    \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 256)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 5)                 645       \n",
      "=================================================================\n",
      "Total params: 864,101\n",
      "Trainable params: 863,989\n",
      "Non-trainable params: 112\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "optimiser = Adam() #write your optimizer\n",
    "model.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
    "print (model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us create the `train_generator` and the `val_generator` which will be used in `.fit_generator`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_generator = train_generator(train_path, train_doc, batch_size)\n",
    "val_generator = val_generator(val_path, val_doc, batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = 'model_init' + '_' + str(curr_dt_time).replace(' ','').replace(':','_') + '/'\n",
    "    \n",
    "if not os.path.exists(model_name):\n",
    "    os.mkdir(model_name)\n",
    "        \n",
    "filepath = model_name + 'model-{epoch:05d}-{loss:.5f}-{categorical_accuracy:.5f}-{val_loss:.5f}-{val_categorical_accuracy:.5f}.h5'\n",
    "\n",
    "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto', period=1)\n",
    "\n",
    "# write the Reducelronplateau code here\n",
    "LR = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=2, cooldown=1, verbose=1)\n",
    "callbacks_list = [checkpoint, LR]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `steps_per_epoch` and `validation_steps` are used by `fit_generator` to decide the number of next() calls it need to make."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (num_train_sequences%batch_size) == 0:\n",
    "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
    "else:\n",
    "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
    "\n",
    "if (num_val_sequences%batch_size) == 0:\n",
    "    validation_steps = int(num_val_sequences/batch_size)\n",
    "else:\n",
    "    validation_steps = (num_val_sequences//batch_size) + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us now fit the model. This will start training the model and with the help of the checkpoints, you'll be able to save the model at the end of each epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "Source path =  ./Project_data/train ; batch size = 10\n",
      "Source path =  ./Project_data/val ; batch size = 10\n",
      "67/67 [==============================] - 34s 506ms/step - loss: 1.7913 - categorical_accuracy: 0.2642 - val_loss: 1.3362 - val_categorical_accuracy: 0.3800\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-10-2123_33_27.004908/model-00001-1.79123-0.26697-1.33616-0.38000.h5\n",
      "Epoch 2/10\n",
      "67/67 [==============================] - 11s 158ms/step - loss: 1.4648 - categorical_accuracy: 0.3557 - val_loss: 1.5657 - val_categorical_accuracy: 0.3400\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-10-2123_33_27.004908/model-00002-1.46481-0.35572-1.56571-0.34000.h5\n",
      "Epoch 3/10\n",
      "67/67 [==============================] - 11s 164ms/step - loss: 1.4933 - categorical_accuracy: 0.3010 - val_loss: 1.2789 - val_categorical_accuracy: 0.4400\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-10-2123_33_27.004908/model-00003-1.49330-0.30100-1.27886-0.44000.h5\n",
      "Epoch 4/10\n",
      "67/67 [==============================] - 11s 165ms/step - loss: 1.3551 - categorical_accuracy: 0.4279 - val_loss: 1.2661 - val_categorical_accuracy: 0.4600\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-10-2123_33_27.004908/model-00004-1.35508-0.42786-1.26606-0.46000.h5\n",
      "Epoch 5/10\n",
      "67/67 [==============================] - 11s 165ms/step - loss: 1.1417 - categorical_accuracy: 0.5473 - val_loss: 1.2926 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-10-2123_33_27.004908/model-00005-1.14169-0.54726-1.29265-0.57000.h5\n",
      "Epoch 6/10\n",
      "67/67 [==============================] - 11s 170ms/step - loss: 1.1451 - categorical_accuracy: 0.5348 - val_loss: 1.1653 - val_categorical_accuracy: 0.5100\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-10-2123_33_27.004908/model-00006-1.14512-0.53483-1.16530-0.51000.h5\n",
      "Epoch 7/10\n",
      "67/67 [==============================] - 12s 175ms/step - loss: 1.1703 - categorical_accuracy: 0.5373 - val_loss: 1.0285 - val_categorical_accuracy: 0.5700\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-10-2123_33_27.004908/model-00007-1.17029-0.53731-1.02848-0.57000.h5\n",
      "Epoch 8/10\n",
      "67/67 [==============================] - 11s 166ms/step - loss: 1.1944 - categorical_accuracy: 0.5299 - val_loss: 1.0894 - val_categorical_accuracy: 0.5800\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-10-2123_33_27.004908/model-00008-1.19442-0.52985-1.08940-0.58000.h5\n",
      "Epoch 9/10\n",
      "67/67 [==============================] - 11s 162ms/step - loss: 1.0257 - categorical_accuracy: 0.5945 - val_loss: 0.9788 - val_categorical_accuracy: 0.6600\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-10-2123_33_27.004908/model-00009-1.02573-0.59453-0.97879-0.66000.h5\n",
      "Epoch 10/10\n",
      "67/67 [==============================] - 11s 171ms/step - loss: 0.9544 - categorical_accuracy: 0.6393 - val_loss: 2.0086 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-10-2123_33_27.004908/model-00010-0.95436-0.63930-2.00859-0.40000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fd198992748>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 10\n",
    "num_epochs = 10\n",
    "model.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "8/8 [==============================] - 1s 145ms/step - loss: 0.7334 - categorical_accuracy: 0.6667 - val_loss: 1.3478 - val_categorical_accuracy: 0.4000\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-10-2123_33_27.004908/model-00001-0.73345-0.66667-1.34779-0.40000.h5\n",
      "Epoch 2/20\n",
      "8/8 [==============================] - 1s 140ms/step - loss: 0.8172 - categorical_accuracy: 0.6458 - val_loss: 0.6057 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-10-2123_33_27.004908/model-00002-0.81718-0.64583-0.60570-0.80000.h5\n",
      "Epoch 3/20\n",
      "8/8 [==============================] - 1s 163ms/step - loss: 0.8426 - categorical_accuracy: 0.7917 - val_loss: 0.7247 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-10-2123_33_27.004908/model-00003-0.84260-0.79167-0.72467-0.70000.h5\n",
      "Epoch 4/20\n",
      "8/8 [==============================] - 1s 166ms/step - loss: 0.6623 - categorical_accuracy: 0.7083 - val_loss: 0.6912 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-10-2123_33_27.004908/model-00004-0.66233-0.70833-0.69119-0.70000.h5\n",
      "\n",
      "Epoch 00004: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\n",
      "Epoch 5/20\n",
      "8/8 [==============================] - 1s 148ms/step - loss: 0.8037 - categorical_accuracy: 0.6875 - val_loss: 1.0085 - val_categorical_accuracy: 0.5000\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-10-2123_33_27.004908/model-00005-0.80366-0.68750-1.00853-0.50000.h5\n",
      "Epoch 6/20\n",
      "8/8 [==============================] - 1s 144ms/step - loss: 0.8339 - categorical_accuracy: 0.7500 - val_loss: 0.6428 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-10-2123_33_27.004908/model-00006-0.83391-0.75000-0.64277-0.70000.h5\n",
      "\n",
      "Epoch 00006: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.\n",
      "Epoch 7/20\n",
      "8/8 [==============================] - 2s 204ms/step - loss: 0.8713 - categorical_accuracy: 0.7083 - val_loss: 0.5018 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-10-2123_33_27.004908/model-00007-0.87133-0.70833-0.50176-0.80000.h5\n",
      "Epoch 8/20\n",
      "8/8 [==============================] - 1s 164ms/step - loss: 0.7323 - categorical_accuracy: 0.7708 - val_loss: 0.6352 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-10-2123_33_27.004908/model-00008-0.73230-0.77083-0.63519-0.80000.h5\n",
      "Epoch 9/20\n",
      "8/8 [==============================] - 1s 147ms/step - loss: 0.5546 - categorical_accuracy: 0.8333 - val_loss: 0.7305 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-10-2123_33_27.004908/model-00009-0.55459-0.83333-0.73054-0.70000.h5\n",
      "\n",
      "Epoch 00009: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.\n",
      "Epoch 10/20\n",
      "8/8 [==============================] - 1s 161ms/step - loss: 0.7688 - categorical_accuracy: 0.6667 - val_loss: 0.8733 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-10-2123_33_27.004908/model-00010-0.76877-0.66667-0.87330-0.65000.h5\n",
      "Epoch 11/20\n",
      "8/8 [==============================] - 1s 180ms/step - loss: 0.8285 - categorical_accuracy: 0.6458 - val_loss: 0.5443 - val_categorical_accuracy: 0.9000\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-10-2123_33_27.004908/model-00011-0.82854-0.64583-0.54426-0.90000.h5\n",
      "\n",
      "Epoch 00011: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.\n",
      "Epoch 12/20\n",
      "8/8 [==============================] - 1s 146ms/step - loss: 0.7721 - categorical_accuracy: 0.6667 - val_loss: 0.6966 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-10-2123_33_27.004908/model-00012-0.77209-0.66667-0.69659-0.70000.h5\n",
      "Epoch 13/20\n",
      "8/8 [==============================] - 1s 153ms/step - loss: 0.6149 - categorical_accuracy: 0.7500 - val_loss: 0.5037 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-10-2123_33_27.004908/model-00013-0.61491-0.75000-0.50371-0.85000.h5\n",
      "\n",
      "Epoch 00013: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.\n",
      "Epoch 14/20\n",
      "8/8 [==============================] - 1s 185ms/step - loss: 0.5400 - categorical_accuracy: 0.7917 - val_loss: 0.7648 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-10-2123_33_27.004908/model-00014-0.53998-0.79167-0.76479-0.70000.h5\n",
      "Epoch 15/20\n",
      "8/8 [==============================] - 1s 143ms/step - loss: 0.5114 - categorical_accuracy: 0.7917 - val_loss: 0.6784 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-10-2123_33_27.004908/model-00015-0.51138-0.79167-0.67843-0.75000.h5\n",
      "\n",
      "Epoch 00015: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.\n",
      "Epoch 16/20\n",
      "8/8 [==============================] - 2s 210ms/step - loss: 0.6584 - categorical_accuracy: 0.7292 - val_loss: 0.7010 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-10-2123_33_27.004908/model-00016-0.65837-0.72917-0.70101-0.75000.h5\n",
      "Epoch 17/20\n",
      "8/8 [==============================] - 2s 189ms/step - loss: 0.7942 - categorical_accuracy: 0.6667 - val_loss: 0.5944 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-10-2123_33_27.004908/model-00017-0.79415-0.66667-0.59441-0.75000.h5\n",
      "\n",
      "Epoch 00017: ReduceLROnPlateau reducing learning rate to 7.812500371073838e-06.\n",
      "Epoch 18/20\n",
      "8/8 [==============================] - 1s 142ms/step - loss: 0.9278 - categorical_accuracy: 0.5833 - val_loss: 0.6079 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-10-2123_33_27.004908/model-00018-0.92785-0.58333-0.60789-0.85000.h5\n",
      "Epoch 19/20\n",
      "8/8 [==============================] - 1s 133ms/step - loss: 1.0036 - categorical_accuracy: 0.6458 - val_loss: 0.6861 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-10-2123_33_27.004908/model-00019-1.00356-0.64583-0.68611-0.85000.h5\n",
      "\n",
      "Epoch 00019: ReduceLROnPlateau reducing learning rate to 3.906250185536919e-06.\n",
      "Epoch 20/20\n",
      "8/8 [==============================] - 1s 169ms/step - loss: 0.7551 - categorical_accuracy: 0.7500 - val_loss: 0.7940 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-10-2123_33_27.004908/model-00020-0.75511-0.75000-0.79403-0.70000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fd19002e8d0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 90\n",
    "num_epochs = 20\n",
    "\n",
    "if (num_train_sequences%batch_size) == 0:\n",
    "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
    "else:\n",
    "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
    "\n",
    "if (num_val_sequences%batch_size) == 0:\n",
    "    validation_steps = int(num_val_sequences/batch_size)\n",
    "else:\n",
    "    validation_steps = (num_val_sequences//batch_size) + 1\n",
    "    \n",
    "\n",
    "model.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "8/8 [==============================] - 1s 175ms/step - loss: 0.4295 - categorical_accuracy: 0.8125 - val_loss: 0.7857 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00001: saving model to model_init_2018-10-2123_33_27.004908/model-00001-0.42952-0.81250-0.78568-0.70000.h5\n",
      "Epoch 2/30\n",
      "8/8 [==============================] - 1s 161ms/step - loss: 0.7759 - categorical_accuracy: 0.7292 - val_loss: 0.4927 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00002: saving model to model_init_2018-10-2123_33_27.004908/model-00002-0.77590-0.72917-0.49267-0.85000.h5\n",
      "Epoch 3/30\n",
      "8/8 [==============================] - 2s 194ms/step - loss: 0.7623 - categorical_accuracy: 0.7083 - val_loss: 0.7179 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00003: saving model to model_init_2018-10-2123_33_27.004908/model-00003-0.76229-0.70833-0.71788-0.70000.h5\n",
      "Epoch 4/30\n",
      "8/8 [==============================] - 1s 184ms/step - loss: 0.5716 - categorical_accuracy: 0.8750 - val_loss: 0.5802 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00004: saving model to model_init_2018-10-2123_33_27.004908/model-00004-0.57156-0.87500-0.58023-0.80000.h5\n",
      "\n",
      "Epoch 00004: ReduceLROnPlateau reducing learning rate to 1.9531250927684596e-06.\n",
      "Epoch 5/30\n",
      "8/8 [==============================] - 1s 155ms/step - loss: 0.4871 - categorical_accuracy: 0.8125 - val_loss: 0.5417 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00005: saving model to model_init_2018-10-2123_33_27.004908/model-00005-0.48712-0.81250-0.54171-0.85000.h5\n",
      "Epoch 6/30\n",
      "8/8 [==============================] - 1s 168ms/step - loss: 0.3835 - categorical_accuracy: 0.9583 - val_loss: 0.6981 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00006: saving model to model_init_2018-10-2123_33_27.004908/model-00006-0.38354-0.95833-0.69807-0.75000.h5\n",
      "\n",
      "Epoch 00006: ReduceLROnPlateau reducing learning rate to 9.765625463842298e-07.\n",
      "Epoch 7/30\n",
      "8/8 [==============================] - 1s 177ms/step - loss: 0.7179 - categorical_accuracy: 0.6875 - val_loss: 0.6427 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00007: saving model to model_init_2018-10-2123_33_27.004908/model-00007-0.71788-0.68750-0.64269-0.85000.h5\n",
      "Epoch 8/30\n",
      "8/8 [==============================] - 2s 217ms/step - loss: 0.5273 - categorical_accuracy: 0.7917 - val_loss: 0.4972 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00008: saving model to model_init_2018-10-2123_33_27.004908/model-00008-0.52733-0.79167-0.49719-0.85000.h5\n",
      "\n",
      "Epoch 00008: ReduceLROnPlateau reducing learning rate to 4.882812731921149e-07.\n",
      "Epoch 9/30\n",
      "8/8 [==============================] - 1s 170ms/step - loss: 0.7609 - categorical_accuracy: 0.6875 - val_loss: 0.8868 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00009: saving model to model_init_2018-10-2123_33_27.004908/model-00009-0.76086-0.68750-0.88677-0.65000.h5\n",
      "Epoch 10/30\n",
      "8/8 [==============================] - 1s 162ms/step - loss: 0.6507 - categorical_accuracy: 0.8125 - val_loss: 0.4098 - val_categorical_accuracy: 0.9500\n",
      "\n",
      "Epoch 00010: saving model to model_init_2018-10-2123_33_27.004908/model-00010-0.65067-0.81250-0.40977-0.95000.h5\n",
      "Epoch 11/30\n",
      "8/8 [==============================] - 1s 153ms/step - loss: 0.5943 - categorical_accuracy: 0.7917 - val_loss: 0.7146 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00011: saving model to model_init_2018-10-2123_33_27.004908/model-00011-0.59427-0.79167-0.71461-0.65000.h5\n",
      "Epoch 12/30\n",
      "8/8 [==============================] - 1s 130ms/step - loss: 0.5323 - categorical_accuracy: 0.8333 - val_loss: 0.6580 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00012: saving model to model_init_2018-10-2123_33_27.004908/model-00012-0.53230-0.83333-0.65802-0.80000.h5\n",
      "\n",
      "Epoch 00012: ReduceLROnPlateau reducing learning rate to 2.4414063659605745e-07.\n",
      "Epoch 13/30\n",
      "8/8 [==============================] - 1s 175ms/step - loss: 0.6319 - categorical_accuracy: 0.7500 - val_loss: 0.6848 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00013: saving model to model_init_2018-10-2123_33_27.004908/model-00013-0.63188-0.75000-0.68484-0.75000.h5\n",
      "Epoch 14/30\n",
      "8/8 [==============================] - 1s 173ms/step - loss: 0.7757 - categorical_accuracy: 0.6458 - val_loss: 0.5028 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00014: saving model to model_init_2018-10-2123_33_27.004908/model-00014-0.77571-0.64583-0.50279-0.85000.h5\n",
      "\n",
      "Epoch 00014: ReduceLROnPlateau reducing learning rate to 1.2207031829802872e-07.\n",
      "Epoch 15/30\n",
      "8/8 [==============================] - 1s 182ms/step - loss: 0.7769 - categorical_accuracy: 0.6875 - val_loss: 0.8883 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00015: saving model to model_init_2018-10-2123_33_27.004908/model-00015-0.77692-0.68750-0.88828-0.70000.h5\n",
      "Epoch 16/30\n",
      "8/8 [==============================] - 1s 145ms/step - loss: 0.6725 - categorical_accuracy: 0.7083 - val_loss: 0.5526 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00016: saving model to model_init_2018-10-2123_33_27.004908/model-00016-0.67248-0.70833-0.55262-0.75000.h5\n",
      "\n",
      "Epoch 00016: ReduceLROnPlateau reducing learning rate to 6.103515914901436e-08.\n",
      "Epoch 17/30\n",
      "8/8 [==============================] - 1s 169ms/step - loss: 0.7197 - categorical_accuracy: 0.7083 - val_loss: 0.5279 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00017: saving model to model_init_2018-10-2123_33_27.004908/model-00017-0.71970-0.70833-0.52793-0.85000.h5\n",
      "Epoch 18/30\n",
      "8/8 [==============================] - 1s 162ms/step - loss: 0.7392 - categorical_accuracy: 0.7917 - val_loss: 0.6286 - val_categorical_accuracy: 0.9000\n",
      "\n",
      "Epoch 00018: saving model to model_init_2018-10-2123_33_27.004908/model-00018-0.73922-0.79167-0.62859-0.90000.h5\n",
      "\n",
      "Epoch 00018: ReduceLROnPlateau reducing learning rate to 3.051757957450718e-08.\n",
      "Epoch 19/30\n",
      "8/8 [==============================] - 1s 158ms/step - loss: 0.6185 - categorical_accuracy: 0.8125 - val_loss: 0.6613 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00019: saving model to model_init_2018-10-2123_33_27.004908/model-00019-0.61851-0.81250-0.66130-0.80000.h5\n",
      "Epoch 20/30\n",
      "8/8 [==============================] - 1s 156ms/step - loss: 0.5423 - categorical_accuracy: 0.8125 - val_loss: 0.4465 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00020: saving model to model_init_2018-10-2123_33_27.004908/model-00020-0.54233-0.81250-0.44647-0.85000.h5\n",
      "\n",
      "Epoch 00020: ReduceLROnPlateau reducing learning rate to 1.525878978725359e-08.\n",
      "Epoch 21/30\n",
      "8/8 [==============================] - 1s 147ms/step - loss: 0.7219 - categorical_accuracy: 0.6875 - val_loss: 0.7360 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00021: saving model to model_init_2018-10-2123_33_27.004908/model-00021-0.72188-0.68750-0.73604-0.65000.h5\n",
      "Epoch 22/30\n",
      "8/8 [==============================] - 1s 169ms/step - loss: 0.6068 - categorical_accuracy: 0.7708 - val_loss: 0.6755 - val_categorical_accuracy: 0.7000\n",
      "\n",
      "Epoch 00022: saving model to model_init_2018-10-2123_33_27.004908/model-00022-0.60678-0.77083-0.67554-0.70000.h5\n",
      "\n",
      "Epoch 00022: ReduceLROnPlateau reducing learning rate to 7.629394893626795e-09.\n",
      "Epoch 23/30\n",
      "8/8 [==============================] - 1s 138ms/step - loss: 0.6748 - categorical_accuracy: 0.7292 - val_loss: 0.7459 - val_categorical_accuracy: 0.7500\n",
      "\n",
      "Epoch 00023: saving model to model_init_2018-10-2123_33_27.004908/model-00023-0.67477-0.72917-0.74588-0.75000.h5\n",
      "Epoch 24/30\n",
      "8/8 [==============================] - 1s 161ms/step - loss: 0.6200 - categorical_accuracy: 0.7500 - val_loss: 0.4217 - val_categorical_accuracy: 0.8500\n",
      "\n",
      "Epoch 00024: saving model to model_init_2018-10-2123_33_27.004908/model-00024-0.62002-0.75000-0.42175-0.85000.h5\n",
      "\n",
      "Epoch 00024: ReduceLROnPlateau reducing learning rate to 3.814697446813398e-09.\n",
      "Epoch 25/30\n",
      "8/8 [==============================] - 1s 167ms/step - loss: 0.8270 - categorical_accuracy: 0.7292 - val_loss: 0.6390 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00025: saving model to model_init_2018-10-2123_33_27.004908/model-00025-0.82698-0.72917-0.63896-0.80000.h5\n",
      "Epoch 26/30\n",
      "8/8 [==============================] - 1s 171ms/step - loss: 0.6023 - categorical_accuracy: 0.8542 - val_loss: 0.8428 - val_categorical_accuracy: 0.6000\n",
      "\n",
      "Epoch 00026: saving model to model_init_2018-10-2123_33_27.004908/model-00026-0.60235-0.85417-0.84277-0.60000.h5\n",
      "\n",
      "Epoch 00026: ReduceLROnPlateau reducing learning rate to 1.907348723406699e-09.\n",
      "Epoch 27/30\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8/8 [==============================] - 1s 159ms/step - loss: 0.6169 - categorical_accuracy: 0.7917 - val_loss: 0.4864 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00027: saving model to model_init_2018-10-2123_33_27.004908/model-00027-0.61689-0.79167-0.48638-0.80000.h5\n",
      "Epoch 28/30\n",
      "8/8 [==============================] - 1s 144ms/step - loss: 0.5624 - categorical_accuracy: 0.8125 - val_loss: 0.7181 - val_categorical_accuracy: 0.6500\n",
      "\n",
      "Epoch 00028: saving model to model_init_2018-10-2123_33_27.004908/model-00028-0.56237-0.81250-0.71811-0.65000.h5\n",
      "\n",
      "Epoch 00028: ReduceLROnPlateau reducing learning rate to 9.536743617033494e-10.\n",
      "Epoch 29/30\n",
      "8/8 [==============================] - 1s 170ms/step - loss: 0.5491 - categorical_accuracy: 0.8333 - val_loss: 0.3813 - val_categorical_accuracy: 0.9000\n",
      "\n",
      "Epoch 00029: saving model to model_init_2018-10-2123_33_27.004908/model-00029-0.54907-0.83333-0.38130-0.90000.h5\n",
      "Epoch 30/30\n",
      "8/8 [==============================] - 1s 147ms/step - loss: 0.5535 - categorical_accuracy: 0.7708 - val_loss: 0.6076 - val_categorical_accuracy: 0.8000\n",
      "\n",
      "Epoch 00030: saving model to model_init_2018-10-2123_33_27.004908/model-00030-0.55353-0.77083-0.60764-0.80000.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x7fd190067550>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "batch_size = 90\n",
    "num_epochs = 30\n",
    "\n",
    "if (num_train_sequences%batch_size) == 0:\n",
    "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
    "else:\n",
    "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
    "\n",
    "if (num_val_sequences%batch_size) == 0:\n",
    "    validation_steps = int(num_val_sequences/batch_size)\n",
    "else:\n",
    "    validation_steps = (num_val_sequences//batch_size) + 1\n",
    "    \n",
    "\n",
    "model.fit_generator(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=1, \n",
    "                    callbacks=callbacks_list, validation_data=val_generator, \n",
    "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}