dlnd_language_translation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Language Translation\n",
    "In this project, you’re going to take a peek into the realm of neural network machine translation.  You’ll be training a sequence to sequence model on a dataset of English and French sentences that can translate new sentences from English to French.\n",
    "## Get the Data\n",
    "Since translating the whole language of English to French will take lots of time to train, we have provided you with a small portion of the English corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import helper\n",
    "import problem_unittests as tests\n",
    "\n",
    "source_path = 'data/small_vocab_en'\n",
    "target_path = 'data/small_vocab_fr'\n",
    "source_text = helper.load_data(source_path)\n",
    "target_text = helper.load_data(target_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the Data\n",
    "Play around with view_sentence_range to view different parts of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset Stats\n",
      "Roughly the number of unique words: 227\n",
      "Number of sentences: 137861\n",
      "Average number of words in a sentence: 13.225277634719028\n",
      "\n",
      "English sentences 0 to 10:\n",
      "new jersey is sometimes quiet during autumn , and it is snowy in april .\n",
      "the united states is usually chilly during july , and it is usually freezing in november .\n",
      "california is usually quiet during march , and it is usually hot in june .\n",
      "the united states is sometimes mild during june , and it is cold in september .\n",
      "your least liked fruit is the grape , but my least liked is the apple .\n",
      "his favorite fruit is the orange , but my favorite is the grape .\n",
      "paris is relaxing during december , but it is usually chilly in july .\n",
      "new jersey is busy during spring , and it is never hot in march .\n",
      "our least liked fruit is the lemon , but my least liked is the grape .\n",
      "the united states is sometimes busy during january , and it is sometimes warm in november .\n",
      "\n",
      "French sentences 0 to 10:\n",
      "new jersey est parfois calme pendant l' automne , et il est neigeux en avril .\n",
      "les états-unis est généralement froid en juillet , et il gèle habituellement en novembre .\n",
      "california est généralement calme en mars , et il est généralement chaud en juin .\n",
      "les états-unis est parfois légère en juin , et il fait froid en septembre .\n",
      "votre moins aimé fruit est le raisin , mais mon moins aimé est la pomme .\n",
      "son fruit préféré est l'orange , mais mon préféré est le raisin .\n",
      "paris est relaxant en décembre , mais il est généralement froid en juillet .\n",
      "new jersey est occupé au printemps , et il est jamais chaude en mars .\n",
      "notre fruit est moins aimé le citron , mais mon moins aimé est le raisin .\n",
      "les états-unis est parfois occupé en janvier , et il est parfois chaud en novembre .\n"
     ]
    }
   ],
   "source": [
    "view_sentence_range = (0, 10)\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import numpy as np\n",
    "\n",
    "print('Dataset Stats')\n",
    "print('Roughly the number of unique words: {}'.format(len({word: None for word in source_text.split()})))\n",
    "\n",
    "sentences = source_text.split('\\n')\n",
    "word_counts = [len(sentence.split()) for sentence in sentences]\n",
    "print('Number of sentences: {}'.format(len(sentences)))\n",
    "print('Average number of words in a sentence: {}'.format(np.average(word_counts)))\n",
    "\n",
    "print()\n",
    "print('English sentences {} to {}:'.format(*view_sentence_range))\n",
    "print('\\n'.join(source_text.split('\\n')[view_sentence_range[0]:view_sentence_range[1]]))\n",
    "print()\n",
    "print('French sentences {} to {}:'.format(*view_sentence_range))\n",
    "print('\\n'.join(target_text.split('\\n')[view_sentence_range[0]:view_sentence_range[1]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement Preprocessing Function\n",
    "### Text to Word Ids\n",
    "As you did with other RNNs, you must turn the text into a number so the computer can understand it. In the function `text_to_ids()`, you'll turn `source_text` and `target_text` from words to ids.  However, you need to add the `<EOS>` word id at the end of each sentence from `target_text`.  This will help the neural network predict when the sentence should end.\n",
    "\n",
    "You can get the `<EOS>` word id by doing:\n",
    "```python\n",
    "target_vocab_to_int['<EOS>']\n",
    "```\n",
    "You can get other word ids using `source_vocab_to_int` and `target_vocab_to_int`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def text_to_ids(source_text, target_text, source_vocab_to_int, target_vocab_to_int):\n",
    "    \"\"\"\n",
    "    Convert source and target text to proper word ids\n",
    "    :param source_text: String that contains all the source text.\n",
    "    :param target_text: String that contains all the target text.\n",
    "    :param source_vocab_to_int: Dictionary to go from the source words to an id\n",
    "    :param target_vocab_to_int: Dictionary to go from the target words to an id\n",
    "    :return: A tuple of lists (source_id_text, target_id_text)\n",
    "    \"\"\"\n",
    "    source_lines = source_text.split('\\n')\n",
    "    target_lines = [line + ' <EOS>' for line in target_text.split('\\n')]\n",
    "\n",
    "    source_ids = list(map(lambda x: [source_vocab_to_int[word] for word in x.split()], source_lines))\n",
    "    target_ids = list(map(lambda x: [target_vocab_to_int[word] for word in x.split()], target_lines))\n",
    "\n",
    "    return source_ids, target_ids\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_text_to_ids(text_to_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocess all the data and save it\n",
    "Running the code cell below will preprocess all the data and save it to file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "helper.preprocess_and_save_data(source_path, target_path, text_to_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Check Point\n",
    "This is your first checkpoint. If you ever decide to come back to this notebook or have to restart the notebook, you can start from here. The preprocessed data has been saved to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import numpy as np\n",
    "import helper\n",
    "\n",
    "(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check the Version of TensorFlow and Access to GPU\n",
    "This will check to make sure you have the correct version of TensorFlow and access to a GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow Version: 1.0.0\n",
      "Default GPU Device: /gpu:0\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "from distutils.version import LooseVersion\n",
    "import warnings\n",
    "import tensorflow as tf\n",
    "\n",
    "# Check TensorFlow Version\n",
    "assert LooseVersion(tf.__version__) in [LooseVersion('1.0.0'), LooseVersion('1.0.1')], 'This project requires TensorFlow version 1.0  You are using {}'.format(tf.__version__)\n",
    "print('TensorFlow Version: {}'.format(tf.__version__))\n",
    "\n",
    "# Check for a GPU\n",
    "if not tf.test.gpu_device_name():\n",
    "    warnings.warn('No GPU found. Please use a GPU to train your neural network.')\n",
    "else:\n",
    "    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the Neural Network\n",
    "You'll build the components necessary to build a Sequence-to-Sequence model by implementing the following functions below:\n",
    "- `model_inputs`\n",
    "- `process_decoding_input`\n",
    "- `encoding_layer`\n",
    "- `decoding_layer_train`\n",
    "- `decoding_layer_infer`\n",
    "- `decoding_layer`\n",
    "- `seq2seq_model`\n",
    "\n",
    "### Input\n",
    "Implement the `model_inputs()` function to create TF Placeholders for the Neural Network. It should create the following placeholders:\n",
    "\n",
    "- Input text placeholder named \"input\" using the TF Placeholder name parameter with rank 2.\n",
    "- Targets placeholder with rank 2.\n",
    "- Learning rate placeholder with rank 0.\n",
    "- Keep probability placeholder named \"keep_prob\" using the TF Placeholder name parameter with rank 0.\n",
    "\n",
    "Return the placeholders in the following the tuple (Input, Targets, Learing Rate, Keep Probability)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def model_inputs():\n",
    "    \"\"\"\n",
    "    Create TF Placeholders for input, targets, and learning rate.\n",
    "    :return: Tuple (input, targets, learning rate, keep probability)\n",
    "    \"\"\"\n",
    "    ph_input = tf.placeholder(tf.int32, shape=(None, None), name='input')\n",
    "    ph_targets = tf.placeholder(tf.int32, shape=(None, None))\n",
    "    ph_lrate = tf.placeholder(tf.float32)\n",
    "    ph_keep_prob = tf.placeholder(tf.float32, name='keep_prob')\n",
    "\n",
    "    return ph_input, ph_targets, ph_lrate, ph_keep_prob\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_model_inputs(model_inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Process Decoding Input\n",
    "Implement `process_decoding_input` using TensorFlow to remove the last word id from each batch in `target_data` and concat the GO ID to the begining of each batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def process_decoding_input(target_data, target_vocab_to_int, batch_size):\n",
    "    \"\"\"\n",
    "    Preprocess target data for dencoding\n",
    "    :param target_data: Target Placehoder\n",
    "    :param target_vocab_to_int: Dictionary to go from the target words to an id\n",
    "    :param batch_size: Batch Size\n",
    "    :return: Preprocessed target data\n",
    "    \"\"\"\n",
    "    go_id = target_vocab_to_int['<GO>']\n",
    "    length = target_data.shape[0]\n",
    "    \n",
    "    ends = tf.strided_slice(target_data, [0,0], [batch_size, -1], [1,1])\n",
    "    finals = tf.concat([tf.fill([batch_size, 1], go_id), ends], 1)\n",
    "    \n",
    "    return finals\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_process_decoding_input(process_decoding_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encoding\n",
    "Implement `encoding_layer()` to create a Encoder RNN layer using [`tf.nn.dynamic_rnn()`](https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob):\n",
    "    \"\"\"\n",
    "    Create encoding layer\n",
    "    :param rnn_inputs: Inputs for the RNN\n",
    "    :param rnn_size: RNN Size\n",
    "    :param num_layers: Number of layers\n",
    "    :param keep_prob: Dropout keep probability\n",
    "    :return: RNN state\n",
    "    \"\"\"\n",
    "    lstm = tf.contrib.rnn.BasicLSTMCell(num_units=rnn_size)\n",
    "    cell = tf.contrib.rnn.MultiRNNCell([lstm] * num_layers)\n",
    "    _, state = tf.nn.dynamic_rnn(cell, rnn_inputs, dtype=tf.float32)#, initial_state=initial_state)\n",
    "    return state\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_encoding_layer(encoding_layer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decoding - Training\n",
    "Create training logits using [`tf.contrib.seq2seq.simple_decoder_fn_train()`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/simple_decoder_fn_train) and [`tf.contrib.seq2seq.dynamic_rnn_decoder()`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_rnn_decoder).  Apply the `output_fn` to the [`tf.contrib.seq2seq.dynamic_rnn_decoder()`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_rnn_decoder) outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def decoding_layer_train(encoder_state, dec_cell, dec_embed_input, sequence_length, decoding_scope,\n",
    "                         output_fn, keep_prob):\n",
    "    \"\"\"\n",
    "    Create a decoding layer for training\n",
    "    :param encoder_state: Encoder State\n",
    "    :param dec_cell: Decoder RNN Cell\n",
    "    :param dec_embed_input: Decoder embedded input\n",
    "    :param sequence_length: Sequence Length\n",
    "    :param decoding_scope: TenorFlow Variable Scope for decoding\n",
    "    :param output_fn: Function to apply the output layer\n",
    "    :param keep_prob: Dropout keep probability\n",
    "    :return: Train Logits\n",
    "    \"\"\"\n",
    "    decoder_fn = tf.contrib.seq2seq.simple_decoder_fn_train(encoder_state, name='decoder_fn_train')\n",
    "    decoder_drop = tf.contrib.rnn.DropoutWrapper(dec_cell, keep_prob)\n",
    "    decoder_outputs,_,__ = tf.contrib.seq2seq.dynamic_rnn_decoder(\n",
    "        decoder_drop, \n",
    "        decoder_fn, \n",
    "        inputs=dec_embed_input,\n",
    "        sequence_length=sequence_length,\n",
    "        scope=decoding_scope\n",
    "    )\n",
    "    return output_fn(decoder_outputs)\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_decoding_layer_train(decoding_layer_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decoding - Inference\n",
    "Create inference logits using [`tf.contrib.seq2seq.simple_decoder_fn_inference()`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/simple_decoder_fn_inference) and [`tf.contrib.seq2seq.dynamic_rnn_decoder()`](https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/dynamic_rnn_decoder). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id,\n",
    "                         maximum_length, vocab_size, decoding_scope, output_fn, keep_prob):\n",
    "    \"\"\"\n",
    "    Create a decoding layer for inference\n",
    "    :param encoder_state: Encoder state\n",
    "    :param dec_cell: Decoder RNN Cell\n",
    "    :param dec_embeddings: Decoder embeddings\n",
    "    :param start_of_sequence_id: GO ID\n",
    "    :param end_of_sequence_id: EOS Id\n",
    "    :param maximum_length: The maximum allowed time steps to decode\n",
    "    :param vocab_size: Size of vocabulary\n",
    "    :param decoding_scope: TensorFlow Variable Scope for decoding\n",
    "    :param output_fn: Function to apply the output layer\n",
    "    :param keep_prob: Dropout keep probability\n",
    "    :return: Inference Logits\n",
    "    \"\"\"\n",
    "    decoder_fn = tf.contrib.seq2seq.simple_decoder_fn_inference(\n",
    "        output_fn=output_fn,\n",
    "        encoder_state=encoder_state,\n",
    "        embeddings=dec_embeddings,\n",
    "        start_of_sequence_id=start_of_sequence_id,\n",
    "        end_of_sequence_id=end_of_sequence_id,\n",
    "        maximum_length=maximum_length,\n",
    "        num_decoder_symbols=vocab_size,\n",
    "        name='inference_decoder_fn'\n",
    "    )\n",
    "    \n",
    "    outputs,_,__ = tf.contrib.seq2seq.dynamic_rnn_decoder(\n",
    "        cell=dec_cell,\n",
    "        decoder_fn=decoder_fn,\n",
    "        scope=decoding_scope,\n",
    "        name='inference_decoder_rnn'\n",
    "    )\n",
    "    return outputs\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_decoding_layer_infer(decoding_layer_infer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the Decoding Layer\n",
    "Implement `decoding_layer()` to create a Decoder RNN layer.\n",
    "\n",
    "- Create RNN cell for decoding using `rnn_size` and `num_layers`.\n",
    "- Create the output fuction using [`lambda`](https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions) to transform it's input, logits, to class logits.\n",
    "- Use the your `decoding_layer_train(encoder_state, dec_cell, dec_embed_input, sequence_length, decoding_scope, output_fn, keep_prob)` function to get the training logits.\n",
    "- Use your `decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id, end_of_sequence_id, maximum_length, vocab_size, decoding_scope, output_fn, keep_prob)` function to get the inference logits.\n",
    "\n",
    "Note: You'll need to use [tf.variable_scope](https://www.tensorflow.org/api_docs/python/tf/variable_scope) to share variables between training and inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def decoding_layer(dec_embed_input, dec_embeddings, encoder_state, vocab_size, sequence_length, rnn_size,\n",
    "                   num_layers, target_vocab_to_int, keep_prob):\n",
    "    \"\"\"\n",
    "    Create decoding layer\n",
    "    :param dec_embed_input: Decoder embedded input\n",
    "    :param dec_embeddings: Decoder embeddings\n",
    "    :param encoder_state: The encoded state\n",
    "    :param vocab_size: Size of vocabulary\n",
    "    :param sequence_length: Sequence Length\n",
    "    :param rnn_size: RNN Size\n",
    "    :param num_layers: Number of layers\n",
    "    :param target_vocab_to_int: Dictionary to go from the target words to an id\n",
    "    :param keep_prob: Dropout keep probability\n",
    "    :return: Tuple of (Training Logits, Inference Logits)\n",
    "    \"\"\"\n",
    "    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)\n",
    "    dec_cell = tf.contrib.rnn.MultiRNNCell([lstm] * num_layers)\n",
    "    output_fn = lambda x: tf.contrib.layers.fully_connected(x, vocab_size, None, scope=dec_scope)\n",
    "    start_of_sequence_id = target_vocab_to_int['<GO>']\n",
    "    end_of_sequence_id = target_vocab_to_int['<EOS>']\n",
    "    maximum_length = sequence_length\n",
    "        \n",
    "    with tf.variable_scope('training') as dec_scope:\n",
    "        train_logits = decoding_layer_train(encoder_state, dec_cell, dec_embed_input,\n",
    "                                            sequence_length, dec_scope, output_fn, keep_prob)\n",
    "        dec_scope.reuse_variables()\n",
    "        inf_logits = decoding_layer_infer(encoder_state, dec_cell, dec_embeddings, start_of_sequence_id,\n",
    "                                          end_of_sequence_id, maximum_length, vocab_size, dec_scope, output_fn, keep_prob)\n",
    "\n",
    "        return train_logits, inf_logits\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_decoding_layer(decoding_layer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the Neural Network\n",
    "Apply the functions you implemented above to:\n",
    "\n",
    "- Apply embedding to the input data for the encoder.\n",
    "- Encode the input using your `encoding_layer(rnn_inputs, rnn_size, num_layers, keep_prob)`.\n",
    "- Process target data using your `process_decoding_input(target_data, target_vocab_to_int, batch_size)` function.\n",
    "- Apply embedding to the target data for the decoder.\n",
    "- Decode the encoded input using your `decoding_layer(dec_embed_input, dec_embeddings, encoder_state, vocab_size, sequence_length, rnn_size, num_layers, target_vocab_to_int, keep_prob)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def seq2seq_model(input_data, target_data, keep_prob, batch_size, sequence_length, source_vocab_size, target_vocab_size,\n",
    "                  enc_embedding_size, dec_embedding_size, rnn_size, num_layers, target_vocab_to_int):\n",
    "    \"\"\"\n",
    "    Build the Sequence-to-Sequence part of the neural network\n",
    "    :param input_data: Input placeholder\n",
    "    :param target_data: Target placeholder\n",
    "    :param keep_prob: Dropout keep probability placeholder\n",
    "    :param batch_size: Batch Size\n",
    "    :param sequence_length: Sequence Length\n",
    "    :param source_vocab_size: Source vocabulary size\n",
    "    :param target_vocab_size: Target vocabulary size\n",
    "    :param enc_embedding_size: Decoder embedding size\n",
    "    :param dec_embedding_size: Encoder embedding size\n",
    "    :param rnn_size: RNN Size\n",
    "    :param num_layers: Number of layers\n",
    "    :param target_vocab_to_int: Dictionary to go from the target words to an id\n",
    "    :return: Tuple of (Training Logits, Inference Logits)\n",
    "    \"\"\"\n",
    "    enc_input = tf.contrib.layers.embed_sequence(input_data, source_vocab_size, enc_embedding_size)\n",
    "    enc_layer = encoding_layer(enc_input, rnn_size, num_layers, keep_prob)    \n",
    "    dec_input = process_decoding_input(target_data, target_vocab_to_int, batch_size)\n",
    "    \n",
    "    dec_embed = tf.Variable(tf.random_uniform([target_vocab_size, dec_embedding_size], minval=0))\n",
    "    target_embed = tf.nn.embedding_lookup(dec_embed, dec_input)\n",
    "    \n",
    "    train_logits, inf_logits = decoding_layer(target_embed, dec_embed, enc_layer, target_vocab_size,\n",
    "                                              sequence_length, rnn_size, num_layers, target_vocab_to_int, keep_prob)\n",
    "    return train_logits, inf_logits\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_seq2seq_model(seq2seq_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Network Training\n",
    "### Hyperparameters\n",
    "Tune the following parameters:\n",
    "\n",
    "- Set `epochs` to the number of epochs.\n",
    "- Set `batch_size` to the batch size.\n",
    "- Set `rnn_size` to the size of the RNNs.\n",
    "- Set `num_layers` to the number of layers.\n",
    "- Set `encoding_embedding_size` to the size of the embedding for the encoder.\n",
    "- Set `decoding_embedding_size` to the size of the embedding for the decoder.\n",
    "- Set `learning_rate` to the learning rate.\n",
    "- Set `keep_probability` to the Dropout keep probability"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Number of Epochs\n",
    "epochs = 10\n",
    "# Batch Size\n",
    "batch_size = 512\n",
    "# RNN Size\n",
    "rnn_size = 128\n",
    "# Number of Layers\n",
    "num_layers = 2\n",
    "# Embedding Size\n",
    "encoding_embedding_size = 256\n",
    "decoding_embedding_size = 256\n",
    "# Learning Rate\n",
    "learning_rate = 0.001\n",
    "# Dropout Keep Probability\n",
    "keep_probability = 0.75"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the Graph\n",
    "Build the graph using the neural network you implemented."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "save_path = 'checkpoints/dev'\n",
    "(source_int_text, target_int_text), (source_vocab_to_int, target_vocab_to_int), _ = helper.load_preprocess()\n",
    "max_source_sentence_length = max([len(sentence) for sentence in source_int_text])\n",
    "\n",
    "train_graph = tf.Graph()\n",
    "with train_graph.as_default():\n",
    "    input_data, targets, lr, keep_prob = model_inputs()\n",
    "    sequence_length = tf.placeholder_with_default(max_source_sentence_length, None, name='sequence_length')\n",
    "    input_shape = tf.shape(input_data)\n",
    "    \n",
    "    train_logits, inference_logits = seq2seq_model(\n",
    "        tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(source_vocab_to_int), len(target_vocab_to_int),\n",
    "        encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers, target_vocab_to_int)\n",
    "\n",
    "    tf.identity(inference_logits, 'logits')\n",
    "    with tf.name_scope(\"optimization\"):\n",
    "        # Loss function\n",
    "        cost = tf.contrib.seq2seq.sequence_loss(\n",
    "            train_logits,\n",
    "            targets,\n",
    "            tf.ones([input_shape[0], sequence_length]))\n",
    "\n",
    "        # Optimizer\n",
    "        optimizer = tf.train.AdamOptimizer(lr)\n",
    "\n",
    "        # Gradient Clipping\n",
    "        gradients = optimizer.compute_gradients(cost)\n",
    "        capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients if grad is not None]\n",
    "        train_op = optimizer.apply_gradients(capped_gradients)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train\n",
    "Train the neural network on the preprocessed data. If you have a hard time getting a good loss, check the forms to see if anyone is having the same problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   0 Batch    0/269 - Train Accuracy:  0.251, Validation Accuracy:  0.317, Loss:  5.869\n",
      "Epoch   0 Batch    1/269 - Train Accuracy:  0.234, Validation Accuracy:  0.311, Loss:  5.700\n",
      "Epoch   0 Batch    2/269 - Train Accuracy:  0.266, Validation Accuracy:  0.310, Loss:  5.503\n",
      "Epoch   0 Batch    3/269 - Train Accuracy:  0.244, Validation Accuracy:  0.310, Loss:  5.353\n",
      "Epoch   0 Batch    4/269 - Train Accuracy:  0.232, Validation Accuracy:  0.310, Loss:  5.205\n",
      "Epoch   0 Batch    5/269 - Train Accuracy:  0.233, Validation Accuracy:  0.310, Loss:  5.036\n",
      "Epoch   0 Batch    6/269 - Train Accuracy:  0.279, Validation Accuracy:  0.310, Loss:  4.748\n",
      "Epoch   0 Batch    7/269 - Train Accuracy:  0.276, Validation Accuracy:  0.310, Loss:  4.582\n",
      "Epoch   0 Batch    8/269 - Train Accuracy:  0.253, Validation Accuracy:  0.321, Loss:  4.528\n",
      "Epoch   0 Batch    9/269 - Train Accuracy:  0.280, Validation Accuracy:  0.322, Loss:  4.277\n",
      "Epoch   0 Batch   10/269 - Train Accuracy:  0.258, Validation Accuracy:  0.333, Loss:  4.250\n",
      "Epoch   0 Batch   11/269 - Train Accuracy:  0.304, Validation Accuracy:  0.341, Loss:  3.989\n",
      "Epoch   0 Batch   12/269 - Train Accuracy:  0.278, Validation Accuracy:  0.341, Loss:  3.987\n",
      "Epoch   0 Batch   13/269 - Train Accuracy:  0.343, Validation Accuracy:  0.341, Loss:  3.603\n",
      "Epoch   0 Batch   14/269 - Train Accuracy:  0.304, Validation Accuracy:  0.342, Loss:  3.685\n",
      "Epoch   0 Batch   15/269 - Train Accuracy:  0.295, Validation Accuracy:  0.342, Loss:  3.624\n",
      "Epoch   0 Batch   16/269 - Train Accuracy:  0.310, Validation Accuracy:  0.342, Loss:  3.503\n",
      "Epoch   0 Batch   17/269 - Train Accuracy:  0.301, Validation Accuracy:  0.342, Loss:  3.459\n",
      "Epoch   0 Batch   18/269 - Train Accuracy:  0.273, Validation Accuracy:  0.344, Loss:  3.529\n",
      "Epoch   0 Batch   19/269 - Train Accuracy:  0.350, Validation Accuracy:  0.354, Loss:  3.186\n",
      "Epoch   0 Batch   20/269 - Train Accuracy:  0.289, Validation Accuracy:  0.356, Loss:  3.390\n",
      "Epoch   0 Batch   21/269 - Train Accuracy:  0.309, Validation Accuracy:  0.374, Loss:  3.357\n",
      "Epoch   0 Batch   22/269 - Train Accuracy:  0.359, Validation Accuracy:  0.385, Loss:  3.162\n",
      "Epoch   0 Batch   23/269 - Train Accuracy:  0.370, Validation Accuracy:  0.387, Loss:  3.116\n",
      "Epoch   0 Batch   24/269 - Train Accuracy:  0.322, Validation Accuracy:  0.388, Loss:  3.264\n",
      "Epoch   0 Batch   25/269 - Train Accuracy:  0.325, Validation Accuracy:  0.387, Loss:  3.227\n",
      "Epoch   0 Batch   26/269 - Train Accuracy:  0.384, Validation Accuracy:  0.383, Loss:  2.933\n",
      "Epoch   0 Batch   27/269 - Train Accuracy:  0.349, Validation Accuracy:  0.383, Loss:  3.056\n",
      "Epoch   0 Batch   28/269 - Train Accuracy:  0.306, Validation Accuracy:  0.382, Loss:  3.204\n",
      "Epoch   0 Batch   29/269 - Train Accuracy:  0.319, Validation Accuracy:  0.386, Loss:  3.137\n",
      "Epoch   0 Batch   30/269 - Train Accuracy:  0.348, Validation Accuracy:  0.386, Loss:  2.995\n",
      "Epoch   0 Batch   31/269 - Train Accuracy:  0.365, Validation Accuracy:  0.389, Loss:  2.950\n",
      "Epoch   0 Batch   32/269 - Train Accuracy:  0.354, Validation Accuracy:  0.389, Loss:  2.969\n",
      "Epoch   0 Batch   33/269 - Train Accuracy:  0.363, Validation Accuracy:  0.391, Loss:  2.883\n",
      "Epoch   0 Batch   34/269 - Train Accuracy:  0.361, Validation Accuracy:  0.390, Loss:  2.898\n",
      "Epoch   0 Batch   35/269 - Train Accuracy:  0.360, Validation Accuracy:  0.389, Loss:  2.875\n",
      "Epoch   0 Batch   36/269 - Train Accuracy:  0.359, Validation Accuracy:  0.388, Loss:  2.874\n",
      "Epoch   0 Batch   37/269 - Train Accuracy:  0.365, Validation Accuracy:  0.393, Loss:  2.860\n",
      "Epoch   0 Batch   38/269 - Train Accuracy:  0.366, Validation Accuracy:  0.399, Loss:  2.849\n",
      "Epoch   0 Batch   39/269 - Train Accuracy:  0.366, Validation Accuracy:  0.400, Loss:  2.823\n",
      "Epoch   0 Batch   40/269 - Train Accuracy:  0.344, Validation Accuracy:  0.406, Loss:  2.934\n",
      "Epoch   0 Batch   41/269 - Train Accuracy:  0.372, Validation Accuracy:  0.406, Loss:  2.811\n",
      "Epoch   0 Batch   42/269 - Train Accuracy:  0.407, Validation Accuracy:  0.411, Loss:  2.683\n",
      "Epoch   0 Batch   43/269 - Train Accuracy:  0.363, Validation Accuracy:  0.415, Loss:  2.876\n",
      "Epoch   0 Batch   44/269 - Train Accuracy:  0.387, Validation Accuracy:  0.412, Loss:  2.749\n",
      "Epoch   0 Batch   45/269 - Train Accuracy:  0.357, Validation Accuracy:  0.417, Loss:  2.872\n",
      "Epoch   0 Batch   46/269 - Train Accuracy:  0.344, Validation Accuracy:  0.414, Loss:  2.895\n",
      "Epoch   0 Batch   47/269 - Train Accuracy:  0.422, Validation Accuracy:  0.424, Loss:  2.595\n",
      "Epoch   0 Batch   48/269 - Train Accuracy:  0.394, Validation Accuracy:  0.420, Loss:  2.684\n",
      "Epoch   0 Batch   49/269 - Train Accuracy:  0.362, Validation Accuracy:  0.415, Loss:  2.810\n",
      "Epoch   0 Batch   50/269 - Train Accuracy:  0.377, Validation Accuracy:  0.428, Loss:  2.807\n",
      "Epoch   0 Batch   51/269 - Train Accuracy:  0.389, Validation Accuracy:  0.423, Loss:  2.717\n",
      "Epoch   0 Batch   52/269 - Train Accuracy:  0.400, Validation Accuracy:  0.425, Loss:  2.656\n",
      "Epoch   0 Batch   53/269 - Train Accuracy:  0.382, Validation Accuracy:  0.439, Loss:  2.773\n",
      "Epoch   0 Batch   54/269 - Train Accuracy:  0.376, Validation Accuracy:  0.430, Loss:  2.762\n",
      "Epoch   0 Batch   55/269 - Train Accuracy:  0.408, Validation Accuracy:  0.437, Loss:  2.644\n",
      "Epoch   0 Batch   56/269 - Train Accuracy:  0.409, Validation Accuracy:  0.434, Loss:  2.627\n",
      "Epoch   0 Batch   57/269 - Train Accuracy:  0.417, Validation Accuracy:  0.443, Loss:  2.624\n",
      "Epoch   0 Batch   58/269 - Train Accuracy:  0.418, Validation Accuracy:  0.440, Loss:  2.610\n",
      "Epoch   0 Batch   59/269 - Train Accuracy:  0.412, Validation Accuracy:  0.443, Loss:  2.588\n",
      "Epoch   0 Batch   60/269 - Train Accuracy:  0.424, Validation Accuracy:  0.440, Loss:  2.539\n",
      "Epoch   0 Batch   61/269 - Train Accuracy:  0.448, Validation Accuracy:  0.448, Loss:  2.473\n",
      "Epoch   0 Batch   62/269 - Train Accuracy:  0.446, Validation Accuracy:  0.449, Loss:  2.483\n",
      "Epoch   0 Batch   63/269 - Train Accuracy:  0.424, Validation Accuracy:  0.449, Loss:  2.552\n",
      "Epoch   0 Batch   64/269 - Train Accuracy:  0.422, Validation Accuracy:  0.453, Loss:  2.576\n",
      "Epoch   0 Batch   65/269 - Train Accuracy:  0.430, Validation Accuracy:  0.455, Loss:  2.540\n",
      "Epoch   0 Batch   66/269 - Train Accuracy:  0.447, Validation Accuracy:  0.453, Loss:  2.476\n",
      "Epoch   0 Batch   67/269 - Train Accuracy:  0.423, Validation Accuracy:  0.455, Loss:  2.560\n",
      "Epoch   0 Batch   68/269 - Train Accuracy:  0.420, Validation Accuracy:  0.444, Loss:  2.553\n",
      "Epoch   0 Batch   69/269 - Train Accuracy:  0.393, Validation Accuracy:  0.455, Loss:  2.701\n",
      "Epoch   0 Batch   70/269 - Train Accuracy:  0.441, Validation Accuracy:  0.460, Loss:  2.517\n",
      "Epoch   0 Batch   71/269 - Train Accuracy:  0.398, Validation Accuracy:  0.452, Loss:  2.641\n",
      "Epoch   0 Batch   72/269 - Train Accuracy:  0.453, Validation Accuracy:  0.462, Loss:  2.432\n",
      "Epoch   0 Batch   73/269 - Train Accuracy:  0.439, Validation Accuracy:  0.464, Loss:  2.502\n",
      "Epoch   0 Batch   74/269 - Train Accuracy:  0.409, Validation Accuracy:  0.453, Loss:  2.577\n",
      "Epoch   0 Batch   75/269 - Train Accuracy:  0.435, Validation Accuracy:  0.465, Loss:  2.490\n",
      "Epoch   0 Batch   76/269 - Train Accuracy:  0.424, Validation Accuracy:  0.464, Loss:  2.545\n",
      "Epoch   0 Batch   77/269 - Train Accuracy:  0.449, Validation Accuracy:  0.469, Loss:  2.474\n",
      "Epoch   0 Batch   78/269 - Train Accuracy:  0.440, Validation Accuracy:  0.467, Loss:  2.495\n",
      "Epoch   0 Batch   79/269 - Train Accuracy:  0.432, Validation Accuracy:  0.464, Loss:  2.479\n",
      "Epoch   0 Batch   80/269 - Train Accuracy:  0.454, Validation Accuracy:  0.470, Loss:  2.411\n",
      "Epoch   0 Batch   81/269 - Train Accuracy:  0.441, Validation Accuracy:  0.462, Loss:  2.471\n",
      "Epoch   0 Batch   82/269 - Train Accuracy:  0.456, Validation Accuracy:  0.471, Loss:  2.397\n",
      "Epoch   0 Batch   83/269 - Train Accuracy:  0.448, Validation Accuracy:  0.471, Loss:  2.407\n",
      "Epoch   0 Batch   84/269 - Train Accuracy:  0.449, Validation Accuracy:  0.475, Loss:  2.432\n",
      "Epoch   0 Batch   85/269 - Train Accuracy:  0.443, Validation Accuracy:  0.475, Loss:  2.447\n",
      "Epoch   0 Batch   86/269 - Train Accuracy:  0.428, Validation Accuracy:  0.455, Loss:  2.441\n",
      "Epoch   0 Batch   87/269 - Train Accuracy:  0.405, Validation Accuracy:  0.471, Loss:  2.597\n",
      "Epoch   0 Batch   88/269 - Train Accuracy:  0.440, Validation Accuracy:  0.457, Loss:  2.416\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   0 Batch   89/269 - Train Accuracy:  0.460, Validation Accuracy:  0.479, Loss:  2.410\n",
      "Epoch   0 Batch   90/269 - Train Accuracy:  0.414, Validation Accuracy:  0.473, Loss:  2.549\n",
      "Epoch   0 Batch   91/269 - Train Accuracy:  0.453, Validation Accuracy:  0.479, Loss:  2.390\n",
      "Epoch   0 Batch   92/269 - Train Accuracy:  0.451, Validation Accuracy:  0.476, Loss:  2.389\n",
      "Epoch   0 Batch   93/269 - Train Accuracy:  0.455, Validation Accuracy:  0.456, Loss:  2.310\n",
      "Epoch   0 Batch   94/269 - Train Accuracy:  0.454, Validation Accuracy:  0.477, Loss:  2.395\n",
      "Epoch   0 Batch   95/269 - Train Accuracy:  0.457, Validation Accuracy:  0.481, Loss:  2.384\n",
      "Epoch   0 Batch   96/269 - Train Accuracy:  0.450, Validation Accuracy:  0.477, Loss:  2.370\n",
      "Epoch   0 Batch   97/269 - Train Accuracy:  0.454, Validation Accuracy:  0.477, Loss:  2.358\n",
      "Epoch   0 Batch   98/269 - Train Accuracy:  0.464, Validation Accuracy:  0.477, Loss:  2.323\n",
      "Epoch   0 Batch   99/269 - Train Accuracy:  0.423, Validation Accuracy:  0.475, Loss:  2.471\n",
      "Epoch   0 Batch  100/269 - Train Accuracy:  0.469, Validation Accuracy:  0.475, Loss:  2.287\n",
      "Epoch   0 Batch  101/269 - Train Accuracy:  0.423, Validation Accuracy:  0.478, Loss:  2.456\n",
      "Epoch   0 Batch  102/269 - Train Accuracy:  0.463, Validation Accuracy:  0.485, Loss:  2.328\n",
      "Epoch   0 Batch  103/269 - Train Accuracy:  0.463, Validation Accuracy:  0.482, Loss:  2.307\n",
      "Epoch   0 Batch  104/269 - Train Accuracy:  0.450, Validation Accuracy:  0.476, Loss:  2.327\n",
      "Epoch   0 Batch  105/269 - Train Accuracy:  0.454, Validation Accuracy:  0.480, Loss:  2.334\n",
      "Epoch   0 Batch  106/269 - Train Accuracy:  0.451, Validation Accuracy:  0.479, Loss:  2.335\n",
      "Epoch   0 Batch  107/269 - Train Accuracy:  0.426, Validation Accuracy:  0.486, Loss:  2.441\n",
      "Epoch   0 Batch  108/269 - Train Accuracy:  0.460, Validation Accuracy:  0.486, Loss:  2.303\n",
      "Epoch   0 Batch  109/269 - Train Accuracy:  0.447, Validation Accuracy:  0.478, Loss:  2.297\n",
      "Epoch   0 Batch  110/269 - Train Accuracy:  0.460, Validation Accuracy:  0.486, Loss:  2.288\n",
      "Epoch   0 Batch  111/269 - Train Accuracy:  0.438, Validation Accuracy:  0.494, Loss:  2.416\n",
      "Epoch   0 Batch  112/269 - Train Accuracy:  0.473, Validation Accuracy:  0.493, Loss:  2.255\n",
      "Epoch   0 Batch  113/269 - Train Accuracy:  0.491, Validation Accuracy:  0.496, Loss:  2.165\n",
      "Epoch   0 Batch  114/269 - Train Accuracy:  0.473, Validation Accuracy:  0.496, Loss:  2.249\n",
      "Epoch   0 Batch  115/269 - Train Accuracy:  0.449, Validation Accuracy:  0.494, Loss:  2.349\n",
      "Epoch   0 Batch  116/269 - Train Accuracy:  0.477, Validation Accuracy:  0.496, Loss:  2.245\n",
      "Epoch   0 Batch  117/269 - Train Accuracy:  0.468, Validation Accuracy:  0.494, Loss:  2.245\n",
      "Epoch   0 Batch  118/269 - Train Accuracy:  0.498, Validation Accuracy:  0.501, Loss:  2.168\n",
      "Epoch   0 Batch  119/269 - Train Accuracy:  0.462, Validation Accuracy:  0.498, Loss:  2.304\n",
      "Epoch   0 Batch  120/269 - Train Accuracy:  0.460, Validation Accuracy:  0.503, Loss:  2.312\n",
      "Epoch   0 Batch  121/269 - Train Accuracy:  0.479, Validation Accuracy:  0.499, Loss:  2.203\n",
      "Epoch   0 Batch  122/269 - Train Accuracy:  0.485, Validation Accuracy:  0.503, Loss:  2.170\n",
      "Epoch   0 Batch  123/269 - Train Accuracy:  0.457, Validation Accuracy:  0.507, Loss:  2.309\n",
      "Epoch   0 Batch  124/269 - Train Accuracy:  0.469, Validation Accuracy:  0.494, Loss:  2.187\n",
      "Epoch   0 Batch  125/269 - Train Accuracy:  0.479, Validation Accuracy:  0.498, Loss:  2.177\n",
      "Epoch   0 Batch  126/269 - Train Accuracy:  0.482, Validation Accuracy:  0.500, Loss:  2.153\n",
      "Epoch   0 Batch  127/269 - Train Accuracy:  0.456, Validation Accuracy:  0.504, Loss:  2.263\n",
      "Epoch   0 Batch  128/269 - Train Accuracy:  0.494, Validation Accuracy:  0.505, Loss:  2.145\n",
      "Epoch   0 Batch  129/269 - Train Accuracy:  0.471, Validation Accuracy:  0.496, Loss:  2.177\n",
      "Epoch   0 Batch  130/269 - Train Accuracy:  0.446, Validation Accuracy:  0.502, Loss:  2.298\n",
      "Epoch   0 Batch  131/269 - Train Accuracy:  0.460, Validation Accuracy:  0.499, Loss:  2.237\n",
      "Epoch   0 Batch  132/269 - Train Accuracy:  0.472, Validation Accuracy:  0.502, Loss:  2.162\n",
      "Epoch   0 Batch  133/269 - Train Accuracy:  0.479, Validation Accuracy:  0.500, Loss:  2.126\n",
      "Epoch   0 Batch  134/269 - Train Accuracy:  0.438, Validation Accuracy:  0.491, Loss:  2.217\n",
      "Epoch   0 Batch  135/269 - Train Accuracy:  0.456, Validation Accuracy:  0.511, Loss:  2.284\n",
      "Epoch   0 Batch  136/269 - Train Accuracy:  0.462, Validation Accuracy:  0.512, Loss:  2.234\n",
      "Epoch   0 Batch  137/269 - Train Accuracy:  0.462, Validation Accuracy:  0.504, Loss:  2.212\n",
      "Epoch   0 Batch  138/269 - Train Accuracy:  0.480, Validation Accuracy:  0.512, Loss:  2.172\n",
      "Epoch   0 Batch  139/269 - Train Accuracy:  0.488, Validation Accuracy:  0.506, Loss:  2.074\n",
      "Epoch   0 Batch  140/269 - Train Accuracy:  0.487, Validation Accuracy:  0.506, Loss:  2.089\n",
      "Epoch   0 Batch  141/269 - Train Accuracy:  0.472, Validation Accuracy:  0.506, Loss:  2.132\n",
      "Epoch   0 Batch  142/269 - Train Accuracy:  0.479, Validation Accuracy:  0.493, Loss:  2.063\n",
      "Epoch   0 Batch  143/269 - Train Accuracy:  0.484, Validation Accuracy:  0.502, Loss:  2.081\n",
      "Epoch   0 Batch  144/269 - Train Accuracy:  0.491, Validation Accuracy:  0.510, Loss:  2.048\n",
      "Epoch   0 Batch  145/269 - Train Accuracy:  0.474, Validation Accuracy:  0.501, Loss:  2.075\n",
      "Epoch   0 Batch  146/269 - Train Accuracy:  0.481, Validation Accuracy:  0.500, Loss:  2.036\n",
      "Epoch   0 Batch  147/269 - Train Accuracy:  0.514, Validation Accuracy:  0.511, Loss:  1.965\n",
      "Epoch   0 Batch  148/269 - Train Accuracy:  0.473, Validation Accuracy:  0.507, Loss:  2.086\n",
      "Epoch   0 Batch  149/269 - Train Accuracy:  0.489, Validation Accuracy:  0.510, Loss:  2.025\n",
      "Epoch   0 Batch  150/269 - Train Accuracy:  0.500, Validation Accuracy:  0.516, Loss:  2.032\n",
      "Epoch   0 Batch  151/269 - Train Accuracy:  0.525, Validation Accuracy:  0.516, Loss:  1.942\n",
      "Epoch   0 Batch  152/269 - Train Accuracy:  0.480, Validation Accuracy:  0.511, Loss:  2.029\n",
      "Epoch   0 Batch  153/269 - Train Accuracy:  0.502, Validation Accuracy:  0.515, Loss:  2.014\n",
      "Epoch   0 Batch  154/269 - Train Accuracy:  0.462, Validation Accuracy:  0.512, Loss:  2.122\n",
      "Epoch   0 Batch  155/269 - Train Accuracy:  0.519, Validation Accuracy:  0.508, Loss:  1.895\n",
      "Epoch   0 Batch  156/269 - Train Accuracy:  0.483, Validation Accuracy:  0.515, Loss:  2.054\n",
      "Epoch   0 Batch  157/269 - Train Accuracy:  0.495, Validation Accuracy:  0.517, Loss:  2.003\n",
      "Epoch   0 Batch  158/269 - Train Accuracy:  0.485, Validation Accuracy:  0.509, Loss:  1.970\n",
      "Epoch   0 Batch  159/269 - Train Accuracy:  0.508, Validation Accuracy:  0.528, Loss:  1.988\n",
      "Epoch   0 Batch  160/269 - Train Accuracy:  0.503, Validation Accuracy:  0.523, Loss:  1.980\n",
      "Epoch   0 Batch  161/269 - Train Accuracy:  0.483, Validation Accuracy:  0.515, Loss:  1.986\n",
      "Epoch   0 Batch  162/269 - Train Accuracy:  0.505, Validation Accuracy:  0.525, Loss:  1.964\n",
      "Epoch   0 Batch  163/269 - Train Accuracy:  0.501, Validation Accuracy:  0.521, Loss:  1.961\n",
      "Epoch   0 Batch  164/269 - Train Accuracy:  0.501, Validation Accuracy:  0.521, Loss:  1.944\n",
      "Epoch   0 Batch  165/269 - Train Accuracy:  0.476, Validation Accuracy:  0.525, Loss:  2.020\n",
      "Epoch   0 Batch  166/269 - Train Accuracy:  0.530, Validation Accuracy:  0.524, Loss:  1.828\n",
      "Epoch   0 Batch  167/269 - Train Accuracy:  0.506, Validation Accuracy:  0.523, Loss:  1.925\n",
      "Epoch   0 Batch  168/269 - Train Accuracy:  0.495, Validation Accuracy:  0.520, Loss:  1.937\n",
      "Epoch   0 Batch  169/269 - Train Accuracy:  0.482, Validation Accuracy:  0.511, Loss:  1.932\n",
      "Epoch   0 Batch  170/269 - Train Accuracy:  0.501, Validation Accuracy:  0.524, Loss:  1.912\n",
      "Epoch   0 Batch  171/269 - Train Accuracy:  0.490, Validation Accuracy:  0.528, Loss:  1.981\n",
      "Epoch   0 Batch  172/269 - Train Accuracy:  0.495, Validation Accuracy:  0.519, Loss:  1.917\n",
      "Epoch   0 Batch  173/269 - Train Accuracy:  0.506, Validation Accuracy:  0.524, Loss:  1.900\n",
      "Epoch   0 Batch  174/269 - Train Accuracy:  0.500, Validation Accuracy:  0.528, Loss:  1.900\n",
      "Epoch   0 Batch  175/269 - Train Accuracy:  0.494, Validation Accuracy:  0.518, Loss:  1.903\n",
      "Epoch   0 Batch  176/269 - Train Accuracy:  0.482, Validation Accuracy:  0.527, Loss:  1.975\n",
      "Epoch   0 Batch  177/269 - Train Accuracy:  0.524, Validation Accuracy:  0.529, Loss:  1.820\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   0 Batch  178/269 - Train Accuracy:  0.479, Validation Accuracy:  0.517, Loss:  1.935\n",
      "Epoch   0 Batch  179/269 - Train Accuracy:  0.514, Validation Accuracy:  0.534, Loss:  1.876\n",
      "Epoch   0 Batch  180/269 - Train Accuracy:  0.507, Validation Accuracy:  0.527, Loss:  1.853\n",
      "Epoch   0 Batch  181/269 - Train Accuracy:  0.493, Validation Accuracy:  0.514, Loss:  1.849\n",
      "Epoch   0 Batch  182/269 - Train Accuracy:  0.511, Validation Accuracy:  0.530, Loss:  1.865\n",
      "Epoch   0 Batch  183/269 - Train Accuracy:  0.569, Validation Accuracy:  0.527, Loss:  1.626\n",
      "Epoch   0 Batch  184/269 - Train Accuracy:  0.465, Validation Accuracy:  0.509, Loss:  1.924\n",
      "Epoch   0 Batch  185/269 - Train Accuracy:  0.524, Validation Accuracy:  0.534, Loss:  1.815\n",
      "Epoch   0 Batch  186/269 - Train Accuracy:  0.483, Validation Accuracy:  0.533, Loss:  1.915\n",
      "Epoch   0 Batch  187/269 - Train Accuracy:  0.495, Validation Accuracy:  0.517, Loss:  1.790\n",
      "Epoch   0 Batch  188/269 - Train Accuracy:  0.527, Validation Accuracy:  0.532, Loss:  1.773\n",
      "Epoch   0 Batch  189/269 - Train Accuracy:  0.517, Validation Accuracy:  0.533, Loss:  1.789\n",
      "Epoch   0 Batch  190/269 - Train Accuracy:  0.500, Validation Accuracy:  0.523, Loss:  1.796\n",
      "Epoch   0 Batch  191/269 - Train Accuracy:  0.514, Validation Accuracy:  0.528, Loss:  1.801\n",
      "Epoch   0 Batch  192/269 - Train Accuracy:  0.516, Validation Accuracy:  0.532, Loss:  1.796\n",
      "Epoch   0 Batch  193/269 - Train Accuracy:  0.493, Validation Accuracy:  0.519, Loss:  1.799\n",
      "Epoch   0 Batch  194/269 - Train Accuracy:  0.509, Validation Accuracy:  0.527, Loss:  1.797\n",
      "Epoch   0 Batch  195/269 - Train Accuracy:  0.512, Validation Accuracy:  0.538, Loss:  1.812\n",
      "Epoch   0 Batch  196/269 - Train Accuracy:  0.496, Validation Accuracy:  0.528, Loss:  1.782\n",
      "Epoch   0 Batch  197/269 - Train Accuracy:  0.469, Validation Accuracy:  0.523, Loss:  1.851\n",
      "Epoch   0 Batch  198/269 - Train Accuracy:  0.497, Validation Accuracy:  0.543, Loss:  1.880\n",
      "Epoch   0 Batch  199/269 - Train Accuracy:  0.506, Validation Accuracy:  0.533, Loss:  1.801\n",
      "Epoch   0 Batch  200/269 - Train Accuracy:  0.487, Validation Accuracy:  0.526, Loss:  1.819\n",
      "Epoch   0 Batch  201/269 - Train Accuracy:  0.514, Validation Accuracy:  0.538, Loss:  1.757\n",
      "Epoch   0 Batch  202/269 - Train Accuracy:  0.511, Validation Accuracy:  0.540, Loss:  1.768\n",
      "Epoch   0 Batch  203/269 - Train Accuracy:  0.488, Validation Accuracy:  0.534, Loss:  1.817\n",
      "Epoch   0 Batch  204/269 - Train Accuracy:  0.497, Validation Accuracy:  0.539, Loss:  1.815\n",
      "Epoch   0 Batch  205/269 - Train Accuracy:  0.506, Validation Accuracy:  0.537, Loss:  1.733\n",
      "Epoch   0 Batch  206/269 - Train Accuracy:  0.479, Validation Accuracy:  0.525, Loss:  1.822\n",
      "Epoch   0 Batch  207/269 - Train Accuracy:  0.537, Validation Accuracy:  0.538, Loss:  1.671\n",
      "Epoch   0 Batch  208/269 - Train Accuracy:  0.491, Validation Accuracy:  0.541, Loss:  1.813\n",
      "Epoch   0 Batch  209/269 - Train Accuracy:  0.483, Validation Accuracy:  0.525, Loss:  1.775\n",
      "Epoch   0 Batch  210/269 - Train Accuracy:  0.521, Validation Accuracy:  0.538, Loss:  1.700\n",
      "Epoch   0 Batch  211/269 - Train Accuracy:  0.521, Validation Accuracy:  0.543, Loss:  1.706\n",
      "Epoch   0 Batch  212/269 - Train Accuracy:  0.525, Validation Accuracy:  0.534, Loss:  1.666\n",
      "Epoch   0 Batch  213/269 - Train Accuracy:  0.521, Validation Accuracy:  0.539, Loss:  1.686\n",
      "Epoch   0 Batch  214/269 - Train Accuracy:  0.527, Validation Accuracy:  0.541, Loss:  1.676\n",
      "Epoch   0 Batch  215/269 - Train Accuracy:  0.540, Validation Accuracy:  0.532, Loss:  1.581\n",
      "Epoch   0 Batch  216/269 - Train Accuracy:  0.493, Validation Accuracy:  0.545, Loss:  1.792\n",
      "Epoch   0 Batch  217/269 - Train Accuracy:  0.498, Validation Accuracy:  0.544, Loss:  1.759\n",
      "Epoch   0 Batch  218/269 - Train Accuracy:  0.504, Validation Accuracy:  0.540, Loss:  1.744\n",
      "Epoch   0 Batch  219/269 - Train Accuracy:  0.513, Validation Accuracy:  0.542, Loss:  1.718\n",
      "Epoch   0 Batch  220/269 - Train Accuracy:  0.538, Validation Accuracy:  0.544, Loss:  1.604\n",
      "Epoch   0 Batch  221/269 - Train Accuracy:  0.532, Validation Accuracy:  0.539, Loss:  1.641\n",
      "Epoch   0 Batch  222/269 - Train Accuracy:  0.536, Validation Accuracy:  0.544, Loss:  1.606\n",
      "Epoch   0 Batch  223/269 - Train Accuracy:  0.529, Validation Accuracy:  0.542, Loss:  1.614\n",
      "Epoch   0 Batch  224/269 - Train Accuracy:  0.519, Validation Accuracy:  0.537, Loss:  1.647\n",
      "Epoch   0 Batch  225/269 - Train Accuracy:  0.512, Validation Accuracy:  0.544, Loss:  1.704\n",
      "Epoch   0 Batch  226/269 - Train Accuracy:  0.517, Validation Accuracy:  0.539, Loss:  1.628\n",
      "Epoch   0 Batch  227/269 - Train Accuracy:  0.579, Validation Accuracy:  0.530, Loss:  1.418\n",
      "Epoch   0 Batch  228/269 - Train Accuracy:  0.509, Validation Accuracy:  0.540, Loss:  1.645\n",
      "Epoch   0 Batch  229/269 - Train Accuracy:  0.526, Validation Accuracy:  0.540, Loss:  1.606\n",
      "Epoch   0 Batch  230/269 - Train Accuracy:  0.503, Validation Accuracy:  0.532, Loss:  1.634\n",
      "Epoch   0 Batch  231/269 - Train Accuracy:  0.495, Validation Accuracy:  0.540, Loss:  1.694\n",
      "Epoch   0 Batch  232/269 - Train Accuracy:  0.484, Validation Accuracy:  0.538, Loss:  1.703\n",
      "Epoch   0 Batch  233/269 - Train Accuracy:  0.513, Validation Accuracy:  0.534, Loss:  1.619\n",
      "Epoch   0 Batch  234/269 - Train Accuracy:  0.518, Validation Accuracy:  0.537, Loss:  1.609\n",
      "Epoch   0 Batch  235/269 - Train Accuracy:  0.525, Validation Accuracy:  0.542, Loss:  1.595\n",
      "Epoch   0 Batch  236/269 - Train Accuracy:  0.509, Validation Accuracy:  0.533, Loss:  1.593\n",
      "Epoch   0 Batch  237/269 - Train Accuracy:  0.516, Validation Accuracy:  0.539, Loss:  1.581\n",
      "Epoch   0 Batch  238/269 - Train Accuracy:  0.525, Validation Accuracy:  0.541, Loss:  1.577\n",
      "Epoch   0 Batch  239/269 - Train Accuracy:  0.517, Validation Accuracy:  0.534, Loss:  1.567\n",
      "Epoch   0 Batch  240/269 - Train Accuracy:  0.546, Validation Accuracy:  0.535, Loss:  1.461\n",
      "Epoch   0 Batch  241/269 - Train Accuracy:  0.529, Validation Accuracy:  0.546, Loss:  1.553\n",
      "Epoch   0 Batch  242/269 - Train Accuracy:  0.499, Validation Accuracy:  0.534, Loss:  1.570\n",
      "Epoch   0 Batch  243/269 - Train Accuracy:  0.526, Validation Accuracy:  0.533, Loss:  1.522\n",
      "Epoch   0 Batch  244/269 - Train Accuracy:  0.537, Validation Accuracy:  0.560, Loss:  1.558\n",
      "Epoch   0 Batch  245/269 - Train Accuracy:  0.481, Validation Accuracy:  0.526, Loss:  1.637\n",
      "Epoch   0 Batch  246/269 - Train Accuracy:  0.510, Validation Accuracy:  0.539, Loss:  1.557\n",
      "Epoch   0 Batch  247/269 - Train Accuracy:  0.520, Validation Accuracy:  0.553, Loss:  1.601\n",
      "Epoch   0 Batch  248/269 - Train Accuracy:  0.500, Validation Accuracy:  0.529, Loss:  1.544\n",
      "Epoch   0 Batch  249/269 - Train Accuracy:  0.543, Validation Accuracy:  0.539, Loss:  1.472\n",
      "Epoch   0 Batch  250/269 - Train Accuracy:  0.511, Validation Accuracy:  0.543, Loss:  1.574\n",
      "Epoch   0 Batch  251/269 - Train Accuracy:  0.511, Validation Accuracy:  0.531, Loss:  1.515\n",
      "Epoch   0 Batch  252/269 - Train Accuracy:  0.492, Validation Accuracy:  0.532, Loss:  1.549\n",
      "Epoch   0 Batch  253/269 - Train Accuracy:  0.502, Validation Accuracy:  0.534, Loss:  1.528\n",
      "Epoch   0 Batch  254/269 - Train Accuracy:  0.510, Validation Accuracy:  0.532, Loss:  1.497\n",
      "Epoch   0 Batch  255/269 - Train Accuracy:  0.539, Validation Accuracy:  0.538, Loss:  1.446\n",
      "Epoch   0 Batch  256/269 - Train Accuracy:  0.487, Validation Accuracy:  0.530, Loss:  1.537\n",
      "Epoch   0 Batch  257/269 - Train Accuracy:  0.497, Validation Accuracy:  0.527, Loss:  1.513\n",
      "Epoch   0 Batch  258/269 - Train Accuracy:  0.510, Validation Accuracy:  0.543, Loss:  1.509\n",
      "Epoch   0 Batch  259/269 - Train Accuracy:  0.504, Validation Accuracy:  0.521, Loss:  1.486\n",
      "Epoch   0 Batch  260/269 - Train Accuracy:  0.496, Validation Accuracy:  0.534, Loss:  1.556\n",
      "Epoch   0 Batch  261/269 - Train Accuracy:  0.471, Validation Accuracy:  0.537, Loss:  1.580\n",
      "Epoch   0 Batch  262/269 - Train Accuracy:  0.515, Validation Accuracy:  0.535, Loss:  1.484\n",
      "Epoch   0 Batch  263/269 - Train Accuracy:  0.483, Validation Accuracy:  0.521, Loss:  1.528\n",
      "Epoch   0 Batch  264/269 - Train Accuracy:  0.485, Validation Accuracy:  0.530, Loss:  1.540\n",
      "Epoch   0 Batch  265/269 - Train Accuracy:  0.477, Validation Accuracy:  0.529, Loss:  1.515\n",
      "Epoch   0 Batch  266/269 - Train Accuracy:  0.501, Validation Accuracy:  0.527, Loss:  1.446\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   0 Batch  267/269 - Train Accuracy:  0.491, Validation Accuracy:  0.525, Loss:  1.477\n",
      "Epoch   1 Batch    0/269 - Train Accuracy:  0.476, Validation Accuracy:  0.524, Loss:  1.527\n",
      "Epoch   1 Batch    1/269 - Train Accuracy:  0.464, Validation Accuracy:  0.518, Loss:  1.507\n",
      "Epoch   1 Batch    2/269 - Train Accuracy:  0.481, Validation Accuracy:  0.526, Loss:  1.472\n",
      "Epoch   1 Batch    3/269 - Train Accuracy:  0.468, Validation Accuracy:  0.516, Loss:  1.494\n",
      "Epoch   1 Batch    4/269 - Train Accuracy:  0.465, Validation Accuracy:  0.525, Loss:  1.498\n",
      "Epoch   1 Batch    5/269 - Train Accuracy:  0.460, Validation Accuracy:  0.514, Loss:  1.507\n",
      "Epoch   1 Batch    6/269 - Train Accuracy:  0.478, Validation Accuracy:  0.503, Loss:  1.381\n",
      "Epoch   1 Batch    7/269 - Train Accuracy:  0.494, Validation Accuracy:  0.513, Loss:  1.412\n",
      "Epoch   1 Batch    8/269 - Train Accuracy:  0.465, Validation Accuracy:  0.516, Loss:  1.478\n",
      "Epoch   1 Batch    9/269 - Train Accuracy:  0.472, Validation Accuracy:  0.512, Loss:  1.428\n",
      "Epoch   1 Batch   10/269 - Train Accuracy:  0.446, Validation Accuracy:  0.505, Loss:  1.452\n",
      "Epoch   1 Batch   11/269 - Train Accuracy:  0.484, Validation Accuracy:  0.509, Loss:  1.413\n",
      "Epoch   1 Batch   12/269 - Train Accuracy:  0.445, Validation Accuracy:  0.501, Loss:  1.460\n",
      "Epoch   1 Batch   13/269 - Train Accuracy:  0.514, Validation Accuracy:  0.510, Loss:  1.306\n",
      "Epoch   1 Batch   14/269 - Train Accuracy:  0.453, Validation Accuracy:  0.493, Loss:  1.390\n",
      "Epoch   1 Batch   15/269 - Train Accuracy:  0.460, Validation Accuracy:  0.501, Loss:  1.393\n",
      "Epoch   1 Batch   16/269 - Train Accuracy:  0.487, Validation Accuracy:  0.505, Loss:  1.375\n",
      "Epoch   1 Batch   17/269 - Train Accuracy:  0.469, Validation Accuracy:  0.498, Loss:  1.361\n",
      "Epoch   1 Batch   18/269 - Train Accuracy:  0.444, Validation Accuracy:  0.492, Loss:  1.417\n",
      "Epoch   1 Batch   19/269 - Train Accuracy:  0.500, Validation Accuracy:  0.502, Loss:  1.295\n",
      "Epoch   1 Batch   20/269 - Train Accuracy:  0.465, Validation Accuracy:  0.507, Loss:  1.414\n",
      "Epoch   1 Batch   21/269 - Train Accuracy:  0.448, Validation Accuracy:  0.497, Loss:  1.437\n",
      "Epoch   1 Batch   22/269 - Train Accuracy:  0.476, Validation Accuracy:  0.497, Loss:  1.336\n",
      "Epoch   1 Batch   23/269 - Train Accuracy:  0.498, Validation Accuracy:  0.502, Loss:  1.340\n",
      "Epoch   1 Batch   24/269 - Train Accuracy:  0.444, Validation Accuracy:  0.503, Loss:  1.401\n",
      "Epoch   1 Batch   25/269 - Train Accuracy:  0.440, Validation Accuracy:  0.494, Loss:  1.404\n",
      "Epoch   1 Batch   26/269 - Train Accuracy:  0.513, Validation Accuracy:  0.509, Loss:  1.257\n",
      "Epoch   1 Batch   27/269 - Train Accuracy:  0.475, Validation Accuracy:  0.504, Loss:  1.331\n",
      "Epoch   1 Batch   28/269 - Train Accuracy:  0.417, Validation Accuracy:  0.489, Loss:  1.418\n",
      "Epoch   1 Batch   29/269 - Train Accuracy:  0.451, Validation Accuracy:  0.507, Loss:  1.367\n",
      "Epoch   1 Batch   30/269 - Train Accuracy:  0.477, Validation Accuracy:  0.502, Loss:  1.314\n",
      "Epoch   1 Batch   31/269 - Train Accuracy:  0.469, Validation Accuracy:  0.490, Loss:  1.293\n",
      "Epoch   1 Batch   32/269 - Train Accuracy:  0.482, Validation Accuracy:  0.516, Loss:  1.305\n",
      "Epoch   1 Batch   33/269 - Train Accuracy:  0.463, Validation Accuracy:  0.489, Loss:  1.271\n",
      "Epoch   1 Batch   34/269 - Train Accuracy:  0.465, Validation Accuracy:  0.489, Loss:  1.285\n",
      "Epoch   1 Batch   35/269 - Train Accuracy:  0.515, Validation Accuracy:  0.522, Loss:  1.286\n",
      "Epoch   1 Batch   36/269 - Train Accuracy:  0.483, Validation Accuracy:  0.502, Loss:  1.293\n",
      "Epoch   1 Batch   37/269 - Train Accuracy:  0.470, Validation Accuracy:  0.492, Loss:  1.282\n",
      "Epoch   1 Batch   38/269 - Train Accuracy:  0.492, Validation Accuracy:  0.513, Loss:  1.287\n",
      "Epoch   1 Batch   39/269 - Train Accuracy:  0.481, Validation Accuracy:  0.502, Loss:  1.272\n",
      "Epoch   1 Batch   40/269 - Train Accuracy:  0.419, Validation Accuracy:  0.481, Loss:  1.315\n",
      "Epoch   1 Batch   41/269 - Train Accuracy:  0.488, Validation Accuracy:  0.511, Loss:  1.275\n",
      "Epoch   1 Batch   42/269 - Train Accuracy:  0.506, Validation Accuracy:  0.513, Loss:  1.200\n",
      "Epoch   1 Batch   43/269 - Train Accuracy:  0.432, Validation Accuracy:  0.478, Loss:  1.293\n",
      "Epoch   1 Batch   44/269 - Train Accuracy:  0.477, Validation Accuracy:  0.501, Loss:  1.247\n",
      "Epoch   1 Batch   45/269 - Train Accuracy:  0.479, Validation Accuracy:  0.521, Loss:  1.298\n",
      "Epoch   1 Batch   46/269 - Train Accuracy:  0.430, Validation Accuracy:  0.486, Loss:  1.299\n",
      "Epoch   1 Batch   47/269 - Train Accuracy:  0.472, Validation Accuracy:  0.478, Loss:  1.163\n",
      "Epoch   1 Batch   48/269 - Train Accuracy:  0.493, Validation Accuracy:  0.512, Loss:  1.204\n",
      "Epoch   1 Batch   49/269 - Train Accuracy:  0.472, Validation Accuracy:  0.515, Loss:  1.266\n",
      "Epoch   1 Batch   50/269 - Train Accuracy:  0.451, Validation Accuracy:  0.496, Loss:  1.267\n",
      "Epoch   1 Batch   51/269 - Train Accuracy:  0.467, Validation Accuracy:  0.502, Loss:  1.237\n",
      "Epoch   1 Batch   52/269 - Train Accuracy:  0.496, Validation Accuracy:  0.521, Loss:  1.197\n",
      "Epoch   1 Batch   53/269 - Train Accuracy:  0.484, Validation Accuracy:  0.524, Loss:  1.265\n",
      "Epoch   1 Batch   54/269 - Train Accuracy:  0.463, Validation Accuracy:  0.504, Loss:  1.251\n",
      "Epoch   1 Batch   55/269 - Train Accuracy:  0.481, Validation Accuracy:  0.506, Loss:  1.188\n",
      "Epoch   1 Batch   56/269 - Train Accuracy:  0.495, Validation Accuracy:  0.514, Loss:  1.191\n",
      "Epoch   1 Batch   57/269 - Train Accuracy:  0.482, Validation Accuracy:  0.509, Loss:  1.193\n",
      "Epoch   1 Batch   58/269 - Train Accuracy:  0.499, Validation Accuracy:  0.512, Loss:  1.179\n",
      "Epoch   1 Batch   59/269 - Train Accuracy:  0.497, Validation Accuracy:  0.515, Loss:  1.153\n",
      "Epoch   1 Batch   60/269 - Train Accuracy:  0.489, Validation Accuracy:  0.511, Loss:  1.136\n",
      "Epoch   1 Batch   61/269 - Train Accuracy:  0.497, Validation Accuracy:  0.500, Loss:  1.106\n",
      "Epoch   1 Batch   62/269 - Train Accuracy:  0.510, Validation Accuracy:  0.514, Loss:  1.129\n",
      "Epoch   1 Batch   63/269 - Train Accuracy:  0.495, Validation Accuracy:  0.518, Loss:  1.164\n",
      "Epoch   1 Batch   64/269 - Train Accuracy:  0.477, Validation Accuracy:  0.513, Loss:  1.152\n",
      "Epoch   1 Batch   65/269 - Train Accuracy:  0.491, Validation Accuracy:  0.515, Loss:  1.137\n",
      "Epoch   1 Batch   66/269 - Train Accuracy:  0.518, Validation Accuracy:  0.526, Loss:  1.112\n",
      "Epoch   1 Batch   67/269 - Train Accuracy:  0.502, Validation Accuracy:  0.530, Loss:  1.155\n",
      "Epoch   1 Batch   68/269 - Train Accuracy:  0.514, Validation Accuracy:  0.538, Loss:  1.152\n",
      "Epoch   1 Batch   69/269 - Train Accuracy:  0.489, Validation Accuracy:  0.535, Loss:  1.226\n",
      "Epoch   1 Batch   70/269 - Train Accuracy:  0.512, Validation Accuracy:  0.521, Loss:  1.127\n",
      "Epoch   1 Batch   71/269 - Train Accuracy:  0.479, Validation Accuracy:  0.514, Loss:  1.181\n",
      "Epoch   1 Batch   72/269 - Train Accuracy:  0.540, Validation Accuracy:  0.536, Loss:  1.098\n",
      "Epoch   1 Batch   73/269 - Train Accuracy:  0.512, Validation Accuracy:  0.540, Loss:  1.141\n",
      "Epoch   1 Batch   74/269 - Train Accuracy:  0.470, Validation Accuracy:  0.513, Loss:  1.144\n",
      "Epoch   1 Batch   75/269 - Train Accuracy:  0.501, Validation Accuracy:  0.523, Loss:  1.110\n",
      "Epoch   1 Batch   76/269 - Train Accuracy:  0.519, Validation Accuracy:  0.551, Loss:  1.136\n",
      "Epoch   1 Batch   77/269 - Train Accuracy:  0.531, Validation Accuracy:  0.538, Loss:  1.105\n",
      "Epoch   1 Batch   78/269 - Train Accuracy:  0.509, Validation Accuracy:  0.531, Loss:  1.106\n",
      "Epoch   1 Batch   79/269 - Train Accuracy:  0.527, Validation Accuracy:  0.547, Loss:  1.095\n",
      "Epoch   1 Batch   80/269 - Train Accuracy:  0.533, Validation Accuracy:  0.537, Loss:  1.073\n",
      "Epoch   1 Batch   81/269 - Train Accuracy:  0.513, Validation Accuracy:  0.528, Loss:  1.109\n",
      "Epoch   1 Batch   82/269 - Train Accuracy:  0.526, Validation Accuracy:  0.546, Loss:  1.059\n",
      "Epoch   1 Batch   83/269 - Train Accuracy:  0.543, Validation Accuracy:  0.549, Loss:  1.069\n",
      "Epoch   1 Batch   84/269 - Train Accuracy:  0.524, Validation Accuracy:  0.544, Loss:  1.059\n",
      "Epoch   1 Batch   85/269 - Train Accuracy:  0.507, Validation Accuracy:  0.533, Loss:  1.079\n",
      "Epoch   1 Batch   86/269 - Train Accuracy:  0.514, Validation Accuracy:  0.552, Loss:  1.084\n",
      "Epoch   1 Batch   87/269 - Train Accuracy:  0.503, Validation Accuracy:  0.551, Loss:  1.143\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   1 Batch   88/269 - Train Accuracy:  0.528, Validation Accuracy:  0.545, Loss:  1.067\n",
      "Epoch   1 Batch   89/269 - Train Accuracy:  0.526, Validation Accuracy:  0.540, Loss:  1.057\n",
      "Epoch   1 Batch   90/269 - Train Accuracy:  0.501, Validation Accuracy:  0.556, Loss:  1.123\n",
      "Epoch   1 Batch   91/269 - Train Accuracy:  0.527, Validation Accuracy:  0.550, Loss:  1.044\n",
      "Epoch   1 Batch   92/269 - Train Accuracy:  0.517, Validation Accuracy:  0.528, Loss:  1.047\n",
      "Epoch   1 Batch   93/269 - Train Accuracy:  0.554, Validation Accuracy:  0.555, Loss:  1.007\n",
      "Epoch   1 Batch   94/269 - Train Accuracy:  0.545, Validation Accuracy:  0.552, Loss:  1.061\n",
      "Epoch   1 Batch   95/269 - Train Accuracy:  0.530, Validation Accuracy:  0.543, Loss:  1.051\n",
      "Epoch   1 Batch   96/269 - Train Accuracy:  0.504, Validation Accuracy:  0.514, Loss:  1.031\n",
      "Epoch   1 Batch   97/269 - Train Accuracy:  0.526, Validation Accuracy:  0.546, Loss:  1.046\n",
      "Epoch   1 Batch   98/269 - Train Accuracy:  0.551, Validation Accuracy:  0.543, Loss:  1.020\n",
      "Epoch   1 Batch   99/269 - Train Accuracy:  0.495, Validation Accuracy:  0.544, Loss:  1.081\n",
      "Epoch   1 Batch  100/269 - Train Accuracy:  0.547, Validation Accuracy:  0.544, Loss:  1.001\n",
      "Epoch   1 Batch  101/269 - Train Accuracy:  0.493, Validation Accuracy:  0.543, Loss:  1.077\n",
      "Epoch   1 Batch  102/269 - Train Accuracy:  0.529, Validation Accuracy:  0.546, Loss:  1.017\n",
      "Epoch   1 Batch  103/269 - Train Accuracy:  0.523, Validation Accuracy:  0.541, Loss:  1.009\n",
      "Epoch   1 Batch  104/269 - Train Accuracy:  0.522, Validation Accuracy:  0.538, Loss:  1.017\n",
      "Epoch   1 Batch  105/269 - Train Accuracy:  0.528, Validation Accuracy:  0.538, Loss:  1.028\n",
      "Epoch   1 Batch  106/269 - Train Accuracy:  0.531, Validation Accuracy:  0.541, Loss:  1.012\n",
      "Epoch   1 Batch  107/269 - Train Accuracy:  0.498, Validation Accuracy:  0.540, Loss:  1.059\n",
      "Epoch   1 Batch  108/269 - Train Accuracy:  0.529, Validation Accuracy:  0.545, Loss:  1.001\n",
      "Epoch   1 Batch  109/269 - Train Accuracy:  0.513, Validation Accuracy:  0.539, Loss:  1.019\n",
      "Epoch   1 Batch  110/269 - Train Accuracy:  0.521, Validation Accuracy:  0.538, Loss:  0.987\n",
      "Epoch   1 Batch  111/269 - Train Accuracy:  0.514, Validation Accuracy:  0.542, Loss:  1.062\n",
      "Epoch   1 Batch  112/269 - Train Accuracy:  0.532, Validation Accuracy:  0.543, Loss:  0.987\n",
      "Epoch   1 Batch  113/269 - Train Accuracy:  0.560, Validation Accuracy:  0.550, Loss:  0.949\n",
      "Epoch   1 Batch  114/269 - Train Accuracy:  0.535, Validation Accuracy:  0.550, Loss:  0.984\n",
      "Epoch   1 Batch  115/269 - Train Accuracy:  0.521, Validation Accuracy:  0.547, Loss:  1.011\n",
      "Epoch   1 Batch  116/269 - Train Accuracy:  0.549, Validation Accuracy:  0.544, Loss:  0.989\n",
      "Epoch   1 Batch  117/269 - Train Accuracy:  0.547, Validation Accuracy:  0.547, Loss:  0.975\n",
      "Epoch   1 Batch  118/269 - Train Accuracy:  0.560, Validation Accuracy:  0.543, Loss:  0.947\n",
      "Epoch   1 Batch  119/269 - Train Accuracy:  0.517, Validation Accuracy:  0.546, Loss:  1.023\n",
      "Epoch   1 Batch  120/269 - Train Accuracy:  0.533, Validation Accuracy:  0.562, Loss:  1.007\n",
      "Epoch   1 Batch  121/269 - Train Accuracy:  0.551, Validation Accuracy:  0.566, Loss:  0.963\n",
      "Epoch   1 Batch  122/269 - Train Accuracy:  0.545, Validation Accuracy:  0.557, Loss:  0.957\n",
      "Epoch   1 Batch  123/269 - Train Accuracy:  0.530, Validation Accuracy:  0.548, Loss:  1.006\n",
      "Epoch   1 Batch  124/269 - Train Accuracy:  0.554, Validation Accuracy:  0.566, Loss:  0.943\n",
      "Epoch   1 Batch  125/269 - Train Accuracy:  0.551, Validation Accuracy:  0.565, Loss:  0.937\n",
      "Epoch   1 Batch  126/269 - Train Accuracy:  0.569, Validation Accuracy:  0.563, Loss:  0.939\n",
      "Epoch   1 Batch  127/269 - Train Accuracy:  0.547, Validation Accuracy:  0.569, Loss:  0.996\n",
      "Epoch   1 Batch  128/269 - Train Accuracy:  0.569, Validation Accuracy:  0.575, Loss:  0.944\n",
      "Epoch   1 Batch  129/269 - Train Accuracy:  0.550, Validation Accuracy:  0.576, Loss:  0.959\n",
      "Epoch   1 Batch  130/269 - Train Accuracy:  0.530, Validation Accuracy:  0.573, Loss:  0.999\n",
      "Epoch   1 Batch  131/269 - Train Accuracy:  0.545, Validation Accuracy:  0.570, Loss:  0.970\n",
      "Epoch   1 Batch  132/269 - Train Accuracy:  0.562, Validation Accuracy:  0.571, Loss:  0.952\n",
      "Epoch   1 Batch  133/269 - Train Accuracy:  0.563, Validation Accuracy:  0.569, Loss:  0.917\n",
      "Epoch   1 Batch  134/269 - Train Accuracy:  0.528, Validation Accuracy:  0.570, Loss:  0.967\n",
      "Epoch   1 Batch  135/269 - Train Accuracy:  0.526, Validation Accuracy:  0.571, Loss:  1.007\n",
      "Epoch   1 Batch  136/269 - Train Accuracy:  0.522, Validation Accuracy:  0.568, Loss:  0.989\n",
      "Epoch   1 Batch  137/269 - Train Accuracy:  0.549, Validation Accuracy:  0.564, Loss:  0.974\n",
      "Epoch   1 Batch  138/269 - Train Accuracy:  0.546, Validation Accuracy:  0.564, Loss:  0.950\n",
      "Epoch   1 Batch  139/269 - Train Accuracy:  0.574, Validation Accuracy:  0.565, Loss:  0.900\n",
      "Epoch   1 Batch  140/269 - Train Accuracy:  0.564, Validation Accuracy:  0.570, Loss:  0.929\n",
      "Epoch   1 Batch  141/269 - Train Accuracy:  0.553, Validation Accuracy:  0.568, Loss:  0.939\n",
      "Epoch   1 Batch  142/269 - Train Accuracy:  0.556, Validation Accuracy:  0.566, Loss:  0.903\n",
      "Epoch   1 Batch  143/269 - Train Accuracy:  0.557, Validation Accuracy:  0.569, Loss:  0.919\n",
      "Epoch   1 Batch  144/269 - Train Accuracy:  0.569, Validation Accuracy:  0.570, Loss:  0.891\n",
      "Epoch   1 Batch  145/269 - Train Accuracy:  0.558, Validation Accuracy:  0.583, Loss:  0.909\n",
      "Epoch   1 Batch  146/269 - Train Accuracy:  0.559, Validation Accuracy:  0.570, Loss:  0.900\n",
      "Epoch   1 Batch  147/269 - Train Accuracy:  0.582, Validation Accuracy:  0.571, Loss:  0.868\n",
      "Epoch   1 Batch  148/269 - Train Accuracy:  0.552, Validation Accuracy:  0.568, Loss:  0.926\n",
      "Epoch   1 Batch  149/269 - Train Accuracy:  0.567, Validation Accuracy:  0.571, Loss:  0.902\n",
      "Epoch   1 Batch  150/269 - Train Accuracy:  0.569, Validation Accuracy:  0.580, Loss:  0.911\n",
      "Epoch   1 Batch  151/269 - Train Accuracy:  0.603, Validation Accuracy:  0.576, Loss:  0.864\n",
      "Epoch   1 Batch  152/269 - Train Accuracy:  0.566, Validation Accuracy:  0.577, Loss:  0.899\n",
      "Epoch   1 Batch  153/269 - Train Accuracy:  0.578, Validation Accuracy:  0.579, Loss:  0.885\n",
      "Epoch   1 Batch  154/269 - Train Accuracy:  0.541, Validation Accuracy:  0.579, Loss:  0.919\n",
      "Epoch   1 Batch  155/269 - Train Accuracy:  0.598, Validation Accuracy:  0.578, Loss:  0.842\n",
      "Epoch   1 Batch  156/269 - Train Accuracy:  0.554, Validation Accuracy:  0.577, Loss:  0.924\n",
      "Epoch   1 Batch  157/269 - Train Accuracy:  0.559, Validation Accuracy:  0.582, Loss:  0.889\n",
      "Epoch   1 Batch  158/269 - Train Accuracy:  0.580, Validation Accuracy:  0.580, Loss:  0.877\n",
      "Epoch   1 Batch  159/269 - Train Accuracy:  0.564, Validation Accuracy:  0.582, Loss:  0.889\n",
      "Epoch   1 Batch  160/269 - Train Accuracy:  0.563, Validation Accuracy:  0.575, Loss:  0.880\n",
      "Epoch   1 Batch  161/269 - Train Accuracy:  0.563, Validation Accuracy:  0.578, Loss:  0.887\n",
      "Epoch   1 Batch  162/269 - Train Accuracy:  0.571, Validation Accuracy:  0.584, Loss:  0.869\n",
      "Epoch   1 Batch  163/269 - Train Accuracy:  0.581, Validation Accuracy:  0.584, Loss:  0.864\n",
      "Epoch   1 Batch  164/269 - Train Accuracy:  0.580, Validation Accuracy:  0.578, Loss:  0.866\n",
      "Epoch   1 Batch  165/269 - Train Accuracy:  0.551, Validation Accuracy:  0.584, Loss:  0.892\n",
      "Epoch   1 Batch  166/269 - Train Accuracy:  0.596, Validation Accuracy:  0.585, Loss:  0.819\n",
      "Epoch   1 Batch  167/269 - Train Accuracy:  0.573, Validation Accuracy:  0.583, Loss:  0.869\n",
      "Epoch   1 Batch  168/269 - Train Accuracy:  0.569, Validation Accuracy:  0.581, Loss:  0.871\n",
      "Epoch   1 Batch  169/269 - Train Accuracy:  0.582, Validation Accuracy:  0.587, Loss:  0.864\n",
      "Epoch   1 Batch  170/269 - Train Accuracy:  0.574, Validation Accuracy:  0.589, Loss:  0.856\n",
      "Epoch   1 Batch  171/269 - Train Accuracy:  0.562, Validation Accuracy:  0.582, Loss:  0.891\n",
      "Epoch   1 Batch  172/269 - Train Accuracy:  0.577, Validation Accuracy:  0.582, Loss:  0.862\n",
      "Epoch   1 Batch  173/269 - Train Accuracy:  0.579, Validation Accuracy:  0.584, Loss:  0.844\n",
      "Epoch   1 Batch  174/269 - Train Accuracy:  0.580, Validation Accuracy:  0.587, Loss:  0.854\n",
      "Epoch   1 Batch  175/269 - Train Accuracy:  0.592, Validation Accuracy:  0.591, Loss:  0.864\n",
      "Epoch   1 Batch  176/269 - Train Accuracy:  0.572, Validation Accuracy:  0.591, Loss:  0.899\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   1 Batch  177/269 - Train Accuracy:  0.596, Validation Accuracy:  0.590, Loss:  0.816\n",
      "Epoch   1 Batch  178/269 - Train Accuracy:  0.572, Validation Accuracy:  0.587, Loss:  0.868\n",
      "Epoch   1 Batch  179/269 - Train Accuracy:  0.581, Validation Accuracy:  0.583, Loss:  0.846\n",
      "Epoch   1 Batch  180/269 - Train Accuracy:  0.578, Validation Accuracy:  0.582, Loss:  0.830\n",
      "Epoch   1 Batch  181/269 - Train Accuracy:  0.572, Validation Accuracy:  0.585, Loss:  0.842\n",
      "Epoch   1 Batch  182/269 - Train Accuracy:  0.586, Validation Accuracy:  0.580, Loss:  0.848\n",
      "Epoch   1 Batch  183/269 - Train Accuracy:  0.633, Validation Accuracy:  0.584, Loss:  0.734\n",
      "Epoch   1 Batch  184/269 - Train Accuracy:  0.559, Validation Accuracy:  0.588, Loss:  0.866\n",
      "Epoch   1 Batch  185/269 - Train Accuracy:  0.596, Validation Accuracy:  0.589, Loss:  0.833\n",
      "Epoch   1 Batch  186/269 - Train Accuracy:  0.571, Validation Accuracy:  0.590, Loss:  0.858\n",
      "Epoch   1 Batch  187/269 - Train Accuracy:  0.595, Validation Accuracy:  0.591, Loss:  0.816\n",
      "Epoch   1 Batch  188/269 - Train Accuracy:  0.602, Validation Accuracy:  0.596, Loss:  0.804\n",
      "Epoch   1 Batch  189/269 - Train Accuracy:  0.600, Validation Accuracy:  0.597, Loss:  0.813\n",
      "Epoch   1 Batch  190/269 - Train Accuracy:  0.586, Validation Accuracy:  0.599, Loss:  0.809\n",
      "Epoch   1 Batch  191/269 - Train Accuracy:  0.605, Validation Accuracy:  0.599, Loss:  0.817\n",
      "Epoch   1 Batch  192/269 - Train Accuracy:  0.602, Validation Accuracy:  0.600, Loss:  0.820\n",
      "Epoch   1 Batch  193/269 - Train Accuracy:  0.594, Validation Accuracy:  0.601, Loss:  0.818\n",
      "Epoch   1 Batch  194/269 - Train Accuracy:  0.604, Validation Accuracy:  0.603, Loss:  0.823\n",
      "Epoch   1 Batch  195/269 - Train Accuracy:  0.589, Validation Accuracy:  0.612, Loss:  0.819\n",
      "Epoch   1 Batch  196/269 - Train Accuracy:  0.586, Validation Accuracy:  0.609, Loss:  0.813\n",
      "Epoch   1 Batch  197/269 - Train Accuracy:  0.582, Validation Accuracy:  0.611, Loss:  0.849\n",
      "Epoch   1 Batch  198/269 - Train Accuracy:  0.592, Validation Accuracy:  0.613, Loss:  0.860\n",
      "Epoch   1 Batch  199/269 - Train Accuracy:  0.595, Validation Accuracy:  0.613, Loss:  0.825\n",
      "Epoch   1 Batch  200/269 - Train Accuracy:  0.591, Validation Accuracy:  0.609, Loss:  0.834\n",
      "Epoch   1 Batch  201/269 - Train Accuracy:  0.596, Validation Accuracy:  0.608, Loss:  0.806\n",
      "Epoch   1 Batch  202/269 - Train Accuracy:  0.603, Validation Accuracy:  0.609, Loss:  0.807\n",
      "Epoch   1 Batch  203/269 - Train Accuracy:  0.588, Validation Accuracy:  0.608, Loss:  0.852\n",
      "Epoch   1 Batch  204/269 - Train Accuracy:  0.586, Validation Accuracy:  0.613, Loss:  0.833\n",
      "Epoch   1 Batch  205/269 - Train Accuracy:  0.596, Validation Accuracy:  0.607, Loss:  0.792\n",
      "Epoch   1 Batch  206/269 - Train Accuracy:  0.593, Validation Accuracy:  0.605, Loss:  0.835\n",
      "Epoch   1 Batch  207/269 - Train Accuracy:  0.630, Validation Accuracy:  0.610, Loss:  0.775\n",
      "Epoch   1 Batch  208/269 - Train Accuracy:  0.592, Validation Accuracy:  0.611, Loss:  0.839\n",
      "Epoch   1 Batch  209/269 - Train Accuracy:  0.607, Validation Accuracy:  0.609, Loss:  0.808\n",
      "Epoch   1 Batch  210/269 - Train Accuracy:  0.615, Validation Accuracy:  0.607, Loss:  0.781\n",
      "Epoch   1 Batch  211/269 - Train Accuracy:  0.604, Validation Accuracy:  0.609, Loss:  0.802\n",
      "Epoch   1 Batch  212/269 - Train Accuracy:  0.621, Validation Accuracy:  0.616, Loss:  0.777\n",
      "Epoch   1 Batch  213/269 - Train Accuracy:  0.608, Validation Accuracy:  0.616, Loss:  0.784\n",
      "Epoch   1 Batch  214/269 - Train Accuracy:  0.615, Validation Accuracy:  0.615, Loss:  0.781\n",
      "Epoch   1 Batch  215/269 - Train Accuracy:  0.628, Validation Accuracy:  0.610, Loss:  0.740\n",
      "Epoch   1 Batch  216/269 - Train Accuracy:  0.579, Validation Accuracy:  0.615, Loss:  0.842\n",
      "Epoch   1 Batch  217/269 - Train Accuracy:  0.581, Validation Accuracy:  0.612, Loss:  0.814\n",
      "Epoch   1 Batch  218/269 - Train Accuracy:  0.594, Validation Accuracy:  0.617, Loss:  0.818\n",
      "Epoch   1 Batch  219/269 - Train Accuracy:  0.603, Validation Accuracy:  0.622, Loss:  0.810\n",
      "Epoch   1 Batch  220/269 - Train Accuracy:  0.618, Validation Accuracy:  0.619, Loss:  0.748\n",
      "Epoch   1 Batch  221/269 - Train Accuracy:  0.619, Validation Accuracy:  0.615, Loss:  0.773\n",
      "Epoch   1 Batch  222/269 - Train Accuracy:  0.616, Validation Accuracy:  0.613, Loss:  0.745\n",
      "Epoch   1 Batch  223/269 - Train Accuracy:  0.603, Validation Accuracy:  0.615, Loss:  0.753\n",
      "Epoch   1 Batch  224/269 - Train Accuracy:  0.619, Validation Accuracy:  0.618, Loss:  0.791\n",
      "Epoch   1 Batch  225/269 - Train Accuracy:  0.604, Validation Accuracy:  0.623, Loss:  0.788\n",
      "Epoch   1 Batch  226/269 - Train Accuracy:  0.597, Validation Accuracy:  0.619, Loss:  0.771\n",
      "Epoch   1 Batch  227/269 - Train Accuracy:  0.661, Validation Accuracy:  0.616, Loss:  0.679\n",
      "Epoch   1 Batch  228/269 - Train Accuracy:  0.601, Validation Accuracy:  0.619, Loss:  0.772\n",
      "Epoch   1 Batch  229/269 - Train Accuracy:  0.609, Validation Accuracy:  0.618, Loss:  0.756\n",
      "Epoch   1 Batch  230/269 - Train Accuracy:  0.603, Validation Accuracy:  0.625, Loss:  0.771\n",
      "Epoch   1 Batch  231/269 - Train Accuracy:  0.591, Validation Accuracy:  0.620, Loss:  0.803\n",
      "Epoch   1 Batch  232/269 - Train Accuracy:  0.589, Validation Accuracy:  0.627, Loss:  0.798\n",
      "Epoch   1 Batch  233/269 - Train Accuracy:  0.627, Validation Accuracy:  0.627, Loss:  0.769\n",
      "Epoch   1 Batch  234/269 - Train Accuracy:  0.621, Validation Accuracy:  0.626, Loss:  0.765\n",
      "Epoch   1 Batch  235/269 - Train Accuracy:  0.628, Validation Accuracy:  0.626, Loss:  0.751\n",
      "Epoch   1 Batch  236/269 - Train Accuracy:  0.611, Validation Accuracy:  0.628, Loss:  0.755\n",
      "Epoch   1 Batch  237/269 - Train Accuracy:  0.608, Validation Accuracy:  0.627, Loss:  0.755\n",
      "Epoch   1 Batch  238/269 - Train Accuracy:  0.638, Validation Accuracy:  0.626, Loss:  0.748\n",
      "Epoch   1 Batch  239/269 - Train Accuracy:  0.623, Validation Accuracy:  0.630, Loss:  0.750\n",
      "Epoch   1 Batch  240/269 - Train Accuracy:  0.650, Validation Accuracy:  0.627, Loss:  0.692\n",
      "Epoch   1 Batch  241/269 - Train Accuracy:  0.622, Validation Accuracy:  0.626, Loss:  0.753\n",
      "Epoch   1 Batch  242/269 - Train Accuracy:  0.610, Validation Accuracy:  0.624, Loss:  0.747\n",
      "Epoch   1 Batch  243/269 - Train Accuracy:  0.638, Validation Accuracy:  0.621, Loss:  0.722\n",
      "Epoch   1 Batch  244/269 - Train Accuracy:  0.606, Validation Accuracy:  0.623, Loss:  0.750\n",
      "Epoch   1 Batch  245/269 - Train Accuracy:  0.605, Validation Accuracy:  0.628, Loss:  0.787\n",
      "Epoch   1 Batch  246/269 - Train Accuracy:  0.606, Validation Accuracy:  0.627, Loss:  0.754\n",
      "Epoch   1 Batch  247/269 - Train Accuracy:  0.606, Validation Accuracy:  0.628, Loss:  0.774\n",
      "Epoch   1 Batch  248/269 - Train Accuracy:  0.618, Validation Accuracy:  0.630, Loss:  0.735\n",
      "Epoch   1 Batch  249/269 - Train Accuracy:  0.647, Validation Accuracy:  0.630, Loss:  0.709\n",
      "Epoch   1 Batch  250/269 - Train Accuracy:  0.608, Validation Accuracy:  0.631, Loss:  0.763\n",
      "Epoch   1 Batch  251/269 - Train Accuracy:  0.646, Validation Accuracy:  0.631, Loss:  0.726\n",
      "Epoch   1 Batch  252/269 - Train Accuracy:  0.617, Validation Accuracy:  0.631, Loss:  0.748\n",
      "Epoch   1 Batch  253/269 - Train Accuracy:  0.615, Validation Accuracy:  0.634, Loss:  0.746\n",
      "Epoch   1 Batch  254/269 - Train Accuracy:  0.624, Validation Accuracy:  0.633, Loss:  0.731\n",
      "Epoch   1 Batch  255/269 - Train Accuracy:  0.644, Validation Accuracy:  0.630, Loss:  0.706\n",
      "Epoch   1 Batch  256/269 - Train Accuracy:  0.608, Validation Accuracy:  0.632, Loss:  0.745\n",
      "Epoch   1 Batch  257/269 - Train Accuracy:  0.603, Validation Accuracy:  0.635, Loss:  0.744\n",
      "Epoch   1 Batch  258/269 - Train Accuracy:  0.624, Validation Accuracy:  0.632, Loss:  0.736\n",
      "Epoch   1 Batch  259/269 - Train Accuracy:  0.633, Validation Accuracy:  0.635, Loss:  0.729\n",
      "Epoch   1 Batch  260/269 - Train Accuracy:  0.609, Validation Accuracy:  0.631, Loss:  0.763\n",
      "Epoch   1 Batch  261/269 - Train Accuracy:  0.594, Validation Accuracy:  0.631, Loss:  0.773\n",
      "Epoch   1 Batch  262/269 - Train Accuracy:  0.625, Validation Accuracy:  0.630, Loss:  0.730\n",
      "Epoch   1 Batch  263/269 - Train Accuracy:  0.627, Validation Accuracy:  0.628, Loss:  0.756\n",
      "Epoch   1 Batch  264/269 - Train Accuracy:  0.597, Validation Accuracy:  0.630, Loss:  0.759\n",
      "Epoch   1 Batch  265/269 - Train Accuracy:  0.611, Validation Accuracy:  0.632, Loss:  0.749\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   1 Batch  266/269 - Train Accuracy:  0.626, Validation Accuracy:  0.628, Loss:  0.712\n",
      "Epoch   1 Batch  267/269 - Train Accuracy:  0.615, Validation Accuracy:  0.626, Loss:  0.737\n",
      "Epoch   2 Batch    0/269 - Train Accuracy:  0.608, Validation Accuracy:  0.629, Loss:  0.762\n",
      "Epoch   2 Batch    1/269 - Train Accuracy:  0.601, Validation Accuracy:  0.630, Loss:  0.740\n",
      "Epoch   2 Batch    2/269 - Train Accuracy:  0.603, Validation Accuracy:  0.634, Loss:  0.730\n",
      "Epoch   2 Batch    3/269 - Train Accuracy:  0.619, Validation Accuracy:  0.633, Loss:  0.739\n",
      "Epoch   2 Batch    4/269 - Train Accuracy:  0.594, Validation Accuracy:  0.633, Loss:  0.751\n",
      "Epoch   2 Batch    5/269 - Train Accuracy:  0.587, Validation Accuracy:  0.634, Loss:  0.752\n",
      "Epoch   2 Batch    6/269 - Train Accuracy:  0.626, Validation Accuracy:  0.637, Loss:  0.692\n",
      "Epoch   2 Batch    7/269 - Train Accuracy:  0.631, Validation Accuracy:  0.638, Loss:  0.705\n",
      "Epoch   2 Batch    8/269 - Train Accuracy:  0.606, Validation Accuracy:  0.637, Loss:  0.746\n",
      "Epoch   2 Batch    9/269 - Train Accuracy:  0.614, Validation Accuracy:  0.636, Loss:  0.726\n",
      "Epoch   2 Batch   10/269 - Train Accuracy:  0.612, Validation Accuracy:  0.635, Loss:  0.731\n",
      "Epoch   2 Batch   11/269 - Train Accuracy:  0.624, Validation Accuracy:  0.637, Loss:  0.722\n",
      "Epoch   2 Batch   12/269 - Train Accuracy:  0.602, Validation Accuracy:  0.638, Loss:  0.743\n",
      "Epoch   2 Batch   13/269 - Train Accuracy:  0.642, Validation Accuracy:  0.639, Loss:  0.661\n",
      "Epoch   2 Batch   14/269 - Train Accuracy:  0.618, Validation Accuracy:  0.640, Loss:  0.709\n",
      "Epoch   2 Batch   15/269 - Train Accuracy:  0.612, Validation Accuracy:  0.640, Loss:  0.703\n",
      "Epoch   2 Batch   16/269 - Train Accuracy:  0.646, Validation Accuracy:  0.639, Loss:  0.701\n",
      "Epoch   2 Batch   17/269 - Train Accuracy:  0.622, Validation Accuracy:  0.634, Loss:  0.693\n",
      "Epoch   2 Batch   18/269 - Train Accuracy:  0.602, Validation Accuracy:  0.632, Loss:  0.718\n",
      "Epoch   2 Batch   19/269 - Train Accuracy:  0.648, Validation Accuracy:  0.630, Loss:  0.658\n",
      "Epoch   2 Batch   20/269 - Train Accuracy:  0.615, Validation Accuracy:  0.637, Loss:  0.726\n",
      "Epoch   2 Batch   21/269 - Train Accuracy:  0.619, Validation Accuracy:  0.637, Loss:  0.746\n",
      "Epoch   2 Batch   22/269 - Train Accuracy:  0.642, Validation Accuracy:  0.637, Loss:  0.679\n",
      "Epoch   2 Batch   23/269 - Train Accuracy:  0.626, Validation Accuracy:  0.637, Loss:  0.696\n",
      "Epoch   2 Batch   24/269 - Train Accuracy:  0.613, Validation Accuracy:  0.638, Loss:  0.737\n",
      "Epoch   2 Batch   25/269 - Train Accuracy:  0.599, Validation Accuracy:  0.640, Loss:  0.739\n",
      "Epoch   2 Batch   26/269 - Train Accuracy:  0.647, Validation Accuracy:  0.639, Loss:  0.659\n",
      "Epoch   2 Batch   27/269 - Train Accuracy:  0.613, Validation Accuracy:  0.635, Loss:  0.691\n",
      "Epoch   2 Batch   28/269 - Train Accuracy:  0.583, Validation Accuracy:  0.638, Loss:  0.746\n",
      "Epoch   2 Batch   29/269 - Train Accuracy:  0.617, Validation Accuracy:  0.639, Loss:  0.715\n",
      "Epoch   2 Batch   30/269 - Train Accuracy:  0.639, Validation Accuracy:  0.641, Loss:  0.686\n",
      "Epoch   2 Batch   31/269 - Train Accuracy:  0.640, Validation Accuracy:  0.643, Loss:  0.675\n",
      "Epoch   2 Batch   32/269 - Train Accuracy:  0.618, Validation Accuracy:  0.638, Loss:  0.682\n",
      "Epoch   2 Batch   33/269 - Train Accuracy:  0.638, Validation Accuracy:  0.637, Loss:  0.665\n",
      "Epoch   2 Batch   34/269 - Train Accuracy:  0.633, Validation Accuracy:  0.640, Loss:  0.681\n",
      "Epoch   2 Batch   35/269 - Train Accuracy:  0.638, Validation Accuracy:  0.642, Loss:  0.694\n",
      "Epoch   2 Batch   36/269 - Train Accuracy:  0.627, Validation Accuracy:  0.642, Loss:  0.690\n",
      "Epoch   2 Batch   37/269 - Train Accuracy:  0.629, Validation Accuracy:  0.637, Loss:  0.679\n",
      "Epoch   2 Batch   38/269 - Train Accuracy:  0.629, Validation Accuracy:  0.640, Loss:  0.684\n",
      "Epoch   2 Batch   39/269 - Train Accuracy:  0.630, Validation Accuracy:  0.644, Loss:  0.678\n",
      "Epoch   2 Batch   40/269 - Train Accuracy:  0.616, Validation Accuracy:  0.646, Loss:  0.711\n",
      "Epoch   2 Batch   41/269 - Train Accuracy:  0.629, Validation Accuracy:  0.646, Loss:  0.692\n",
      "Epoch   2 Batch   42/269 - Train Accuracy:  0.647, Validation Accuracy:  0.647, Loss:  0.650\n",
      "Epoch   2 Batch   43/269 - Train Accuracy:  0.624, Validation Accuracy:  0.647, Loss:  0.703\n",
      "Epoch   2 Batch   44/269 - Train Accuracy:  0.636, Validation Accuracy:  0.644, Loss:  0.680\n",
      "Epoch   2 Batch   45/269 - Train Accuracy:  0.616, Validation Accuracy:  0.643, Loss:  0.712\n",
      "Epoch   2 Batch   46/269 - Train Accuracy:  0.618, Validation Accuracy:  0.642, Loss:  0.702\n",
      "Epoch   2 Batch   47/269 - Train Accuracy:  0.653, Validation Accuracy:  0.640, Loss:  0.633\n",
      "Epoch   2 Batch   48/269 - Train Accuracy:  0.624, Validation Accuracy:  0.643, Loss:  0.664\n",
      "Epoch   2 Batch   49/269 - Train Accuracy:  0.606, Validation Accuracy:  0.645, Loss:  0.696\n",
      "Epoch   2 Batch   50/269 - Train Accuracy:  0.616, Validation Accuracy:  0.644, Loss:  0.702\n",
      "Epoch   2 Batch   51/269 - Train Accuracy:  0.626, Validation Accuracy:  0.639, Loss:  0.676\n",
      "Epoch   2 Batch   52/269 - Train Accuracy:  0.620, Validation Accuracy:  0.640, Loss:  0.656\n",
      "Epoch   2 Batch   53/269 - Train Accuracy:  0.612, Validation Accuracy:  0.645, Loss:  0.706\n",
      "Epoch   2 Batch   54/269 - Train Accuracy:  0.632, Validation Accuracy:  0.647, Loss:  0.691\n",
      "Epoch   2 Batch   55/269 - Train Accuracy:  0.642, Validation Accuracy:  0.642, Loss:  0.661\n",
      "Epoch   2 Batch   56/269 - Train Accuracy:  0.638, Validation Accuracy:  0.644, Loss:  0.669\n",
      "Epoch   2 Batch   57/269 - Train Accuracy:  0.630, Validation Accuracy:  0.646, Loss:  0.680\n",
      "Epoch   2 Batch   58/269 - Train Accuracy:  0.634, Validation Accuracy:  0.645, Loss:  0.659\n",
      "Epoch   2 Batch   59/269 - Train Accuracy:  0.639, Validation Accuracy:  0.643, Loss:  0.638\n",
      "Epoch   2 Batch   60/269 - Train Accuracy:  0.634, Validation Accuracy:  0.645, Loss:  0.640\n",
      "Epoch   2 Batch   61/269 - Train Accuracy:  0.651, Validation Accuracy:  0.644, Loss:  0.621\n",
      "Epoch   2 Batch   62/269 - Train Accuracy:  0.642, Validation Accuracy:  0.646, Loss:  0.638\n",
      "Epoch   2 Batch   63/269 - Train Accuracy:  0.623, Validation Accuracy:  0.644, Loss:  0.668\n",
      "Epoch   2 Batch   64/269 - Train Accuracy:  0.622, Validation Accuracy:  0.647, Loss:  0.654\n",
      "Epoch   2 Batch   65/269 - Train Accuracy:  0.636, Validation Accuracy:  0.647, Loss:  0.655\n",
      "Epoch   2 Batch   66/269 - Train Accuracy:  0.639, Validation Accuracy:  0.649, Loss:  0.640\n",
      "Epoch   2 Batch   67/269 - Train Accuracy:  0.633, Validation Accuracy:  0.649, Loss:  0.669\n",
      "Epoch   2 Batch   68/269 - Train Accuracy:  0.620, Validation Accuracy:  0.646, Loss:  0.661\n",
      "Epoch   2 Batch   69/269 - Train Accuracy:  0.612, Validation Accuracy:  0.649, Loss:  0.718\n",
      "Epoch   2 Batch   70/269 - Train Accuracy:  0.657, Validation Accuracy:  0.653, Loss:  0.660\n",
      "Epoch   2 Batch   71/269 - Train Accuracy:  0.625, Validation Accuracy:  0.652, Loss:  0.687\n",
      "Epoch   2 Batch   72/269 - Train Accuracy:  0.638, Validation Accuracy:  0.648, Loss:  0.648\n",
      "Epoch   2 Batch   73/269 - Train Accuracy:  0.633, Validation Accuracy:  0.645, Loss:  0.668\n",
      "Epoch   2 Batch   74/269 - Train Accuracy:  0.631, Validation Accuracy:  0.649, Loss:  0.672\n",
      "Epoch   2 Batch   75/269 - Train Accuracy:  0.638, Validation Accuracy:  0.648, Loss:  0.649\n",
      "Epoch   2 Batch   76/269 - Train Accuracy:  0.623, Validation Accuracy:  0.652, Loss:  0.670\n",
      "Epoch   2 Batch   77/269 - Train Accuracy:  0.642, Validation Accuracy:  0.651, Loss:  0.650\n",
      "Epoch   2 Batch   78/269 - Train Accuracy:  0.641, Validation Accuracy:  0.651, Loss:  0.644\n",
      "Epoch   2 Batch   79/269 - Train Accuracy:  0.633, Validation Accuracy:  0.651, Loss:  0.645\n",
      "Epoch   2 Batch   80/269 - Train Accuracy:  0.642, Validation Accuracy:  0.650, Loss:  0.643\n",
      "Epoch   2 Batch   81/269 - Train Accuracy:  0.644, Validation Accuracy:  0.650, Loss:  0.663\n",
      "Epoch   2 Batch   82/269 - Train Accuracy:  0.650, Validation Accuracy:  0.649, Loss:  0.623\n",
      "Epoch   2 Batch   83/269 - Train Accuracy:  0.634, Validation Accuracy:  0.650, Loss:  0.652\n",
      "Epoch   2 Batch   84/269 - Train Accuracy:  0.636, Validation Accuracy:  0.658, Loss:  0.634\n",
      "Epoch   2 Batch   85/269 - Train Accuracy:  0.640, Validation Accuracy:  0.658, Loss:  0.646\n",
      "Epoch   2 Batch   86/269 - Train Accuracy:  0.613, Validation Accuracy:  0.657, Loss:  0.643\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   2 Batch   87/269 - Train Accuracy:  0.614, Validation Accuracy:  0.652, Loss:  0.687\n",
      "Epoch   2 Batch   88/269 - Train Accuracy:  0.629, Validation Accuracy:  0.655, Loss:  0.643\n",
      "Epoch   2 Batch   89/269 - Train Accuracy:  0.646, Validation Accuracy:  0.660, Loss:  0.641\n",
      "Epoch   2 Batch   90/269 - Train Accuracy:  0.607, Validation Accuracy:  0.655, Loss:  0.678\n",
      "Epoch   2 Batch   91/269 - Train Accuracy:  0.640, Validation Accuracy:  0.651, Loss:  0.624\n",
      "Epoch   2 Batch   92/269 - Train Accuracy:  0.636, Validation Accuracy:  0.653, Loss:  0.636\n",
      "Epoch   2 Batch   93/269 - Train Accuracy:  0.659, Validation Accuracy:  0.658, Loss:  0.607\n",
      "Epoch   2 Batch   94/269 - Train Accuracy:  0.640, Validation Accuracy:  0.656, Loss:  0.650\n",
      "Epoch   2 Batch   95/269 - Train Accuracy:  0.634, Validation Accuracy:  0.655, Loss:  0.641\n",
      "Epoch   2 Batch   96/269 - Train Accuracy:  0.638, Validation Accuracy:  0.654, Loss:  0.638\n",
      "Epoch   2 Batch   97/269 - Train Accuracy:  0.634, Validation Accuracy:  0.652, Loss:  0.636\n",
      "Epoch   2 Batch   98/269 - Train Accuracy:  0.644, Validation Accuracy:  0.661, Loss:  0.637\n",
      "Epoch   2 Batch   99/269 - Train Accuracy:  0.634, Validation Accuracy:  0.660, Loss:  0.661\n",
      "Epoch   2 Batch  100/269 - Train Accuracy:  0.665, Validation Accuracy:  0.658, Loss:  0.626\n",
      "Epoch   2 Batch  101/269 - Train Accuracy:  0.613, Validation Accuracy:  0.657, Loss:  0.668\n",
      "Epoch   2 Batch  102/269 - Train Accuracy:  0.640, Validation Accuracy:  0.657, Loss:  0.635\n",
      "Epoch   2 Batch  103/269 - Train Accuracy:  0.635, Validation Accuracy:  0.659, Loss:  0.621\n",
      "Epoch   2 Batch  104/269 - Train Accuracy:  0.642, Validation Accuracy:  0.659, Loss:  0.626\n",
      "Epoch   2 Batch  105/269 - Train Accuracy:  0.636, Validation Accuracy:  0.659, Loss:  0.643\n",
      "Epoch   2 Batch  106/269 - Train Accuracy:  0.630, Validation Accuracy:  0.654, Loss:  0.628\n",
      "Epoch   2 Batch  107/269 - Train Accuracy:  0.609, Validation Accuracy:  0.655, Loss:  0.665\n",
      "Epoch   2 Batch  108/269 - Train Accuracy:  0.637, Validation Accuracy:  0.658, Loss:  0.634\n",
      "Epoch   2 Batch  109/269 - Train Accuracy:  0.612, Validation Accuracy:  0.659, Loss:  0.639\n",
      "Epoch   2 Batch  110/269 - Train Accuracy:  0.636, Validation Accuracy:  0.658, Loss:  0.621\n",
      "Epoch   2 Batch  111/269 - Train Accuracy:  0.609, Validation Accuracy:  0.653, Loss:  0.662\n",
      "Epoch   2 Batch  112/269 - Train Accuracy:  0.641, Validation Accuracy:  0.654, Loss:  0.625\n",
      "Epoch   2 Batch  113/269 - Train Accuracy:  0.647, Validation Accuracy:  0.657, Loss:  0.603\n",
      "Epoch   2 Batch  114/269 - Train Accuracy:  0.643, Validation Accuracy:  0.657, Loss:  0.623\n",
      "Epoch   2 Batch  115/269 - Train Accuracy:  0.636, Validation Accuracy:  0.660, Loss:  0.649\n",
      "Epoch   2 Batch  116/269 - Train Accuracy:  0.642, Validation Accuracy:  0.656, Loss:  0.633\n",
      "Epoch   2 Batch  117/269 - Train Accuracy:  0.642, Validation Accuracy:  0.656, Loss:  0.623\n",
      "Epoch   2 Batch  118/269 - Train Accuracy:  0.675, Validation Accuracy:  0.659, Loss:  0.606\n",
      "Epoch   2 Batch  119/269 - Train Accuracy:  0.643, Validation Accuracy:  0.661, Loss:  0.649\n",
      "Epoch   2 Batch  120/269 - Train Accuracy:  0.632, Validation Accuracy:  0.662, Loss:  0.641\n",
      "Epoch   2 Batch  121/269 - Train Accuracy:  0.637, Validation Accuracy:  0.661, Loss:  0.617\n",
      "Epoch   2 Batch  122/269 - Train Accuracy:  0.651, Validation Accuracy:  0.660, Loss:  0.612\n",
      "Epoch   2 Batch  123/269 - Train Accuracy:  0.625, Validation Accuracy:  0.661, Loss:  0.643\n",
      "Epoch   2 Batch  124/269 - Train Accuracy:  0.630, Validation Accuracy:  0.656, Loss:  0.611\n",
      "Epoch   2 Batch  125/269 - Train Accuracy:  0.643, Validation Accuracy:  0.655, Loss:  0.612\n",
      "Epoch   2 Batch  126/269 - Train Accuracy:  0.634, Validation Accuracy:  0.656, Loss:  0.610\n",
      "Epoch   2 Batch  127/269 - Train Accuracy:  0.617, Validation Accuracy:  0.656, Loss:  0.642\n",
      "Epoch   2 Batch  128/269 - Train Accuracy:  0.656, Validation Accuracy:  0.659, Loss:  0.617\n",
      "Epoch   2 Batch  129/269 - Train Accuracy:  0.641, Validation Accuracy:  0.658, Loss:  0.614\n",
      "Epoch   2 Batch  130/269 - Train Accuracy:  0.619, Validation Accuracy:  0.657, Loss:  0.643\n",
      "Epoch   2 Batch  131/269 - Train Accuracy:  0.624, Validation Accuracy:  0.653, Loss:  0.633\n",
      "Epoch   2 Batch  132/269 - Train Accuracy:  0.635, Validation Accuracy:  0.656, Loss:  0.626\n",
      "Epoch   2 Batch  133/269 - Train Accuracy:  0.648, Validation Accuracy:  0.657, Loss:  0.597\n",
      "Epoch   2 Batch  134/269 - Train Accuracy:  0.635, Validation Accuracy:  0.658, Loss:  0.627\n",
      "Epoch   2 Batch  135/269 - Train Accuracy:  0.610, Validation Accuracy:  0.658, Loss:  0.661\n",
      "Epoch   2 Batch  136/269 - Train Accuracy:  0.619, Validation Accuracy:  0.660, Loss:  0.652\n",
      "Epoch   2 Batch  137/269 - Train Accuracy:  0.638, Validation Accuracy:  0.664, Loss:  0.644\n",
      "Epoch   2 Batch  138/269 - Train Accuracy:  0.640, Validation Accuracy:  0.660, Loss:  0.630\n",
      "Epoch   2 Batch  139/269 - Train Accuracy:  0.656, Validation Accuracy:  0.659, Loss:  0.591\n",
      "Epoch   2 Batch  140/269 - Train Accuracy:  0.653, Validation Accuracy:  0.658, Loss:  0.620\n",
      "Epoch   2 Batch  141/269 - Train Accuracy:  0.637, Validation Accuracy:  0.662, Loss:  0.629\n",
      "Epoch   2 Batch  142/269 - Train Accuracy:  0.652, Validation Accuracy:  0.663, Loss:  0.593\n",
      "Epoch   2 Batch  143/269 - Train Accuracy:  0.637, Validation Accuracy:  0.662, Loss:  0.610\n",
      "Epoch   2 Batch  144/269 - Train Accuracy:  0.646, Validation Accuracy:  0.659, Loss:  0.581\n",
      "Epoch   2 Batch  145/269 - Train Accuracy:  0.646, Validation Accuracy:  0.659, Loss:  0.604\n",
      "Epoch   2 Batch  146/269 - Train Accuracy:  0.641, Validation Accuracy:  0.662, Loss:  0.597\n",
      "Epoch   2 Batch  147/269 - Train Accuracy:  0.661, Validation Accuracy:  0.663, Loss:  0.579\n",
      "Epoch   2 Batch  148/269 - Train Accuracy:  0.642, Validation Accuracy:  0.660, Loss:  0.611\n",
      "Epoch   2 Batch  149/269 - Train Accuracy:  0.648, Validation Accuracy:  0.664, Loss:  0.610\n",
      "Epoch   2 Batch  150/269 - Train Accuracy:  0.650, Validation Accuracy:  0.662, Loss:  0.608\n",
      "Epoch   2 Batch  151/269 - Train Accuracy:  0.675, Validation Accuracy:  0.661, Loss:  0.580\n",
      "Epoch   2 Batch  152/269 - Train Accuracy:  0.643, Validation Accuracy:  0.661, Loss:  0.604\n",
      "Epoch   2 Batch  153/269 - Train Accuracy:  0.643, Validation Accuracy:  0.660, Loss:  0.594\n",
      "Epoch   2 Batch  154/269 - Train Accuracy:  0.633, Validation Accuracy:  0.662, Loss:  0.616\n",
      "Epoch   2 Batch  155/269 - Train Accuracy:  0.672, Validation Accuracy:  0.665, Loss:  0.570\n",
      "Epoch   2 Batch  156/269 - Train Accuracy:  0.630, Validation Accuracy:  0.663, Loss:  0.630\n",
      "Epoch   2 Batch  157/269 - Train Accuracy:  0.637, Validation Accuracy:  0.663, Loss:  0.598\n",
      "Epoch   2 Batch  158/269 - Train Accuracy:  0.646, Validation Accuracy:  0.664, Loss:  0.598\n",
      "Epoch   2 Batch  159/269 - Train Accuracy:  0.646, Validation Accuracy:  0.665, Loss:  0.602\n",
      "Epoch   2 Batch  160/269 - Train Accuracy:  0.655, Validation Accuracy:  0.667, Loss:  0.594\n",
      "Epoch   2 Batch  161/269 - Train Accuracy:  0.642, Validation Accuracy:  0.664, Loss:  0.600\n",
      "Epoch   2 Batch  162/269 - Train Accuracy:  0.646, Validation Accuracy:  0.661, Loss:  0.591\n",
      "Epoch   2 Batch  163/269 - Train Accuracy:  0.649, Validation Accuracy:  0.664, Loss:  0.590\n",
      "Epoch   2 Batch  164/269 - Train Accuracy:  0.655, Validation Accuracy:  0.663, Loss:  0.589\n",
      "Epoch   2 Batch  165/269 - Train Accuracy:  0.628, Validation Accuracy:  0.665, Loss:  0.612\n",
      "Epoch   2 Batch  166/269 - Train Accuracy:  0.661, Validation Accuracy:  0.669, Loss:  0.564\n",
      "Epoch   2 Batch  167/269 - Train Accuracy:  0.649, Validation Accuracy:  0.667, Loss:  0.591\n",
      "Epoch   2 Batch  168/269 - Train Accuracy:  0.643, Validation Accuracy:  0.669, Loss:  0.596\n",
      "Epoch   2 Batch  169/269 - Train Accuracy:  0.648, Validation Accuracy:  0.670, Loss:  0.592\n",
      "Epoch   2 Batch  170/269 - Train Accuracy:  0.653, Validation Accuracy:  0.668, Loss:  0.586\n",
      "Epoch   2 Batch  171/269 - Train Accuracy:  0.648, Validation Accuracy:  0.665, Loss:  0.614\n",
      "Epoch   2 Batch  172/269 - Train Accuracy:  0.643, Validation Accuracy:  0.664, Loss:  0.596\n",
      "Epoch   2 Batch  173/269 - Train Accuracy:  0.655, Validation Accuracy:  0.664, Loss:  0.575\n",
      "Epoch   2 Batch  174/269 - Train Accuracy:  0.638, Validation Accuracy:  0.668, Loss:  0.588\n",
      "Epoch   2 Batch  175/269 - Train Accuracy:  0.649, Validation Accuracy:  0.670, Loss:  0.604\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   2 Batch  176/269 - Train Accuracy:  0.629, Validation Accuracy:  0.670, Loss:  0.627\n",
      "Epoch   2 Batch  177/269 - Train Accuracy:  0.667, Validation Accuracy:  0.670, Loss:  0.560\n",
      "Epoch   2 Batch  178/269 - Train Accuracy:  0.642, Validation Accuracy:  0.668, Loss:  0.599\n",
      "Epoch   2 Batch  179/269 - Train Accuracy:  0.660, Validation Accuracy:  0.670, Loss:  0.586\n",
      "Epoch   2 Batch  180/269 - Train Accuracy:  0.652, Validation Accuracy:  0.675, Loss:  0.579\n",
      "Epoch   2 Batch  181/269 - Train Accuracy:  0.637, Validation Accuracy:  0.672, Loss:  0.585\n",
      "Epoch   2 Batch  182/269 - Train Accuracy:  0.660, Validation Accuracy:  0.670, Loss:  0.585\n",
      "Epoch   2 Batch  183/269 - Train Accuracy:  0.697, Validation Accuracy:  0.669, Loss:  0.513\n",
      "Epoch   2 Batch  184/269 - Train Accuracy:  0.635, Validation Accuracy:  0.672, Loss:  0.607\n",
      "Epoch   2 Batch  185/269 - Train Accuracy:  0.659, Validation Accuracy:  0.671, Loss:  0.578\n",
      "Epoch   2 Batch  186/269 - Train Accuracy:  0.638, Validation Accuracy:  0.674, Loss:  0.599\n",
      "Epoch   2 Batch  187/269 - Train Accuracy:  0.649, Validation Accuracy:  0.672, Loss:  0.571\n",
      "Epoch   2 Batch  188/269 - Train Accuracy:  0.661, Validation Accuracy:  0.670, Loss:  0.563\n",
      "Epoch   2 Batch  189/269 - Train Accuracy:  0.652, Validation Accuracy:  0.666, Loss:  0.564\n",
      "Epoch   2 Batch  190/269 - Train Accuracy:  0.653, Validation Accuracy:  0.671, Loss:  0.567\n",
      "Epoch   2 Batch  191/269 - Train Accuracy:  0.662, Validation Accuracy:  0.672, Loss:  0.574\n",
      "Epoch   2 Batch  192/269 - Train Accuracy:  0.655, Validation Accuracy:  0.669, Loss:  0.577\n",
      "Epoch   2 Batch  193/269 - Train Accuracy:  0.652, Validation Accuracy:  0.672, Loss:  0.572\n",
      "Epoch   2 Batch  194/269 - Train Accuracy:  0.667, Validation Accuracy:  0.673, Loss:  0.579\n",
      "Epoch   2 Batch  195/269 - Train Accuracy:  0.653, Validation Accuracy:  0.674, Loss:  0.574\n",
      "Epoch   2 Batch  196/269 - Train Accuracy:  0.635, Validation Accuracy:  0.670, Loss:  0.573\n",
      "Epoch   2 Batch  197/269 - Train Accuracy:  0.628, Validation Accuracy:  0.670, Loss:  0.599\n",
      "Epoch   2 Batch  198/269 - Train Accuracy:  0.638, Validation Accuracy:  0.671, Loss:  0.603\n",
      "Epoch   2 Batch  199/269 - Train Accuracy:  0.652, Validation Accuracy:  0.675, Loss:  0.582\n",
      "Epoch   2 Batch  200/269 - Train Accuracy:  0.658, Validation Accuracy:  0.672, Loss:  0.590\n",
      "Epoch   2 Batch  201/269 - Train Accuracy:  0.651, Validation Accuracy:  0.673, Loss:  0.571\n",
      "Epoch   2 Batch  202/269 - Train Accuracy:  0.654, Validation Accuracy:  0.674, Loss:  0.569\n",
      "Epoch   2 Batch  203/269 - Train Accuracy:  0.644, Validation Accuracy:  0.675, Loss:  0.603\n",
      "Epoch   2 Batch  204/269 - Train Accuracy:  0.628, Validation Accuracy:  0.669, Loss:  0.597\n",
      "Epoch   2 Batch  205/269 - Train Accuracy:  0.653, Validation Accuracy:  0.671, Loss:  0.559\n",
      "Epoch   2 Batch  206/269 - Train Accuracy:  0.650, Validation Accuracy:  0.673, Loss:  0.589\n",
      "Epoch   2 Batch  207/269 - Train Accuracy:  0.670, Validation Accuracy:  0.673, Loss:  0.553\n",
      "Epoch   2 Batch  208/269 - Train Accuracy:  0.643, Validation Accuracy:  0.667, Loss:  0.591\n",
      "Epoch   2 Batch  209/269 - Train Accuracy:  0.657, Validation Accuracy:  0.663, Loss:  0.575\n",
      "Epoch   2 Batch  210/269 - Train Accuracy:  0.661, Validation Accuracy:  0.667, Loss:  0.557\n",
      "Epoch   2 Batch  211/269 - Train Accuracy:  0.643, Validation Accuracy:  0.669, Loss:  0.577\n",
      "Epoch   2 Batch  212/269 - Train Accuracy:  0.664, Validation Accuracy:  0.664, Loss:  0.562\n",
      "Epoch   2 Batch  213/269 - Train Accuracy:  0.654, Validation Accuracy:  0.674, Loss:  0.577\n",
      "Epoch   2 Batch  214/269 - Train Accuracy:  0.666, Validation Accuracy:  0.665, Loss:  0.562\n",
      "Epoch   2 Batch  215/269 - Train Accuracy:  0.676, Validation Accuracy:  0.666, Loss:  0.543\n",
      "Epoch   2 Batch  216/269 - Train Accuracy:  0.636, Validation Accuracy:  0.671, Loss:  0.605\n",
      "Epoch   2 Batch  217/269 - Train Accuracy:  0.630, Validation Accuracy:  0.670, Loss:  0.594\n",
      "Epoch   2 Batch  218/269 - Train Accuracy:  0.635, Validation Accuracy:  0.668, Loss:  0.587\n",
      "Epoch   2 Batch  219/269 - Train Accuracy:  0.660, Validation Accuracy:  0.668, Loss:  0.599\n",
      "Epoch   2 Batch  220/269 - Train Accuracy:  0.655, Validation Accuracy:  0.662, Loss:  0.537\n",
      "Epoch   2 Batch  221/269 - Train Accuracy:  0.685, Validation Accuracy:  0.662, Loss:  0.577\n",
      "Epoch   2 Batch  222/269 - Train Accuracy:  0.678, Validation Accuracy:  0.664, Loss:  0.546\n",
      "Epoch   2 Batch  223/269 - Train Accuracy:  0.648, Validation Accuracy:  0.669, Loss:  0.559\n",
      "Epoch   2 Batch  224/269 - Train Accuracy:  0.663, Validation Accuracy:  0.670, Loss:  0.582\n",
      "Epoch   2 Batch  225/269 - Train Accuracy:  0.655, Validation Accuracy:  0.672, Loss:  0.578\n",
      "Epoch   2 Batch  226/269 - Train Accuracy:  0.652, Validation Accuracy:  0.667, Loss:  0.561\n",
      "Epoch   2 Batch  227/269 - Train Accuracy:  0.693, Validation Accuracy:  0.658, Loss:  0.498\n",
      "Epoch   2 Batch  228/269 - Train Accuracy:  0.652, Validation Accuracy:  0.666, Loss:  0.567\n",
      "Epoch   2 Batch  229/269 - Train Accuracy:  0.653, Validation Accuracy:  0.665, Loss:  0.547\n",
      "Epoch   2 Batch  230/269 - Train Accuracy:  0.650, Validation Accuracy:  0.666, Loss:  0.558\n",
      "Epoch   2 Batch  231/269 - Train Accuracy:  0.635, Validation Accuracy:  0.667, Loss:  0.591\n",
      "Epoch   2 Batch  232/269 - Train Accuracy:  0.630, Validation Accuracy:  0.657, Loss:  0.587\n",
      "Epoch   2 Batch  233/269 - Train Accuracy:  0.664, Validation Accuracy:  0.660, Loss:  0.566\n",
      "Epoch   2 Batch  234/269 - Train Accuracy:  0.658, Validation Accuracy:  0.667, Loss:  0.556\n",
      "Epoch   2 Batch  235/269 - Train Accuracy:  0.668, Validation Accuracy:  0.669, Loss:  0.544\n",
      "Epoch   2 Batch  236/269 - Train Accuracy:  0.653, Validation Accuracy:  0.675, Loss:  0.550\n",
      "Epoch   2 Batch  237/269 - Train Accuracy:  0.647, Validation Accuracy:  0.675, Loss:  0.557\n",
      "Epoch   2 Batch  238/269 - Train Accuracy:  0.676, Validation Accuracy:  0.673, Loss:  0.544\n",
      "Epoch   2 Batch  239/269 - Train Accuracy:  0.668, Validation Accuracy:  0.671, Loss:  0.553\n",
      "Epoch   2 Batch  240/269 - Train Accuracy:  0.677, Validation Accuracy:  0.673, Loss:  0.502\n",
      "Epoch   2 Batch  241/269 - Train Accuracy:  0.656, Validation Accuracy:  0.673, Loss:  0.556\n",
      "Epoch   2 Batch  242/269 - Train Accuracy:  0.652, Validation Accuracy:  0.674, Loss:  0.552\n",
      "Epoch   2 Batch  243/269 - Train Accuracy:  0.671, Validation Accuracy:  0.674, Loss:  0.530\n",
      "Epoch   2 Batch  244/269 - Train Accuracy:  0.654, Validation Accuracy:  0.676, Loss:  0.557\n",
      "Epoch   2 Batch  245/269 - Train Accuracy:  0.645, Validation Accuracy:  0.676, Loss:  0.580\n",
      "Epoch   2 Batch  246/269 - Train Accuracy:  0.639, Validation Accuracy:  0.678, Loss:  0.549\n",
      "Epoch   2 Batch  247/269 - Train Accuracy:  0.654, Validation Accuracy:  0.679, Loss:  0.569\n",
      "Epoch   2 Batch  248/269 - Train Accuracy:  0.663, Validation Accuracy:  0.680, Loss:  0.539\n",
      "Epoch   2 Batch  249/269 - Train Accuracy:  0.691, Validation Accuracy:  0.677, Loss:  0.520\n",
      "Epoch   2 Batch  250/269 - Train Accuracy:  0.653, Validation Accuracy:  0.675, Loss:  0.561\n",
      "Epoch   2 Batch  251/269 - Train Accuracy:  0.684, Validation Accuracy:  0.676, Loss:  0.534\n",
      "Epoch   2 Batch  252/269 - Train Accuracy:  0.657, Validation Accuracy:  0.675, Loss:  0.550\n",
      "Epoch   2 Batch  253/269 - Train Accuracy:  0.661, Validation Accuracy:  0.675, Loss:  0.554\n",
      "Epoch   2 Batch  254/269 - Train Accuracy:  0.664, Validation Accuracy:  0.680, Loss:  0.539\n",
      "Epoch   2 Batch  255/269 - Train Accuracy:  0.679, Validation Accuracy:  0.677, Loss:  0.523\n",
      "Epoch   2 Batch  256/269 - Train Accuracy:  0.656, Validation Accuracy:  0.677, Loss:  0.551\n",
      "Epoch   2 Batch  257/269 - Train Accuracy:  0.645, Validation Accuracy:  0.678, Loss:  0.550\n",
      "Epoch   2 Batch  258/269 - Train Accuracy:  0.663, Validation Accuracy:  0.681, Loss:  0.541\n",
      "Epoch   2 Batch  259/269 - Train Accuracy:  0.683, Validation Accuracy:  0.680, Loss:  0.534\n",
      "Epoch   2 Batch  260/269 - Train Accuracy:  0.647, Validation Accuracy:  0.679, Loss:  0.561\n",
      "Epoch   2 Batch  261/269 - Train Accuracy:  0.645, Validation Accuracy:  0.678, Loss:  0.568\n",
      "Epoch   2 Batch  262/269 - Train Accuracy:  0.668, Validation Accuracy:  0.679, Loss:  0.538\n",
      "Epoch   2 Batch  263/269 - Train Accuracy:  0.665, Validation Accuracy:  0.680, Loss:  0.557\n",
      "Epoch   2 Batch  264/269 - Train Accuracy:  0.643, Validation Accuracy:  0.681, Loss:  0.561\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   2 Batch  265/269 - Train Accuracy:  0.655, Validation Accuracy:  0.680, Loss:  0.556\n",
      "Epoch   2 Batch  266/269 - Train Accuracy:  0.672, Validation Accuracy:  0.678, Loss:  0.527\n",
      "Epoch   2 Batch  267/269 - Train Accuracy:  0.674, Validation Accuracy:  0.679, Loss:  0.541\n",
      "Epoch   3 Batch    0/269 - Train Accuracy:  0.658, Validation Accuracy:  0.682, Loss:  0.562\n",
      "Epoch   3 Batch    1/269 - Train Accuracy:  0.649, Validation Accuracy:  0.682, Loss:  0.552\n",
      "Epoch   3 Batch    2/269 - Train Accuracy:  0.660, Validation Accuracy:  0.680, Loss:  0.540\n",
      "Epoch   3 Batch    3/269 - Train Accuracy:  0.676, Validation Accuracy:  0.682, Loss:  0.547\n",
      "Epoch   3 Batch    4/269 - Train Accuracy:  0.647, Validation Accuracy:  0.680, Loss:  0.557\n",
      "Epoch   3 Batch    5/269 - Train Accuracy:  0.650, Validation Accuracy:  0.681, Loss:  0.555\n",
      "Epoch   3 Batch    6/269 - Train Accuracy:  0.664, Validation Accuracy:  0.684, Loss:  0.520\n",
      "Epoch   3 Batch    7/269 - Train Accuracy:  0.669, Validation Accuracy:  0.688, Loss:  0.526\n",
      "Epoch   3 Batch    8/269 - Train Accuracy:  0.658, Validation Accuracy:  0.687, Loss:  0.556\n",
      "Epoch   3 Batch    9/269 - Train Accuracy:  0.668, Validation Accuracy:  0.686, Loss:  0.543\n",
      "Epoch   3 Batch   10/269 - Train Accuracy:  0.678, Validation Accuracy:  0.682, Loss:  0.549\n",
      "Epoch   3 Batch   11/269 - Train Accuracy:  0.676, Validation Accuracy:  0.684, Loss:  0.538\n",
      "Epoch   3 Batch   12/269 - Train Accuracy:  0.663, Validation Accuracy:  0.685, Loss:  0.559\n",
      "Epoch   3 Batch   13/269 - Train Accuracy:  0.692, Validation Accuracy:  0.687, Loss:  0.495\n",
      "Epoch   3 Batch   14/269 - Train Accuracy:  0.674, Validation Accuracy:  0.687, Loss:  0.530\n",
      "Epoch   3 Batch   15/269 - Train Accuracy:  0.662, Validation Accuracy:  0.687, Loss:  0.521\n",
      "Epoch   3 Batch   16/269 - Train Accuracy:  0.692, Validation Accuracy:  0.687, Loss:  0.525\n",
      "Epoch   3 Batch   17/269 - Train Accuracy:  0.682, Validation Accuracy:  0.688, Loss:  0.518\n",
      "Epoch   3 Batch   18/269 - Train Accuracy:  0.659, Validation Accuracy:  0.690, Loss:  0.537\n",
      "Epoch   3 Batch   19/269 - Train Accuracy:  0.706, Validation Accuracy:  0.686, Loss:  0.488\n",
      "Epoch   3 Batch   20/269 - Train Accuracy:  0.671, Validation Accuracy:  0.688, Loss:  0.542\n",
      "Epoch   3 Batch   21/269 - Train Accuracy:  0.661, Validation Accuracy:  0.689, Loss:  0.556\n",
      "Epoch   3 Batch   22/269 - Train Accuracy:  0.686, Validation Accuracy:  0.688, Loss:  0.506\n",
      "Epoch   3 Batch   23/269 - Train Accuracy:  0.674, Validation Accuracy:  0.684, Loss:  0.513\n",
      "Epoch   3 Batch   24/269 - Train Accuracy:  0.671, Validation Accuracy:  0.686, Loss:  0.552\n",
      "Epoch   3 Batch   25/269 - Train Accuracy:  0.651, Validation Accuracy:  0.690, Loss:  0.553\n",
      "Epoch   3 Batch   26/269 - Train Accuracy:  0.692, Validation Accuracy:  0.687, Loss:  0.490\n",
      "Epoch   3 Batch   27/269 - Train Accuracy:  0.662, Validation Accuracy:  0.686, Loss:  0.514\n",
      "Epoch   3 Batch   28/269 - Train Accuracy:  0.633, Validation Accuracy:  0.685, Loss:  0.559\n",
      "Epoch   3 Batch   29/269 - Train Accuracy:  0.665, Validation Accuracy:  0.684, Loss:  0.538\n",
      "Epoch   3 Batch   30/269 - Train Accuracy:  0.683, Validation Accuracy:  0.689, Loss:  0.514\n",
      "Epoch   3 Batch   31/269 - Train Accuracy:  0.686, Validation Accuracy:  0.691, Loss:  0.503\n",
      "Epoch   3 Batch   32/269 - Train Accuracy:  0.667, Validation Accuracy:  0.690, Loss:  0.507\n",
      "Epoch   3 Batch   33/269 - Train Accuracy:  0.687, Validation Accuracy:  0.686, Loss:  0.501\n",
      "Epoch   3 Batch   34/269 - Train Accuracy:  0.671, Validation Accuracy:  0.686, Loss:  0.508\n",
      "Epoch   3 Batch   35/269 - Train Accuracy:  0.682, Validation Accuracy:  0.684, Loss:  0.526\n",
      "Epoch   3 Batch   36/269 - Train Accuracy:  0.674, Validation Accuracy:  0.691, Loss:  0.515\n",
      "Epoch   3 Batch   37/269 - Train Accuracy:  0.688, Validation Accuracy:  0.695, Loss:  0.503\n",
      "Epoch   3 Batch   38/269 - Train Accuracy:  0.681, Validation Accuracy:  0.690, Loss:  0.511\n",
      "Epoch   3 Batch   39/269 - Train Accuracy:  0.683, Validation Accuracy:  0.685, Loss:  0.509\n",
      "Epoch   3 Batch   40/269 - Train Accuracy:  0.664, Validation Accuracy:  0.684, Loss:  0.535\n",
      "Epoch   3 Batch   41/269 - Train Accuracy:  0.673, Validation Accuracy:  0.686, Loss:  0.518\n",
      "Epoch   3 Batch   42/269 - Train Accuracy:  0.693, Validation Accuracy:  0.688, Loss:  0.485\n",
      "Epoch   3 Batch   43/269 - Train Accuracy:  0.669, Validation Accuracy:  0.688, Loss:  0.526\n",
      "Epoch   3 Batch   44/269 - Train Accuracy:  0.682, Validation Accuracy:  0.691, Loss:  0.510\n",
      "Epoch   3 Batch   45/269 - Train Accuracy:  0.670, Validation Accuracy:  0.689, Loss:  0.534\n",
      "Epoch   3 Batch   46/269 - Train Accuracy:  0.660, Validation Accuracy:  0.689, Loss:  0.527\n",
      "Epoch   3 Batch   47/269 - Train Accuracy:  0.691, Validation Accuracy:  0.689, Loss:  0.472\n",
      "Epoch   3 Batch   48/269 - Train Accuracy:  0.693, Validation Accuracy:  0.693, Loss:  0.498\n",
      "Epoch   3 Batch   49/269 - Train Accuracy:  0.667, Validation Accuracy:  0.693, Loss:  0.518\n",
      "Epoch   3 Batch   50/269 - Train Accuracy:  0.668, Validation Accuracy:  0.691, Loss:  0.525\n",
      "Epoch   3 Batch   51/269 - Train Accuracy:  0.678, Validation Accuracy:  0.687, Loss:  0.502\n",
      "Epoch   3 Batch   52/269 - Train Accuracy:  0.670, Validation Accuracy:  0.688, Loss:  0.489\n",
      "Epoch   3 Batch   53/269 - Train Accuracy:  0.676, Validation Accuracy:  0.688, Loss:  0.527\n",
      "Epoch   3 Batch   54/269 - Train Accuracy:  0.688, Validation Accuracy:  0.692, Loss:  0.517\n",
      "Epoch   3 Batch   55/269 - Train Accuracy:  0.694, Validation Accuracy:  0.692, Loss:  0.494\n",
      "Epoch   3 Batch   56/269 - Train Accuracy:  0.693, Validation Accuracy:  0.690, Loss:  0.500\n",
      "Epoch   3 Batch   57/269 - Train Accuracy:  0.675, Validation Accuracy:  0.692, Loss:  0.512\n",
      "Epoch   3 Batch   58/269 - Train Accuracy:  0.696, Validation Accuracy:  0.694, Loss:  0.490\n",
      "Epoch   3 Batch   59/269 - Train Accuracy:  0.700, Validation Accuracy:  0.693, Loss:  0.479\n",
      "Epoch   3 Batch   60/269 - Train Accuracy:  0.682, Validation Accuracy:  0.694, Loss:  0.480\n",
      "Epoch   3 Batch   61/269 - Train Accuracy:  0.697, Validation Accuracy:  0.691, Loss:  0.463\n",
      "Epoch   3 Batch   62/269 - Train Accuracy:  0.696, Validation Accuracy:  0.694, Loss:  0.476\n",
      "Epoch   3 Batch   63/269 - Train Accuracy:  0.690, Validation Accuracy:  0.697, Loss:  0.501\n",
      "Epoch   3 Batch   64/269 - Train Accuracy:  0.678, Validation Accuracy:  0.693, Loss:  0.485\n",
      "Epoch   3 Batch   65/269 - Train Accuracy:  0.683, Validation Accuracy:  0.692, Loss:  0.492\n",
      "Epoch   3 Batch   66/269 - Train Accuracy:  0.682, Validation Accuracy:  0.694, Loss:  0.479\n",
      "Epoch   3 Batch   67/269 - Train Accuracy:  0.694, Validation Accuracy:  0.697, Loss:  0.498\n",
      "Epoch   3 Batch   68/269 - Train Accuracy:  0.668, Validation Accuracy:  0.694, Loss:  0.495\n",
      "Epoch   3 Batch   69/269 - Train Accuracy:  0.648, Validation Accuracy:  0.696, Loss:  0.537\n",
      "Epoch   3 Batch   70/269 - Train Accuracy:  0.705, Validation Accuracy:  0.693, Loss:  0.493\n",
      "Epoch   3 Batch   71/269 - Train Accuracy:  0.689, Validation Accuracy:  0.699, Loss:  0.519\n",
      "Epoch   3 Batch   72/269 - Train Accuracy:  0.691, Validation Accuracy:  0.696, Loss:  0.487\n",
      "Epoch   3 Batch   73/269 - Train Accuracy:  0.690, Validation Accuracy:  0.696, Loss:  0.498\n",
      "Epoch   3 Batch   74/269 - Train Accuracy:  0.679, Validation Accuracy:  0.696, Loss:  0.503\n",
      "Epoch   3 Batch   75/269 - Train Accuracy:  0.675, Validation Accuracy:  0.701, Loss:  0.489\n",
      "Epoch   3 Batch   76/269 - Train Accuracy:  0.670, Validation Accuracy:  0.701, Loss:  0.498\n",
      "Epoch   3 Batch   77/269 - Train Accuracy:  0.700, Validation Accuracy:  0.699, Loss:  0.484\n",
      "Epoch   3 Batch   78/269 - Train Accuracy:  0.701, Validation Accuracy:  0.695, Loss:  0.481\n",
      "Epoch   3 Batch   79/269 - Train Accuracy:  0.684, Validation Accuracy:  0.702, Loss:  0.485\n",
      "Epoch   3 Batch   80/269 - Train Accuracy:  0.696, Validation Accuracy:  0.701, Loss:  0.483\n",
      "Epoch   3 Batch   81/269 - Train Accuracy:  0.693, Validation Accuracy:  0.695, Loss:  0.497\n",
      "Epoch   3 Batch   82/269 - Train Accuracy:  0.702, Validation Accuracy:  0.699, Loss:  0.471\n",
      "Epoch   3 Batch   83/269 - Train Accuracy:  0.694, Validation Accuracy:  0.699, Loss:  0.493\n",
      "Epoch   3 Batch   84/269 - Train Accuracy:  0.694, Validation Accuracy:  0.698, Loss:  0.477\n",
      "Epoch   3 Batch   85/269 - Train Accuracy:  0.702, Validation Accuracy:  0.702, Loss:  0.480\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   3 Batch   86/269 - Train Accuracy:  0.676, Validation Accuracy:  0.700, Loss:  0.477\n",
      "Epoch   3 Batch   87/269 - Train Accuracy:  0.667, Validation Accuracy:  0.702, Loss:  0.513\n",
      "Epoch   3 Batch   88/269 - Train Accuracy:  0.695, Validation Accuracy:  0.700, Loss:  0.479\n",
      "Epoch   3 Batch   89/269 - Train Accuracy:  0.701, Validation Accuracy:  0.698, Loss:  0.477\n",
      "Epoch   3 Batch   90/269 - Train Accuracy:  0.668, Validation Accuracy:  0.696, Loss:  0.507\n",
      "Epoch   3 Batch   91/269 - Train Accuracy:  0.711, Validation Accuracy:  0.700, Loss:  0.467\n",
      "Epoch   3 Batch   92/269 - Train Accuracy:  0.690, Validation Accuracy:  0.703, Loss:  0.474\n",
      "Epoch   3 Batch   93/269 - Train Accuracy:  0.697, Validation Accuracy:  0.703, Loss:  0.458\n",
      "Epoch   3 Batch   94/269 - Train Accuracy:  0.708, Validation Accuracy:  0.707, Loss:  0.488\n",
      "Epoch   3 Batch   95/269 - Train Accuracy:  0.697, Validation Accuracy:  0.704, Loss:  0.477\n",
      "Epoch   3 Batch   96/269 - Train Accuracy:  0.691, Validation Accuracy:  0.699, Loss:  0.479\n",
      "Epoch   3 Batch   97/269 - Train Accuracy:  0.686, Validation Accuracy:  0.708, Loss:  0.480\n",
      "Epoch   3 Batch   98/269 - Train Accuracy:  0.694, Validation Accuracy:  0.705, Loss:  0.476\n",
      "Epoch   3 Batch   99/269 - Train Accuracy:  0.664, Validation Accuracy:  0.696, Loss:  0.502\n",
      "Epoch   3 Batch  100/269 - Train Accuracy:  0.720, Validation Accuracy:  0.704, Loss:  0.475\n",
      "Epoch   3 Batch  101/269 - Train Accuracy:  0.685, Validation Accuracy:  0.705, Loss:  0.502\n",
      "Epoch   3 Batch  102/269 - Train Accuracy:  0.698, Validation Accuracy:  0.700, Loss:  0.486\n",
      "Epoch   3 Batch  103/269 - Train Accuracy:  0.685, Validation Accuracy:  0.695, Loss:  0.466\n",
      "Epoch   3 Batch  104/269 - Train Accuracy:  0.706, Validation Accuracy:  0.706, Loss:  0.474\n",
      "Epoch   3 Batch  105/269 - Train Accuracy:  0.690, Validation Accuracy:  0.707, Loss:  0.479\n",
      "Epoch   3 Batch  106/269 - Train Accuracy:  0.678, Validation Accuracy:  0.701, Loss:  0.466\n",
      "Epoch   3 Batch  107/269 - Train Accuracy:  0.678, Validation Accuracy:  0.704, Loss:  0.497\n",
      "Epoch   3 Batch  108/269 - Train Accuracy:  0.696, Validation Accuracy:  0.713, Loss:  0.478\n",
      "Epoch   3 Batch  109/269 - Train Accuracy:  0.680, Validation Accuracy:  0.717, Loss:  0.479\n",
      "Epoch   3 Batch  110/269 - Train Accuracy:  0.694, Validation Accuracy:  0.705, Loss:  0.466\n",
      "Epoch   3 Batch  111/269 - Train Accuracy:  0.679, Validation Accuracy:  0.707, Loss:  0.494\n",
      "Epoch   3 Batch  112/269 - Train Accuracy:  0.704, Validation Accuracy:  0.710, Loss:  0.466\n",
      "Epoch   3 Batch  113/269 - Train Accuracy:  0.700, Validation Accuracy:  0.708, Loss:  0.454\n",
      "Epoch   3 Batch  114/269 - Train Accuracy:  0.701, Validation Accuracy:  0.703, Loss:  0.468\n",
      "Epoch   3 Batch  115/269 - Train Accuracy:  0.696, Validation Accuracy:  0.711, Loss:  0.488\n",
      "Epoch   3 Batch  116/269 - Train Accuracy:  0.708, Validation Accuracy:  0.711, Loss:  0.473\n",
      "Epoch   3 Batch  117/269 - Train Accuracy:  0.700, Validation Accuracy:  0.710, Loss:  0.466\n",
      "Epoch   3 Batch  118/269 - Train Accuracy:  0.726, Validation Accuracy:  0.706, Loss:  0.454\n",
      "Epoch   3 Batch  119/269 - Train Accuracy:  0.699, Validation Accuracy:  0.709, Loss:  0.484\n",
      "Epoch   3 Batch  120/269 - Train Accuracy:  0.695, Validation Accuracy:  0.711, Loss:  0.478\n",
      "Epoch   3 Batch  121/269 - Train Accuracy:  0.698, Validation Accuracy:  0.710, Loss:  0.461\n",
      "Epoch   3 Batch  122/269 - Train Accuracy:  0.714, Validation Accuracy:  0.709, Loss:  0.452\n",
      "Epoch   3 Batch  123/269 - Train Accuracy:  0.690, Validation Accuracy:  0.712, Loss:  0.478\n",
      "Epoch   3 Batch  124/269 - Train Accuracy:  0.705, Validation Accuracy:  0.710, Loss:  0.457\n",
      "Epoch   3 Batch  125/269 - Train Accuracy:  0.701, Validation Accuracy:  0.708, Loss:  0.456\n",
      "Epoch   3 Batch  126/269 - Train Accuracy:  0.703, Validation Accuracy:  0.708, Loss:  0.457\n",
      "Epoch   3 Batch  127/269 - Train Accuracy:  0.689, Validation Accuracy:  0.706, Loss:  0.473\n",
      "Epoch   3 Batch  128/269 - Train Accuracy:  0.714, Validation Accuracy:  0.710, Loss:  0.460\n",
      "Epoch   3 Batch  129/269 - Train Accuracy:  0.714, Validation Accuracy:  0.708, Loss:  0.452\n",
      "Epoch   3 Batch  130/269 - Train Accuracy:  0.687, Validation Accuracy:  0.710, Loss:  0.479\n",
      "Epoch   3 Batch  131/269 - Train Accuracy:  0.692, Validation Accuracy:  0.703, Loss:  0.471\n",
      "Epoch   3 Batch  132/269 - Train Accuracy:  0.711, Validation Accuracy:  0.710, Loss:  0.468\n",
      "Epoch   3 Batch  133/269 - Train Accuracy:  0.722, Validation Accuracy:  0.718, Loss:  0.442\n",
      "Epoch   3 Batch  134/269 - Train Accuracy:  0.694, Validation Accuracy:  0.713, Loss:  0.463\n",
      "Epoch   3 Batch  135/269 - Train Accuracy:  0.685, Validation Accuracy:  0.718, Loss:  0.489\n",
      "Epoch   3 Batch  136/269 - Train Accuracy:  0.694, Validation Accuracy:  0.720, Loss:  0.486\n",
      "Epoch   3 Batch  137/269 - Train Accuracy:  0.696, Validation Accuracy:  0.710, Loss:  0.483\n",
      "Epoch   3 Batch  138/269 - Train Accuracy:  0.705, Validation Accuracy:  0.710, Loss:  0.468\n",
      "Epoch   3 Batch  139/269 - Train Accuracy:  0.733, Validation Accuracy:  0.715, Loss:  0.438\n",
      "Epoch   3 Batch  140/269 - Train Accuracy:  0.699, Validation Accuracy:  0.713, Loss:  0.465\n",
      "Epoch   3 Batch  141/269 - Train Accuracy:  0.692, Validation Accuracy:  0.713, Loss:  0.472\n",
      "Epoch   3 Batch  142/269 - Train Accuracy:  0.713, Validation Accuracy:  0.717, Loss:  0.445\n",
      "Epoch   3 Batch  143/269 - Train Accuracy:  0.703, Validation Accuracy:  0.716, Loss:  0.454\n",
      "Epoch   3 Batch  144/269 - Train Accuracy:  0.716, Validation Accuracy:  0.712, Loss:  0.431\n",
      "Epoch   3 Batch  145/269 - Train Accuracy:  0.716, Validation Accuracy:  0.721, Loss:  0.447\n",
      "Epoch   3 Batch  146/269 - Train Accuracy:  0.719, Validation Accuracy:  0.722, Loss:  0.440\n",
      "Epoch   3 Batch  147/269 - Train Accuracy:  0.723, Validation Accuracy:  0.717, Loss:  0.429\n",
      "Epoch   3 Batch  148/269 - Train Accuracy:  0.704, Validation Accuracy:  0.718, Loss:  0.454\n",
      "Epoch   3 Batch  149/269 - Train Accuracy:  0.709, Validation Accuracy:  0.723, Loss:  0.454\n",
      "Epoch   3 Batch  150/269 - Train Accuracy:  0.726, Validation Accuracy:  0.720, Loss:  0.445\n",
      "Epoch   3 Batch  151/269 - Train Accuracy:  0.734, Validation Accuracy:  0.718, Loss:  0.427\n",
      "Epoch   3 Batch  152/269 - Train Accuracy:  0.714, Validation Accuracy:  0.717, Loss:  0.448\n",
      "Epoch   3 Batch  153/269 - Train Accuracy:  0.714, Validation Accuracy:  0.724, Loss:  0.442\n",
      "Epoch   3 Batch  154/269 - Train Accuracy:  0.707, Validation Accuracy:  0.721, Loss:  0.458\n",
      "Epoch   3 Batch  155/269 - Train Accuracy:  0.729, Validation Accuracy:  0.718, Loss:  0.423\n",
      "Epoch   3 Batch  156/269 - Train Accuracy:  0.707, Validation Accuracy:  0.723, Loss:  0.458\n",
      "Epoch   3 Batch  157/269 - Train Accuracy:  0.714, Validation Accuracy:  0.724, Loss:  0.441\n",
      "Epoch   3 Batch  158/269 - Train Accuracy:  0.720, Validation Accuracy:  0.722, Loss:  0.440\n",
      "Epoch   3 Batch  159/269 - Train Accuracy:  0.717, Validation Accuracy:  0.722, Loss:  0.443\n",
      "Epoch   3 Batch  160/269 - Train Accuracy:  0.722, Validation Accuracy:  0.720, Loss:  0.438\n",
      "Epoch   3 Batch  161/269 - Train Accuracy:  0.706, Validation Accuracy:  0.724, Loss:  0.444\n",
      "Epoch   3 Batch  162/269 - Train Accuracy:  0.718, Validation Accuracy:  0.726, Loss:  0.434\n",
      "Epoch   3 Batch  163/269 - Train Accuracy:  0.716, Validation Accuracy:  0.724, Loss:  0.435\n",
      "Epoch   3 Batch  164/269 - Train Accuracy:  0.727, Validation Accuracy:  0.723, Loss:  0.434\n",
      "Epoch   3 Batch  165/269 - Train Accuracy:  0.705, Validation Accuracy:  0.724, Loss:  0.450\n",
      "Epoch   3 Batch  166/269 - Train Accuracy:  0.732, Validation Accuracy:  0.728, Loss:  0.416\n",
      "Epoch   3 Batch  167/269 - Train Accuracy:  0.713, Validation Accuracy:  0.731, Loss:  0.434\n",
      "Epoch   3 Batch  168/269 - Train Accuracy:  0.703, Validation Accuracy:  0.728, Loss:  0.441\n",
      "Epoch   3 Batch  169/269 - Train Accuracy:  0.725, Validation Accuracy:  0.729, Loss:  0.433\n",
      "Epoch   3 Batch  170/269 - Train Accuracy:  0.718, Validation Accuracy:  0.732, Loss:  0.431\n",
      "Epoch   3 Batch  171/269 - Train Accuracy:  0.713, Validation Accuracy:  0.727, Loss:  0.452\n",
      "Epoch   3 Batch  172/269 - Train Accuracy:  0.721, Validation Accuracy:  0.737, Loss:  0.441\n",
      "Epoch   3 Batch  173/269 - Train Accuracy:  0.730, Validation Accuracy:  0.734, Loss:  0.422\n",
      "Epoch   3 Batch  174/269 - Train Accuracy:  0.716, Validation Accuracy:  0.722, Loss:  0.432\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   3 Batch  175/269 - Train Accuracy:  0.733, Validation Accuracy:  0.729, Loss:  0.446\n",
      "Epoch   3 Batch  176/269 - Train Accuracy:  0.701, Validation Accuracy:  0.731, Loss:  0.458\n",
      "Epoch   3 Batch  177/269 - Train Accuracy:  0.732, Validation Accuracy:  0.733, Loss:  0.409\n",
      "Epoch   3 Batch  178/269 - Train Accuracy:  0.721, Validation Accuracy:  0.730, Loss:  0.441\n",
      "Epoch   3 Batch  179/269 - Train Accuracy:  0.725, Validation Accuracy:  0.725, Loss:  0.429\n",
      "Epoch   3 Batch  180/269 - Train Accuracy:  0.734, Validation Accuracy:  0.732, Loss:  0.422\n",
      "Epoch   3 Batch  181/269 - Train Accuracy:  0.727, Validation Accuracy:  0.731, Loss:  0.426\n",
      "Epoch   3 Batch  182/269 - Train Accuracy:  0.736, Validation Accuracy:  0.733, Loss:  0.427\n",
      "Epoch   3 Batch  183/269 - Train Accuracy:  0.764, Validation Accuracy:  0.735, Loss:  0.374\n",
      "Epoch   3 Batch  184/269 - Train Accuracy:  0.711, Validation Accuracy:  0.731, Loss:  0.437\n",
      "Epoch   3 Batch  185/269 - Train Accuracy:  0.727, Validation Accuracy:  0.731, Loss:  0.420\n",
      "Epoch   3 Batch  186/269 - Train Accuracy:  0.716, Validation Accuracy:  0.730, Loss:  0.436\n",
      "Epoch   3 Batch  187/269 - Train Accuracy:  0.733, Validation Accuracy:  0.726, Loss:  0.411\n",
      "Epoch   3 Batch  188/269 - Train Accuracy:  0.741, Validation Accuracy:  0.729, Loss:  0.410\n",
      "Epoch   3 Batch  189/269 - Train Accuracy:  0.738, Validation Accuracy:  0.732, Loss:  0.409\n",
      "Epoch   3 Batch  190/269 - Train Accuracy:  0.733, Validation Accuracy:  0.731, Loss:  0.409\n",
      "Epoch   3 Batch  191/269 - Train Accuracy:  0.735, Validation Accuracy:  0.739, Loss:  0.419\n",
      "Epoch   3 Batch  192/269 - Train Accuracy:  0.732, Validation Accuracy:  0.737, Loss:  0.419\n",
      "Epoch   3 Batch  193/269 - Train Accuracy:  0.742, Validation Accuracy:  0.736, Loss:  0.415\n",
      "Epoch   3 Batch  194/269 - Train Accuracy:  0.747, Validation Accuracy:  0.737, Loss:  0.421\n",
      "Epoch   3 Batch  195/269 - Train Accuracy:  0.733, Validation Accuracy:  0.738, Loss:  0.414\n",
      "Epoch   3 Batch  196/269 - Train Accuracy:  0.735, Validation Accuracy:  0.732, Loss:  0.411\n",
      "Epoch   3 Batch  197/269 - Train Accuracy:  0.712, Validation Accuracy:  0.734, Loss:  0.433\n",
      "Epoch   3 Batch  198/269 - Train Accuracy:  0.715, Validation Accuracy:  0.739, Loss:  0.438\n",
      "Epoch   3 Batch  199/269 - Train Accuracy:  0.720, Validation Accuracy:  0.742, Loss:  0.424\n",
      "Epoch   3 Batch  200/269 - Train Accuracy:  0.731, Validation Accuracy:  0.748, Loss:  0.431\n",
      "Epoch   3 Batch  201/269 - Train Accuracy:  0.730, Validation Accuracy:  0.744, Loss:  0.414\n",
      "Epoch   3 Batch  202/269 - Train Accuracy:  0.733, Validation Accuracy:  0.741, Loss:  0.413\n",
      "Epoch   3 Batch  203/269 - Train Accuracy:  0.731, Validation Accuracy:  0.740, Loss:  0.440\n",
      "Epoch   3 Batch  204/269 - Train Accuracy:  0.718, Validation Accuracy:  0.732, Loss:  0.434\n",
      "Epoch   3 Batch  205/269 - Train Accuracy:  0.734, Validation Accuracy:  0.725, Loss:  0.403\n",
      "Epoch   3 Batch  206/269 - Train Accuracy:  0.715, Validation Accuracy:  0.739, Loss:  0.432\n",
      "Epoch   3 Batch  207/269 - Train Accuracy:  0.738, Validation Accuracy:  0.746, Loss:  0.401\n",
      "Epoch   3 Batch  208/269 - Train Accuracy:  0.732, Validation Accuracy:  0.739, Loss:  0.425\n",
      "Epoch   3 Batch  209/269 - Train Accuracy:  0.749, Validation Accuracy:  0.742, Loss:  0.417\n",
      "Epoch   3 Batch  210/269 - Train Accuracy:  0.739, Validation Accuracy:  0.746, Loss:  0.403\n",
      "Epoch   3 Batch  211/269 - Train Accuracy:  0.735, Validation Accuracy:  0.740, Loss:  0.412\n",
      "Epoch   3 Batch  212/269 - Train Accuracy:  0.743, Validation Accuracy:  0.739, Loss:  0.406\n",
      "Epoch   3 Batch  213/269 - Train Accuracy:  0.746, Validation Accuracy:  0.740, Loss:  0.410\n",
      "Epoch   3 Batch  214/269 - Train Accuracy:  0.736, Validation Accuracy:  0.743, Loss:  0.407\n",
      "Epoch   3 Batch  215/269 - Train Accuracy:  0.758, Validation Accuracy:  0.740, Loss:  0.378\n",
      "Epoch   3 Batch  216/269 - Train Accuracy:  0.724, Validation Accuracy:  0.743, Loss:  0.427\n",
      "Epoch   3 Batch  217/269 - Train Accuracy:  0.714, Validation Accuracy:  0.745, Loss:  0.422\n",
      "Epoch   3 Batch  218/269 - Train Accuracy:  0.726, Validation Accuracy:  0.736, Loss:  0.419\n",
      "Epoch   3 Batch  219/269 - Train Accuracy:  0.753, Validation Accuracy:  0.740, Loss:  0.423\n",
      "Epoch   3 Batch  220/269 - Train Accuracy:  0.744, Validation Accuracy:  0.745, Loss:  0.384\n",
      "Epoch   3 Batch  221/269 - Train Accuracy:  0.750, Validation Accuracy:  0.737, Loss:  0.403\n",
      "Epoch   3 Batch  222/269 - Train Accuracy:  0.763, Validation Accuracy:  0.746, Loss:  0.389\n",
      "Epoch   3 Batch  223/269 - Train Accuracy:  0.724, Validation Accuracy:  0.750, Loss:  0.393\n",
      "Epoch   3 Batch  224/269 - Train Accuracy:  0.744, Validation Accuracy:  0.748, Loss:  0.411\n",
      "Epoch   3 Batch  225/269 - Train Accuracy:  0.730, Validation Accuracy:  0.749, Loss:  0.404\n",
      "Epoch   3 Batch  226/269 - Train Accuracy:  0.754, Validation Accuracy:  0.748, Loss:  0.400\n",
      "Epoch   3 Batch  227/269 - Train Accuracy:  0.773, Validation Accuracy:  0.751, Loss:  0.357\n",
      "Epoch   3 Batch  228/269 - Train Accuracy:  0.730, Validation Accuracy:  0.753, Loss:  0.399\n",
      "Epoch   3 Batch  229/269 - Train Accuracy:  0.744, Validation Accuracy:  0.748, Loss:  0.393\n",
      "Epoch   3 Batch  230/269 - Train Accuracy:  0.742, Validation Accuracy:  0.743, Loss:  0.396\n",
      "Epoch   3 Batch  231/269 - Train Accuracy:  0.731, Validation Accuracy:  0.747, Loss:  0.414\n",
      "Epoch   3 Batch  232/269 - Train Accuracy:  0.731, Validation Accuracy:  0.744, Loss:  0.413\n",
      "Epoch   3 Batch  233/269 - Train Accuracy:  0.753, Validation Accuracy:  0.748, Loss:  0.401\n",
      "Epoch   3 Batch  234/269 - Train Accuracy:  0.746, Validation Accuracy:  0.746, Loss:  0.391\n",
      "Epoch   3 Batch  235/269 - Train Accuracy:  0.756, Validation Accuracy:  0.751, Loss:  0.385\n",
      "Epoch   3 Batch  236/269 - Train Accuracy:  0.735, Validation Accuracy:  0.746, Loss:  0.387\n",
      "Epoch   3 Batch  237/269 - Train Accuracy:  0.745, Validation Accuracy:  0.745, Loss:  0.394\n",
      "Epoch   3 Batch  238/269 - Train Accuracy:  0.761, Validation Accuracy:  0.739, Loss:  0.387\n",
      "Epoch   3 Batch  239/269 - Train Accuracy:  0.755, Validation Accuracy:  0.748, Loss:  0.390\n",
      "Epoch   3 Batch  240/269 - Train Accuracy:  0.763, Validation Accuracy:  0.750, Loss:  0.356\n",
      "Epoch   3 Batch  241/269 - Train Accuracy:  0.749, Validation Accuracy:  0.750, Loss:  0.397\n",
      "Epoch   3 Batch  242/269 - Train Accuracy:  0.742, Validation Accuracy:  0.750, Loss:  0.385\n",
      "Epoch   3 Batch  243/269 - Train Accuracy:  0.758, Validation Accuracy:  0.756, Loss:  0.372\n",
      "Epoch   3 Batch  244/269 - Train Accuracy:  0.728, Validation Accuracy:  0.753, Loss:  0.393\n",
      "Epoch   3 Batch  245/269 - Train Accuracy:  0.742, Validation Accuracy:  0.756, Loss:  0.411\n",
      "Epoch   3 Batch  246/269 - Train Accuracy:  0.748, Validation Accuracy:  0.755, Loss:  0.383\n",
      "Epoch   3 Batch  247/269 - Train Accuracy:  0.743, Validation Accuracy:  0.747, Loss:  0.397\n",
      "Epoch   3 Batch  248/269 - Train Accuracy:  0.765, Validation Accuracy:  0.746, Loss:  0.384\n",
      "Epoch   3 Batch  249/269 - Train Accuracy:  0.764, Validation Accuracy:  0.751, Loss:  0.368\n",
      "Epoch   3 Batch  250/269 - Train Accuracy:  0.747, Validation Accuracy:  0.751, Loss:  0.395\n",
      "Epoch   3 Batch  251/269 - Train Accuracy:  0.769, Validation Accuracy:  0.753, Loss:  0.377\n",
      "Epoch   3 Batch  252/269 - Train Accuracy:  0.745, Validation Accuracy:  0.755, Loss:  0.385\n",
      "Epoch   3 Batch  253/269 - Train Accuracy:  0.727, Validation Accuracy:  0.754, Loss:  0.392\n",
      "Epoch   3 Batch  254/269 - Train Accuracy:  0.745, Validation Accuracy:  0.754, Loss:  0.378\n",
      "Epoch   3 Batch  255/269 - Train Accuracy:  0.757, Validation Accuracy:  0.757, Loss:  0.374\n",
      "Epoch   3 Batch  256/269 - Train Accuracy:  0.730, Validation Accuracy:  0.754, Loss:  0.386\n",
      "Epoch   3 Batch  257/269 - Train Accuracy:  0.736, Validation Accuracy:  0.753, Loss:  0.389\n",
      "Epoch   3 Batch  258/269 - Train Accuracy:  0.752, Validation Accuracy:  0.748, Loss:  0.381\n",
      "Epoch   3 Batch  259/269 - Train Accuracy:  0.738, Validation Accuracy:  0.747, Loss:  0.378\n",
      "Epoch   3 Batch  260/269 - Train Accuracy:  0.733, Validation Accuracy:  0.754, Loss:  0.395\n",
      "Epoch   3 Batch  261/269 - Train Accuracy:  0.733, Validation Accuracy:  0.756, Loss:  0.398\n",
      "Epoch   3 Batch  262/269 - Train Accuracy:  0.756, Validation Accuracy:  0.753, Loss:  0.380\n",
      "Epoch   3 Batch  263/269 - Train Accuracy:  0.761, Validation Accuracy:  0.751, Loss:  0.395\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   3 Batch  264/269 - Train Accuracy:  0.744, Validation Accuracy:  0.752, Loss:  0.394\n",
      "Epoch   3 Batch  265/269 - Train Accuracy:  0.732, Validation Accuracy:  0.761, Loss:  0.393\n",
      "Epoch   3 Batch  266/269 - Train Accuracy:  0.763, Validation Accuracy:  0.766, Loss:  0.370\n",
      "Epoch   3 Batch  267/269 - Train Accuracy:  0.759, Validation Accuracy:  0.759, Loss:  0.379\n",
      "Epoch   4 Batch    0/269 - Train Accuracy:  0.745, Validation Accuracy:  0.751, Loss:  0.397\n",
      "Epoch   4 Batch    1/269 - Train Accuracy:  0.752, Validation Accuracy:  0.756, Loss:  0.387\n",
      "Epoch   4 Batch    2/269 - Train Accuracy:  0.735, Validation Accuracy:  0.751, Loss:  0.378\n",
      "Epoch   4 Batch    3/269 - Train Accuracy:  0.763, Validation Accuracy:  0.757, Loss:  0.383\n",
      "Epoch   4 Batch    4/269 - Train Accuracy:  0.734, Validation Accuracy:  0.747, Loss:  0.391\n",
      "Epoch   4 Batch    5/269 - Train Accuracy:  0.718, Validation Accuracy:  0.745, Loss:  0.386\n",
      "Epoch   4 Batch    6/269 - Train Accuracy:  0.756, Validation Accuracy:  0.751, Loss:  0.365\n",
      "Epoch   4 Batch    7/269 - Train Accuracy:  0.754, Validation Accuracy:  0.753, Loss:  0.366\n",
      "Epoch   4 Batch    8/269 - Train Accuracy:  0.737, Validation Accuracy:  0.749, Loss:  0.388\n",
      "Epoch   4 Batch    9/269 - Train Accuracy:  0.757, Validation Accuracy:  0.751, Loss:  0.380\n",
      "Epoch   4 Batch   10/269 - Train Accuracy:  0.764, Validation Accuracy:  0.763, Loss:  0.380\n",
      "Epoch   4 Batch   11/269 - Train Accuracy:  0.771, Validation Accuracy:  0.762, Loss:  0.375\n",
      "Epoch   4 Batch   12/269 - Train Accuracy:  0.744, Validation Accuracy:  0.743, Loss:  0.393\n",
      "Epoch   4 Batch   13/269 - Train Accuracy:  0.770, Validation Accuracy:  0.751, Loss:  0.347\n",
      "Epoch   4 Batch   14/269 - Train Accuracy:  0.743, Validation Accuracy:  0.760, Loss:  0.368\n",
      "Epoch   4 Batch   15/269 - Train Accuracy:  0.762, Validation Accuracy:  0.764, Loss:  0.357\n",
      "Epoch   4 Batch   16/269 - Train Accuracy:  0.762, Validation Accuracy:  0.765, Loss:  0.369\n",
      "Epoch   4 Batch   17/269 - Train Accuracy:  0.768, Validation Accuracy:  0.765, Loss:  0.356\n",
      "Epoch   4 Batch   18/269 - Train Accuracy:  0.763, Validation Accuracy:  0.763, Loss:  0.371\n",
      "Epoch   4 Batch   19/269 - Train Accuracy:  0.778, Validation Accuracy:  0.765, Loss:  0.338\n",
      "Epoch   4 Batch   20/269 - Train Accuracy:  0.775, Validation Accuracy:  0.765, Loss:  0.375\n",
      "Epoch   4 Batch   21/269 - Train Accuracy:  0.746, Validation Accuracy:  0.767, Loss:  0.388\n",
      "Epoch   4 Batch   22/269 - Train Accuracy:  0.785, Validation Accuracy:  0.767, Loss:  0.349\n",
      "Epoch   4 Batch   23/269 - Train Accuracy:  0.767, Validation Accuracy:  0.770, Loss:  0.356\n",
      "Epoch   4 Batch   24/269 - Train Accuracy:  0.774, Validation Accuracy:  0.769, Loss:  0.378\n",
      "Epoch   4 Batch   25/269 - Train Accuracy:  0.757, Validation Accuracy:  0.757, Loss:  0.381\n",
      "Epoch   4 Batch   26/269 - Train Accuracy:  0.768, Validation Accuracy:  0.769, Loss:  0.341\n",
      "Epoch   4 Batch   27/269 - Train Accuracy:  0.764, Validation Accuracy:  0.771, Loss:  0.353\n",
      "Epoch   4 Batch   28/269 - Train Accuracy:  0.728, Validation Accuracy:  0.764, Loss:  0.387\n",
      "Epoch   4 Batch   29/269 - Train Accuracy:  0.752, Validation Accuracy:  0.761, Loss:  0.376\n",
      "Epoch   4 Batch   30/269 - Train Accuracy:  0.764, Validation Accuracy:  0.767, Loss:  0.360\n",
      "Epoch   4 Batch   31/269 - Train Accuracy:  0.775, Validation Accuracy:  0.761, Loss:  0.348\n",
      "Epoch   4 Batch   32/269 - Train Accuracy:  0.764, Validation Accuracy:  0.761, Loss:  0.352\n",
      "Epoch   4 Batch   33/269 - Train Accuracy:  0.772, Validation Accuracy:  0.754, Loss:  0.344\n",
      "Epoch   4 Batch   34/269 - Train Accuracy:  0.766, Validation Accuracy:  0.756, Loss:  0.350\n",
      "Epoch   4 Batch   35/269 - Train Accuracy:  0.768, Validation Accuracy:  0.764, Loss:  0.366\n",
      "Epoch   4 Batch   36/269 - Train Accuracy:  0.748, Validation Accuracy:  0.763, Loss:  0.356\n",
      "Epoch   4 Batch   37/269 - Train Accuracy:  0.760, Validation Accuracy:  0.755, Loss:  0.347\n",
      "Epoch   4 Batch   38/269 - Train Accuracy:  0.760, Validation Accuracy:  0.760, Loss:  0.352\n",
      "Epoch   4 Batch   39/269 - Train Accuracy:  0.770, Validation Accuracy:  0.767, Loss:  0.352\n",
      "Epoch   4 Batch   40/269 - Train Accuracy:  0.741, Validation Accuracy:  0.765, Loss:  0.366\n",
      "Epoch   4 Batch   41/269 - Train Accuracy:  0.757, Validation Accuracy:  0.770, Loss:  0.356\n",
      "Epoch   4 Batch   42/269 - Train Accuracy:  0.776, Validation Accuracy:  0.764, Loss:  0.331\n",
      "Epoch   4 Batch   43/269 - Train Accuracy:  0.759, Validation Accuracy:  0.776, Loss:  0.359\n",
      "Epoch   4 Batch   44/269 - Train Accuracy:  0.765, Validation Accuracy:  0.779, Loss:  0.350\n",
      "Epoch   4 Batch   45/269 - Train Accuracy:  0.753, Validation Accuracy:  0.773, Loss:  0.366\n",
      "Epoch   4 Batch   46/269 - Train Accuracy:  0.770, Validation Accuracy:  0.763, Loss:  0.362\n",
      "Epoch   4 Batch   47/269 - Train Accuracy:  0.779, Validation Accuracy:  0.773, Loss:  0.324\n",
      "Epoch   4 Batch   48/269 - Train Accuracy:  0.785, Validation Accuracy:  0.777, Loss:  0.340\n",
      "Epoch   4 Batch   49/269 - Train Accuracy:  0.754, Validation Accuracy:  0.778, Loss:  0.353\n",
      "Epoch   4 Batch   50/269 - Train Accuracy:  0.747, Validation Accuracy:  0.773, Loss:  0.364\n",
      "Epoch   4 Batch   51/269 - Train Accuracy:  0.772, Validation Accuracy:  0.769, Loss:  0.340\n",
      "Epoch   4 Batch   52/269 - Train Accuracy:  0.749, Validation Accuracy:  0.770, Loss:  0.329\n",
      "Epoch   4 Batch   53/269 - Train Accuracy:  0.768, Validation Accuracy:  0.773, Loss:  0.362\n",
      "Epoch   4 Batch   54/269 - Train Accuracy:  0.774, Validation Accuracy:  0.771, Loss:  0.357\n",
      "Epoch   4 Batch   55/269 - Train Accuracy:  0.786, Validation Accuracy:  0.774, Loss:  0.337\n",
      "Epoch   4 Batch   56/269 - Train Accuracy:  0.757, Validation Accuracy:  0.774, Loss:  0.342\n",
      "Epoch   4 Batch   57/269 - Train Accuracy:  0.763, Validation Accuracy:  0.773, Loss:  0.352\n",
      "Epoch   4 Batch   58/269 - Train Accuracy:  0.784, Validation Accuracy:  0.777, Loss:  0.332\n",
      "Epoch   4 Batch   59/269 - Train Accuracy:  0.799, Validation Accuracy:  0.779, Loss:  0.319\n",
      "Epoch   4 Batch   60/269 - Train Accuracy:  0.776, Validation Accuracy:  0.771, Loss:  0.323\n",
      "Epoch   4 Batch   61/269 - Train Accuracy:  0.783, Validation Accuracy:  0.773, Loss:  0.315\n",
      "Epoch   4 Batch   62/269 - Train Accuracy:  0.781, Validation Accuracy:  0.768, Loss:  0.327\n",
      "Epoch   4 Batch   63/269 - Train Accuracy:  0.780, Validation Accuracy:  0.771, Loss:  0.345\n",
      "Epoch   4 Batch   64/269 - Train Accuracy:  0.779, Validation Accuracy:  0.774, Loss:  0.328\n",
      "Epoch   4 Batch   65/269 - Train Accuracy:  0.768, Validation Accuracy:  0.774, Loss:  0.335\n",
      "Epoch   4 Batch   66/269 - Train Accuracy:  0.786, Validation Accuracy:  0.769, Loss:  0.326\n",
      "Epoch   4 Batch   67/269 - Train Accuracy:  0.771, Validation Accuracy:  0.775, Loss:  0.340\n",
      "Epoch   4 Batch   68/269 - Train Accuracy:  0.742, Validation Accuracy:  0.771, Loss:  0.340\n",
      "Epoch   4 Batch   69/269 - Train Accuracy:  0.753, Validation Accuracy:  0.779, Loss:  0.367\n",
      "Epoch   4 Batch   70/269 - Train Accuracy:  0.788, Validation Accuracy:  0.780, Loss:  0.335\n",
      "Epoch   4 Batch   71/269 - Train Accuracy:  0.772, Validation Accuracy:  0.782, Loss:  0.356\n",
      "Epoch   4 Batch   72/269 - Train Accuracy:  0.779, Validation Accuracy:  0.770, Loss:  0.331\n",
      "Epoch   4 Batch   73/269 - Train Accuracy:  0.752, Validation Accuracy:  0.773, Loss:  0.340\n",
      "Epoch   4 Batch   74/269 - Train Accuracy:  0.776, Validation Accuracy:  0.780, Loss:  0.340\n",
      "Epoch   4 Batch   75/269 - Train Accuracy:  0.768, Validation Accuracy:  0.777, Loss:  0.331\n",
      "Epoch   4 Batch   76/269 - Train Accuracy:  0.753, Validation Accuracy:  0.778, Loss:  0.334\n",
      "Epoch   4 Batch   77/269 - Train Accuracy:  0.782, Validation Accuracy:  0.777, Loss:  0.325\n",
      "Epoch   4 Batch   78/269 - Train Accuracy:  0.784, Validation Accuracy:  0.774, Loss:  0.328\n",
      "Epoch   4 Batch   79/269 - Train Accuracy:  0.768, Validation Accuracy:  0.777, Loss:  0.328\n",
      "Epoch   4 Batch   80/269 - Train Accuracy:  0.781, Validation Accuracy:  0.778, Loss:  0.326\n",
      "Epoch   4 Batch   81/269 - Train Accuracy:  0.768, Validation Accuracy:  0.772, Loss:  0.341\n",
      "Epoch   4 Batch   82/269 - Train Accuracy:  0.792, Validation Accuracy:  0.770, Loss:  0.318\n",
      "Epoch   4 Batch   83/269 - Train Accuracy:  0.764, Validation Accuracy:  0.776, Loss:  0.335\n",
      "Epoch   4 Batch   84/269 - Train Accuracy:  0.789, Validation Accuracy:  0.780, Loss:  0.322\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   4 Batch   85/269 - Train Accuracy:  0.793, Validation Accuracy:  0.781, Loss:  0.324\n",
      "Epoch   4 Batch   86/269 - Train Accuracy:  0.780, Validation Accuracy:  0.780, Loss:  0.322\n",
      "Epoch   4 Batch   87/269 - Train Accuracy:  0.761, Validation Accuracy:  0.788, Loss:  0.350\n",
      "Epoch   4 Batch   88/269 - Train Accuracy:  0.763, Validation Accuracy:  0.782, Loss:  0.327\n",
      "Epoch   4 Batch   89/269 - Train Accuracy:  0.787, Validation Accuracy:  0.779, Loss:  0.322\n",
      "Epoch   4 Batch   90/269 - Train Accuracy:  0.752, Validation Accuracy:  0.777, Loss:  0.336\n",
      "Epoch   4 Batch   91/269 - Train Accuracy:  0.787, Validation Accuracy:  0.780, Loss:  0.312\n",
      "Epoch   4 Batch   92/269 - Train Accuracy:  0.796, Validation Accuracy:  0.779, Loss:  0.314\n",
      "Epoch   4 Batch   93/269 - Train Accuracy:  0.778, Validation Accuracy:  0.779, Loss:  0.308\n",
      "Epoch   4 Batch   94/269 - Train Accuracy:  0.783, Validation Accuracy:  0.785, Loss:  0.326\n",
      "Epoch   4 Batch   95/269 - Train Accuracy:  0.783, Validation Accuracy:  0.779, Loss:  0.314\n",
      "Epoch   4 Batch   96/269 - Train Accuracy:  0.757, Validation Accuracy:  0.773, Loss:  0.321\n",
      "Epoch   4 Batch   97/269 - Train Accuracy:  0.769, Validation Accuracy:  0.779, Loss:  0.320\n",
      "Epoch   4 Batch   98/269 - Train Accuracy:  0.794, Validation Accuracy:  0.777, Loss:  0.318\n",
      "Epoch   4 Batch   99/269 - Train Accuracy:  0.781, Validation Accuracy:  0.784, Loss:  0.329\n",
      "Epoch   4 Batch  100/269 - Train Accuracy:  0.800, Validation Accuracy:  0.788, Loss:  0.312\n",
      "Epoch   4 Batch  101/269 - Train Accuracy:  0.763, Validation Accuracy:  0.785, Loss:  0.336\n",
      "Epoch   4 Batch  102/269 - Train Accuracy:  0.780, Validation Accuracy:  0.790, Loss:  0.317\n",
      "Epoch   4 Batch  103/269 - Train Accuracy:  0.796, Validation Accuracy:  0.788, Loss:  0.313\n",
      "Epoch   4 Batch  104/269 - Train Accuracy:  0.789, Validation Accuracy:  0.788, Loss:  0.313\n",
      "Epoch   4 Batch  105/269 - Train Accuracy:  0.773, Validation Accuracy:  0.786, Loss:  0.316\n",
      "Epoch   4 Batch  106/269 - Train Accuracy:  0.772, Validation Accuracy:  0.789, Loss:  0.307\n",
      "Epoch   4 Batch  107/269 - Train Accuracy:  0.789, Validation Accuracy:  0.787, Loss:  0.328\n",
      "Epoch   4 Batch  108/269 - Train Accuracy:  0.784, Validation Accuracy:  0.786, Loss:  0.318\n",
      "Epoch   4 Batch  109/269 - Train Accuracy:  0.774, Validation Accuracy:  0.785, Loss:  0.318\n",
      "Epoch   4 Batch  110/269 - Train Accuracy:  0.787, Validation Accuracy:  0.786, Loss:  0.313\n",
      "Epoch   4 Batch  111/269 - Train Accuracy:  0.770, Validation Accuracy:  0.785, Loss:  0.331\n",
      "Epoch   4 Batch  112/269 - Train Accuracy:  0.776, Validation Accuracy:  0.787, Loss:  0.309\n",
      "Epoch   4 Batch  113/269 - Train Accuracy:  0.777, Validation Accuracy:  0.778, Loss:  0.301\n",
      "Epoch   4 Batch  114/269 - Train Accuracy:  0.787, Validation Accuracy:  0.785, Loss:  0.311\n",
      "Epoch   4 Batch  115/269 - Train Accuracy:  0.778, Validation Accuracy:  0.788, Loss:  0.323\n",
      "Epoch   4 Batch  116/269 - Train Accuracy:  0.803, Validation Accuracy:  0.788, Loss:  0.319\n",
      "Epoch   4 Batch  117/269 - Train Accuracy:  0.788, Validation Accuracy:  0.779, Loss:  0.308\n",
      "Epoch   4 Batch  118/269 - Train Accuracy:  0.803, Validation Accuracy:  0.778, Loss:  0.300\n",
      "Epoch   4 Batch  119/269 - Train Accuracy:  0.779, Validation Accuracy:  0.789, Loss:  0.326\n",
      "Epoch   4 Batch  120/269 - Train Accuracy:  0.774, Validation Accuracy:  0.790, Loss:  0.317\n",
      "Epoch   4 Batch  121/269 - Train Accuracy:  0.793, Validation Accuracy:  0.789, Loss:  0.304\n",
      "Epoch   4 Batch  122/269 - Train Accuracy:  0.794, Validation Accuracy:  0.787, Loss:  0.300\n",
      "Epoch   4 Batch  123/269 - Train Accuracy:  0.782, Validation Accuracy:  0.789, Loss:  0.314\n",
      "Epoch   4 Batch  124/269 - Train Accuracy:  0.790, Validation Accuracy:  0.786, Loss:  0.301\n",
      "Epoch   4 Batch  125/269 - Train Accuracy:  0.800, Validation Accuracy:  0.789, Loss:  0.299\n",
      "Epoch   4 Batch  126/269 - Train Accuracy:  0.780, Validation Accuracy:  0.778, Loss:  0.304\n",
      "Epoch   4 Batch  127/269 - Train Accuracy:  0.776, Validation Accuracy:  0.789, Loss:  0.313\n",
      "Epoch   4 Batch  128/269 - Train Accuracy:  0.797, Validation Accuracy:  0.787, Loss:  0.303\n",
      "Epoch   4 Batch  129/269 - Train Accuracy:  0.787, Validation Accuracy:  0.789, Loss:  0.300\n",
      "Epoch   4 Batch  130/269 - Train Accuracy:  0.782, Validation Accuracy:  0.791, Loss:  0.316\n",
      "Epoch   4 Batch  131/269 - Train Accuracy:  0.770, Validation Accuracy:  0.790, Loss:  0.312\n",
      "Epoch   4 Batch  132/269 - Train Accuracy:  0.775, Validation Accuracy:  0.789, Loss:  0.310\n",
      "Epoch   4 Batch  133/269 - Train Accuracy:  0.817, Validation Accuracy:  0.788, Loss:  0.288\n",
      "Epoch   4 Batch  134/269 - Train Accuracy:  0.780, Validation Accuracy:  0.788, Loss:  0.305\n",
      "Epoch   4 Batch  135/269 - Train Accuracy:  0.779, Validation Accuracy:  0.790, Loss:  0.321\n",
      "Epoch   4 Batch  136/269 - Train Accuracy:  0.770, Validation Accuracy:  0.791, Loss:  0.320\n",
      "Epoch   4 Batch  137/269 - Train Accuracy:  0.785, Validation Accuracy:  0.789, Loss:  0.319\n",
      "Epoch   4 Batch  138/269 - Train Accuracy:  0.799, Validation Accuracy:  0.793, Loss:  0.302\n",
      "Epoch   4 Batch  139/269 - Train Accuracy:  0.799, Validation Accuracy:  0.797, Loss:  0.288\n",
      "Epoch   4 Batch  140/269 - Train Accuracy:  0.798, Validation Accuracy:  0.797, Loss:  0.307\n",
      "Epoch   4 Batch  141/269 - Train Accuracy:  0.789, Validation Accuracy:  0.798, Loss:  0.308\n",
      "Epoch   4 Batch  142/269 - Train Accuracy:  0.785, Validation Accuracy:  0.804, Loss:  0.293\n",
      "Epoch   4 Batch  143/269 - Train Accuracy:  0.789, Validation Accuracy:  0.802, Loss:  0.295\n",
      "Epoch   4 Batch  144/269 - Train Accuracy:  0.806, Validation Accuracy:  0.797, Loss:  0.278\n",
      "Epoch   4 Batch  145/269 - Train Accuracy:  0.801, Validation Accuracy:  0.798, Loss:  0.292\n",
      "Epoch   4 Batch  146/269 - Train Accuracy:  0.791, Validation Accuracy:  0.795, Loss:  0.291\n",
      "Epoch   4 Batch  147/269 - Train Accuracy:  0.806, Validation Accuracy:  0.804, Loss:  0.283\n",
      "Epoch   4 Batch  148/269 - Train Accuracy:  0.795, Validation Accuracy:  0.804, Loss:  0.303\n",
      "Epoch   4 Batch  149/269 - Train Accuracy:  0.776, Validation Accuracy:  0.798, Loss:  0.301\n",
      "Epoch   4 Batch  150/269 - Train Accuracy:  0.812, Validation Accuracy:  0.789, Loss:  0.287\n",
      "Epoch   4 Batch  151/269 - Train Accuracy:  0.815, Validation Accuracy:  0.799, Loss:  0.279\n",
      "Epoch   4 Batch  152/269 - Train Accuracy:  0.799, Validation Accuracy:  0.804, Loss:  0.293\n",
      "Epoch   4 Batch  153/269 - Train Accuracy:  0.809, Validation Accuracy:  0.799, Loss:  0.291\n",
      "Epoch   4 Batch  154/269 - Train Accuracy:  0.812, Validation Accuracy:  0.803, Loss:  0.299\n",
      "Epoch   4 Batch  155/269 - Train Accuracy:  0.803, Validation Accuracy:  0.807, Loss:  0.279\n",
      "Epoch   4 Batch  156/269 - Train Accuracy:  0.793, Validation Accuracy:  0.810, Loss:  0.298\n",
      "Epoch   4 Batch  157/269 - Train Accuracy:  0.799, Validation Accuracy:  0.809, Loss:  0.281\n",
      "Epoch   4 Batch  158/269 - Train Accuracy:  0.797, Validation Accuracy:  0.812, Loss:  0.286\n",
      "Epoch   4 Batch  159/269 - Train Accuracy:  0.795, Validation Accuracy:  0.805, Loss:  0.290\n",
      "Epoch   4 Batch  160/269 - Train Accuracy:  0.796, Validation Accuracy:  0.802, Loss:  0.289\n",
      "Epoch   4 Batch  161/269 - Train Accuracy:  0.792, Validation Accuracy:  0.802, Loss:  0.288\n",
      "Epoch   4 Batch  162/269 - Train Accuracy:  0.813, Validation Accuracy:  0.807, Loss:  0.282\n",
      "Epoch   4 Batch  163/269 - Train Accuracy:  0.808, Validation Accuracy:  0.803, Loss:  0.285\n",
      "Epoch   4 Batch  164/269 - Train Accuracy:  0.805, Validation Accuracy:  0.809, Loss:  0.283\n",
      "Epoch   4 Batch  165/269 - Train Accuracy:  0.807, Validation Accuracy:  0.805, Loss:  0.292\n",
      "Epoch   4 Batch  166/269 - Train Accuracy:  0.817, Validation Accuracy:  0.802, Loss:  0.274\n",
      "Epoch   4 Batch  167/269 - Train Accuracy:  0.811, Validation Accuracy:  0.807, Loss:  0.281\n",
      "Epoch   4 Batch  168/269 - Train Accuracy:  0.805, Validation Accuracy:  0.813, Loss:  0.288\n",
      "Epoch   4 Batch  169/269 - Train Accuracy:  0.811, Validation Accuracy:  0.808, Loss:  0.280\n",
      "Epoch   4 Batch  170/269 - Train Accuracy:  0.803, Validation Accuracy:  0.811, Loss:  0.283\n",
      "Epoch   4 Batch  171/269 - Train Accuracy:  0.801, Validation Accuracy:  0.810, Loss:  0.291\n",
      "Epoch   4 Batch  172/269 - Train Accuracy:  0.794, Validation Accuracy:  0.808, Loss:  0.292\n",
      "Epoch   4 Batch  173/269 - Train Accuracy:  0.810, Validation Accuracy:  0.805, Loss:  0.274\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   4 Batch  174/269 - Train Accuracy:  0.809, Validation Accuracy:  0.808, Loss:  0.281\n",
      "Epoch   4 Batch  175/269 - Train Accuracy:  0.811, Validation Accuracy:  0.807, Loss:  0.293\n",
      "Epoch   4 Batch  176/269 - Train Accuracy:  0.801, Validation Accuracy:  0.807, Loss:  0.300\n",
      "Epoch   4 Batch  177/269 - Train Accuracy:  0.808, Validation Accuracy:  0.809, Loss:  0.266\n",
      "Epoch   4 Batch  178/269 - Train Accuracy:  0.815, Validation Accuracy:  0.809, Loss:  0.284\n",
      "Epoch   4 Batch  179/269 - Train Accuracy:  0.794, Validation Accuracy:  0.811, Loss:  0.278\n",
      "Epoch   4 Batch  180/269 - Train Accuracy:  0.826, Validation Accuracy:  0.814, Loss:  0.274\n",
      "Epoch   4 Batch  181/269 - Train Accuracy:  0.811, Validation Accuracy:  0.817, Loss:  0.278\n",
      "Epoch   4 Batch  182/269 - Train Accuracy:  0.817, Validation Accuracy:  0.818, Loss:  0.279\n",
      "Epoch   4 Batch  183/269 - Train Accuracy:  0.834, Validation Accuracy:  0.814, Loss:  0.243\n",
      "Epoch   4 Batch  184/269 - Train Accuracy:  0.800, Validation Accuracy:  0.814, Loss:  0.283\n",
      "Epoch   4 Batch  185/269 - Train Accuracy:  0.818, Validation Accuracy:  0.809, Loss:  0.276\n",
      "Epoch   4 Batch  186/269 - Train Accuracy:  0.797, Validation Accuracy:  0.806, Loss:  0.279\n",
      "Epoch   4 Batch  187/269 - Train Accuracy:  0.813, Validation Accuracy:  0.813, Loss:  0.268\n",
      "Epoch   4 Batch  188/269 - Train Accuracy:  0.816, Validation Accuracy:  0.813, Loss:  0.265\n",
      "Epoch   4 Batch  189/269 - Train Accuracy:  0.817, Validation Accuracy:  0.814, Loss:  0.264\n",
      "Epoch   4 Batch  190/269 - Train Accuracy:  0.821, Validation Accuracy:  0.816, Loss:  0.264\n",
      "Epoch   4 Batch  191/269 - Train Accuracy:  0.797, Validation Accuracy:  0.821, Loss:  0.273\n",
      "Epoch   4 Batch  192/269 - Train Accuracy:  0.818, Validation Accuracy:  0.813, Loss:  0.274\n",
      "Epoch   4 Batch  193/269 - Train Accuracy:  0.835, Validation Accuracy:  0.816, Loss:  0.267\n",
      "Epoch   4 Batch  194/269 - Train Accuracy:  0.823, Validation Accuracy:  0.814, Loss:  0.270\n",
      "Epoch   4 Batch  195/269 - Train Accuracy:  0.820, Validation Accuracy:  0.814, Loss:  0.264\n",
      "Epoch   4 Batch  196/269 - Train Accuracy:  0.816, Validation Accuracy:  0.814, Loss:  0.263\n",
      "Epoch   4 Batch  197/269 - Train Accuracy:  0.799, Validation Accuracy:  0.819, Loss:  0.279\n",
      "Epoch   4 Batch  198/269 - Train Accuracy:  0.807, Validation Accuracy:  0.821, Loss:  0.281\n",
      "Epoch   4 Batch  199/269 - Train Accuracy:  0.802, Validation Accuracy:  0.815, Loss:  0.274\n",
      "Epoch   4 Batch  200/269 - Train Accuracy:  0.807, Validation Accuracy:  0.814, Loss:  0.279\n",
      "Epoch   4 Batch  201/269 - Train Accuracy:  0.818, Validation Accuracy:  0.821, Loss:  0.270\n",
      "Epoch   4 Batch  202/269 - Train Accuracy:  0.799, Validation Accuracy:  0.815, Loss:  0.266\n",
      "Epoch   4 Batch  203/269 - Train Accuracy:  0.799, Validation Accuracy:  0.816, Loss:  0.284\n",
      "Epoch   4 Batch  204/269 - Train Accuracy:  0.819, Validation Accuracy:  0.813, Loss:  0.279\n",
      "Epoch   4 Batch  205/269 - Train Accuracy:  0.823, Validation Accuracy:  0.817, Loss:  0.260\n",
      "Epoch   4 Batch  206/269 - Train Accuracy:  0.800, Validation Accuracy:  0.815, Loss:  0.278\n",
      "Epoch   4 Batch  207/269 - Train Accuracy:  0.813, Validation Accuracy:  0.817, Loss:  0.257\n",
      "Epoch   4 Batch  208/269 - Train Accuracy:  0.829, Validation Accuracy:  0.817, Loss:  0.275\n",
      "Epoch   4 Batch  209/269 - Train Accuracy:  0.820, Validation Accuracy:  0.813, Loss:  0.266\n",
      "Epoch   4 Batch  210/269 - Train Accuracy:  0.820, Validation Accuracy:  0.810, Loss:  0.257\n",
      "Epoch   4 Batch  211/269 - Train Accuracy:  0.827, Validation Accuracy:  0.821, Loss:  0.261\n",
      "Epoch   4 Batch  212/269 - Train Accuracy:  0.814, Validation Accuracy:  0.817, Loss:  0.261\n",
      "Epoch   4 Batch  213/269 - Train Accuracy:  0.818, Validation Accuracy:  0.817, Loss:  0.262\n",
      "Epoch   4 Batch  214/269 - Train Accuracy:  0.810, Validation Accuracy:  0.821, Loss:  0.263\n",
      "Epoch   4 Batch  215/269 - Train Accuracy:  0.830, Validation Accuracy:  0.812, Loss:  0.245\n",
      "Epoch   4 Batch  216/269 - Train Accuracy:  0.799, Validation Accuracy:  0.813, Loss:  0.279\n",
      "Epoch   4 Batch  217/269 - Train Accuracy:  0.809, Validation Accuracy:  0.811, Loss:  0.270\n",
      "Epoch   4 Batch  218/269 - Train Accuracy:  0.823, Validation Accuracy:  0.822, Loss:  0.267\n",
      "Epoch   4 Batch  219/269 - Train Accuracy:  0.835, Validation Accuracy:  0.824, Loss:  0.273\n",
      "Epoch   4 Batch  220/269 - Train Accuracy:  0.824, Validation Accuracy:  0.821, Loss:  0.244\n",
      "Epoch   4 Batch  221/269 - Train Accuracy:  0.828, Validation Accuracy:  0.813, Loss:  0.260\n",
      "Epoch   4 Batch  222/269 - Train Accuracy:  0.856, Validation Accuracy:  0.829, Loss:  0.249\n",
      "Epoch   4 Batch  223/269 - Train Accuracy:  0.816, Validation Accuracy:  0.832, Loss:  0.251\n",
      "Epoch   4 Batch  224/269 - Train Accuracy:  0.838, Validation Accuracy:  0.825, Loss:  0.266\n",
      "Epoch   4 Batch  225/269 - Train Accuracy:  0.804, Validation Accuracy:  0.813, Loss:  0.255\n",
      "Epoch   4 Batch  226/269 - Train Accuracy:  0.838, Validation Accuracy:  0.818, Loss:  0.257\n",
      "Epoch   4 Batch  227/269 - Train Accuracy:  0.844, Validation Accuracy:  0.822, Loss:  0.234\n",
      "Epoch   4 Batch  228/269 - Train Accuracy:  0.818, Validation Accuracy:  0.829, Loss:  0.253\n",
      "Epoch   4 Batch  229/269 - Train Accuracy:  0.812, Validation Accuracy:  0.818, Loss:  0.252\n",
      "Epoch   4 Batch  230/269 - Train Accuracy:  0.818, Validation Accuracy:  0.827, Loss:  0.251\n",
      "Epoch   4 Batch  231/269 - Train Accuracy:  0.806, Validation Accuracy:  0.820, Loss:  0.261\n",
      "Epoch   4 Batch  232/269 - Train Accuracy:  0.812, Validation Accuracy:  0.813, Loss:  0.264\n",
      "Epoch   4 Batch  233/269 - Train Accuracy:  0.826, Validation Accuracy:  0.816, Loss:  0.255\n",
      "Epoch   4 Batch  234/269 - Train Accuracy:  0.829, Validation Accuracy:  0.818, Loss:  0.247\n",
      "Epoch   4 Batch  235/269 - Train Accuracy:  0.837, Validation Accuracy:  0.820, Loss:  0.242\n",
      "Epoch   4 Batch  236/269 - Train Accuracy:  0.824, Validation Accuracy:  0.823, Loss:  0.241\n",
      "Epoch   4 Batch  237/269 - Train Accuracy:  0.824, Validation Accuracy:  0.820, Loss:  0.251\n",
      "Epoch   4 Batch  238/269 - Train Accuracy:  0.831, Validation Accuracy:  0.824, Loss:  0.247\n",
      "Epoch   4 Batch  239/269 - Train Accuracy:  0.831, Validation Accuracy:  0.827, Loss:  0.248\n",
      "Epoch   4 Batch  240/269 - Train Accuracy:  0.843, Validation Accuracy:  0.830, Loss:  0.229\n",
      "Epoch   4 Batch  241/269 - Train Accuracy:  0.829, Validation Accuracy:  0.825, Loss:  0.257\n",
      "Epoch   4 Batch  242/269 - Train Accuracy:  0.816, Validation Accuracy:  0.828, Loss:  0.245\n",
      "Epoch   4 Batch  243/269 - Train Accuracy:  0.837, Validation Accuracy:  0.833, Loss:  0.237\n",
      "Epoch   4 Batch  244/269 - Train Accuracy:  0.830, Validation Accuracy:  0.828, Loss:  0.251\n",
      "Epoch   4 Batch  245/269 - Train Accuracy:  0.811, Validation Accuracy:  0.826, Loss:  0.260\n",
      "Epoch   4 Batch  246/269 - Train Accuracy:  0.815, Validation Accuracy:  0.819, Loss:  0.241\n",
      "Epoch   4 Batch  247/269 - Train Accuracy:  0.813, Validation Accuracy:  0.825, Loss:  0.252\n",
      "Epoch   4 Batch  248/269 - Train Accuracy:  0.833, Validation Accuracy:  0.826, Loss:  0.239\n",
      "Epoch   4 Batch  249/269 - Train Accuracy:  0.843, Validation Accuracy:  0.825, Loss:  0.232\n",
      "Epoch   4 Batch  250/269 - Train Accuracy:  0.832, Validation Accuracy:  0.819, Loss:  0.250\n",
      "Epoch   4 Batch  251/269 - Train Accuracy:  0.844, Validation Accuracy:  0.826, Loss:  0.239\n",
      "Epoch   4 Batch  252/269 - Train Accuracy:  0.829, Validation Accuracy:  0.834, Loss:  0.239\n",
      "Epoch   4 Batch  253/269 - Train Accuracy:  0.813, Validation Accuracy:  0.832, Loss:  0.251\n",
      "Epoch   4 Batch  254/269 - Train Accuracy:  0.838, Validation Accuracy:  0.835, Loss:  0.239\n",
      "Epoch   4 Batch  255/269 - Train Accuracy:  0.838, Validation Accuracy:  0.834, Loss:  0.238\n",
      "Epoch   4 Batch  256/269 - Train Accuracy:  0.829, Validation Accuracy:  0.827, Loss:  0.240\n",
      "Epoch   4 Batch  257/269 - Train Accuracy:  0.810, Validation Accuracy:  0.824, Loss:  0.251\n",
      "Epoch   4 Batch  258/269 - Train Accuracy:  0.851, Validation Accuracy:  0.824, Loss:  0.241\n",
      "Epoch   4 Batch  259/269 - Train Accuracy:  0.831, Validation Accuracy:  0.824, Loss:  0.237\n",
      "Epoch   4 Batch  260/269 - Train Accuracy:  0.825, Validation Accuracy:  0.827, Loss:  0.249\n",
      "Epoch   4 Batch  261/269 - Train Accuracy:  0.822, Validation Accuracy:  0.831, Loss:  0.249\n",
      "Epoch   4 Batch  262/269 - Train Accuracy:  0.845, Validation Accuracy:  0.829, Loss:  0.238\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   4 Batch  263/269 - Train Accuracy:  0.852, Validation Accuracy:  0.831, Loss:  0.249\n",
      "Epoch   4 Batch  264/269 - Train Accuracy:  0.815, Validation Accuracy:  0.830, Loss:  0.249\n",
      "Epoch   4 Batch  265/269 - Train Accuracy:  0.828, Validation Accuracy:  0.831, Loss:  0.246\n",
      "Epoch   4 Batch  266/269 - Train Accuracy:  0.851, Validation Accuracy:  0.831, Loss:  0.231\n",
      "Epoch   4 Batch  267/269 - Train Accuracy:  0.838, Validation Accuracy:  0.829, Loss:  0.239\n",
      "Epoch   5 Batch    0/269 - Train Accuracy:  0.831, Validation Accuracy:  0.835, Loss:  0.252\n",
      "Epoch   5 Batch    1/269 - Train Accuracy:  0.826, Validation Accuracy:  0.833, Loss:  0.245\n",
      "Epoch   5 Batch    2/269 - Train Accuracy:  0.831, Validation Accuracy:  0.828, Loss:  0.238\n",
      "Epoch   5 Batch    3/269 - Train Accuracy:  0.828, Validation Accuracy:  0.839, Loss:  0.243\n",
      "Epoch   5 Batch    4/269 - Train Accuracy:  0.816, Validation Accuracy:  0.833, Loss:  0.244\n",
      "Epoch   5 Batch    5/269 - Train Accuracy:  0.822, Validation Accuracy:  0.838, Loss:  0.244\n",
      "Epoch   5 Batch    6/269 - Train Accuracy:  0.833, Validation Accuracy:  0.832, Loss:  0.227\n",
      "Epoch   5 Batch    7/269 - Train Accuracy:  0.836, Validation Accuracy:  0.833, Loss:  0.232\n",
      "Epoch   5 Batch    8/269 - Train Accuracy:  0.831, Validation Accuracy:  0.827, Loss:  0.239\n",
      "Epoch   5 Batch    9/269 - Train Accuracy:  0.833, Validation Accuracy:  0.837, Loss:  0.243\n",
      "Epoch   5 Batch   10/269 - Train Accuracy:  0.851, Validation Accuracy:  0.841, Loss:  0.234\n",
      "Epoch   5 Batch   11/269 - Train Accuracy:  0.839, Validation Accuracy:  0.844, Loss:  0.240\n",
      "Epoch   5 Batch   12/269 - Train Accuracy:  0.830, Validation Accuracy:  0.835, Loss:  0.248\n",
      "Epoch   5 Batch   13/269 - Train Accuracy:  0.839, Validation Accuracy:  0.834, Loss:  0.216\n",
      "Epoch   5 Batch   14/269 - Train Accuracy:  0.834, Validation Accuracy:  0.836, Loss:  0.230\n",
      "Epoch   5 Batch   15/269 - Train Accuracy:  0.842, Validation Accuracy:  0.837, Loss:  0.220\n",
      "Epoch   5 Batch   16/269 - Train Accuracy:  0.826, Validation Accuracy:  0.837, Loss:  0.235\n",
      "Epoch   5 Batch   17/269 - Train Accuracy:  0.842, Validation Accuracy:  0.844, Loss:  0.220\n",
      "Epoch   5 Batch   18/269 - Train Accuracy:  0.837, Validation Accuracy:  0.840, Loss:  0.231\n",
      "Epoch   5 Batch   19/269 - Train Accuracy:  0.855, Validation Accuracy:  0.838, Loss:  0.210\n",
      "Epoch   5 Batch   20/269 - Train Accuracy:  0.844, Validation Accuracy:  0.836, Loss:  0.232\n",
      "Epoch   5 Batch   21/269 - Train Accuracy:  0.824, Validation Accuracy:  0.841, Loss:  0.247\n",
      "Epoch   5 Batch   22/269 - Train Accuracy:  0.865, Validation Accuracy:  0.842, Loss:  0.217\n",
      "Epoch   5 Batch   23/269 - Train Accuracy:  0.839, Validation Accuracy:  0.845, Loss:  0.226\n",
      "Epoch   5 Batch   24/269 - Train Accuracy:  0.844, Validation Accuracy:  0.838, Loss:  0.234\n",
      "Epoch   5 Batch   25/269 - Train Accuracy:  0.849, Validation Accuracy:  0.838, Loss:  0.243\n",
      "Epoch   5 Batch   26/269 - Train Accuracy:  0.852, Validation Accuracy:  0.850, Loss:  0.214\n",
      "Epoch   5 Batch   27/269 - Train Accuracy:  0.848, Validation Accuracy:  0.851, Loss:  0.220\n",
      "Epoch   5 Batch   28/269 - Train Accuracy:  0.811, Validation Accuracy:  0.842, Loss:  0.242\n",
      "Epoch   5 Batch   29/269 - Train Accuracy:  0.854, Validation Accuracy:  0.849, Loss:  0.237\n",
      "Epoch   5 Batch   30/269 - Train Accuracy:  0.848, Validation Accuracy:  0.845, Loss:  0.224\n",
      "Epoch   5 Batch   31/269 - Train Accuracy:  0.863, Validation Accuracy:  0.836, Loss:  0.216\n",
      "Epoch   5 Batch   32/269 - Train Accuracy:  0.842, Validation Accuracy:  0.836, Loss:  0.219\n",
      "Epoch   5 Batch   33/269 - Train Accuracy:  0.853, Validation Accuracy:  0.841, Loss:  0.214\n",
      "Epoch   5 Batch   34/269 - Train Accuracy:  0.857, Validation Accuracy:  0.843, Loss:  0.220\n",
      "Epoch   5 Batch   35/269 - Train Accuracy:  0.844, Validation Accuracy:  0.845, Loss:  0.230\n",
      "Epoch   5 Batch   36/269 - Train Accuracy:  0.824, Validation Accuracy:  0.843, Loss:  0.223\n",
      "Epoch   5 Batch   37/269 - Train Accuracy:  0.839, Validation Accuracy:  0.844, Loss:  0.220\n",
      "Epoch   5 Batch   38/269 - Train Accuracy:  0.847, Validation Accuracy:  0.840, Loss:  0.221\n",
      "Epoch   5 Batch   39/269 - Train Accuracy:  0.853, Validation Accuracy:  0.846, Loss:  0.222\n",
      "Epoch   5 Batch   40/269 - Train Accuracy:  0.828, Validation Accuracy:  0.852, Loss:  0.229\n",
      "Epoch   5 Batch   41/269 - Train Accuracy:  0.834, Validation Accuracy:  0.845, Loss:  0.222\n",
      "Epoch   5 Batch   42/269 - Train Accuracy:  0.863, Validation Accuracy:  0.853, Loss:  0.204\n",
      "Epoch   5 Batch   43/269 - Train Accuracy:  0.852, Validation Accuracy:  0.854, Loss:  0.223\n",
      "Epoch   5 Batch   44/269 - Train Accuracy:  0.850, Validation Accuracy:  0.846, Loss:  0.218\n",
      "Epoch   5 Batch   45/269 - Train Accuracy:  0.841, Validation Accuracy:  0.849, Loss:  0.227\n",
      "Epoch   5 Batch   46/269 - Train Accuracy:  0.846, Validation Accuracy:  0.848, Loss:  0.222\n",
      "Epoch   5 Batch   47/269 - Train Accuracy:  0.863, Validation Accuracy:  0.845, Loss:  0.201\n",
      "Epoch   5 Batch   48/269 - Train Accuracy:  0.862, Validation Accuracy:  0.844, Loss:  0.213\n",
      "Epoch   5 Batch   49/269 - Train Accuracy:  0.836, Validation Accuracy:  0.846, Loss:  0.216\n",
      "Epoch   5 Batch   50/269 - Train Accuracy:  0.838, Validation Accuracy:  0.851, Loss:  0.227\n",
      "Epoch   5 Batch   51/269 - Train Accuracy:  0.865, Validation Accuracy:  0.853, Loss:  0.212\n",
      "Epoch   5 Batch   52/269 - Train Accuracy:  0.850, Validation Accuracy:  0.849, Loss:  0.202\n",
      "Epoch   5 Batch   53/269 - Train Accuracy:  0.860, Validation Accuracy:  0.850, Loss:  0.228\n",
      "Epoch   5 Batch   54/269 - Train Accuracy:  0.844, Validation Accuracy:  0.848, Loss:  0.218\n",
      "Epoch   5 Batch   55/269 - Train Accuracy:  0.862, Validation Accuracy:  0.847, Loss:  0.210\n",
      "Epoch   5 Batch   56/269 - Train Accuracy:  0.843, Validation Accuracy:  0.844, Loss:  0.214\n",
      "Epoch   5 Batch   57/269 - Train Accuracy:  0.847, Validation Accuracy:  0.841, Loss:  0.223\n",
      "Epoch   5 Batch   58/269 - Train Accuracy:  0.859, Validation Accuracy:  0.844, Loss:  0.208\n",
      "Epoch   5 Batch   59/269 - Train Accuracy:  0.875, Validation Accuracy:  0.852, Loss:  0.193\n",
      "Epoch   5 Batch   60/269 - Train Accuracy:  0.850, Validation Accuracy:  0.851, Loss:  0.203\n",
      "Epoch   5 Batch   61/269 - Train Accuracy:  0.873, Validation Accuracy:  0.852, Loss:  0.195\n",
      "Epoch   5 Batch   62/269 - Train Accuracy:  0.859, Validation Accuracy:  0.851, Loss:  0.208\n",
      "Epoch   5 Batch   63/269 - Train Accuracy:  0.856, Validation Accuracy:  0.853, Loss:  0.217\n",
      "Epoch   5 Batch   64/269 - Train Accuracy:  0.853, Validation Accuracy:  0.850, Loss:  0.202\n",
      "Epoch   5 Batch   65/269 - Train Accuracy:  0.848, Validation Accuracy:  0.850, Loss:  0.208\n",
      "Epoch   5 Batch   66/269 - Train Accuracy:  0.858, Validation Accuracy:  0.851, Loss:  0.206\n",
      "Epoch   5 Batch   67/269 - Train Accuracy:  0.853, Validation Accuracy:  0.853, Loss:  0.216\n",
      "Epoch   5 Batch   68/269 - Train Accuracy:  0.843, Validation Accuracy:  0.848, Loss:  0.217\n",
      "Epoch   5 Batch   69/269 - Train Accuracy:  0.837, Validation Accuracy:  0.853, Loss:  0.231\n",
      "Epoch   5 Batch   70/269 - Train Accuracy:  0.864, Validation Accuracy:  0.855, Loss:  0.206\n",
      "Epoch   5 Batch   71/269 - Train Accuracy:  0.854, Validation Accuracy:  0.852, Loss:  0.222\n",
      "Epoch   5 Batch   72/269 - Train Accuracy:  0.855, Validation Accuracy:  0.849, Loss:  0.209\n",
      "Epoch   5 Batch   73/269 - Train Accuracy:  0.851, Validation Accuracy:  0.851, Loss:  0.214\n",
      "Epoch   5 Batch   74/269 - Train Accuracy:  0.864, Validation Accuracy:  0.856, Loss:  0.207\n",
      "Epoch   5 Batch   75/269 - Train Accuracy:  0.864, Validation Accuracy:  0.861, Loss:  0.205\n",
      "Epoch   5 Batch   76/269 - Train Accuracy:  0.839, Validation Accuracy:  0.859, Loss:  0.206\n",
      "Epoch   5 Batch   77/269 - Train Accuracy:  0.851, Validation Accuracy:  0.854, Loss:  0.204\n",
      "Epoch   5 Batch   78/269 - Train Accuracy:  0.863, Validation Accuracy:  0.843, Loss:  0.204\n",
      "Epoch   5 Batch   79/269 - Train Accuracy:  0.838, Validation Accuracy:  0.847, Loss:  0.205\n",
      "Epoch   5 Batch   80/269 - Train Accuracy:  0.855, Validation Accuracy:  0.849, Loss:  0.201\n",
      "Epoch   5 Batch   81/269 - Train Accuracy:  0.849, Validation Accuracy:  0.851, Loss:  0.216\n",
      "Epoch   5 Batch   82/269 - Train Accuracy:  0.860, Validation Accuracy:  0.850, Loss:  0.198\n",
      "Epoch   5 Batch   83/269 - Train Accuracy:  0.859, Validation Accuracy:  0.853, Loss:  0.211\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   5 Batch   84/269 - Train Accuracy:  0.863, Validation Accuracy:  0.853, Loss:  0.201\n",
      "Epoch   5 Batch   85/269 - Train Accuracy:  0.851, Validation Accuracy:  0.852, Loss:  0.203\n",
      "Epoch   5 Batch   86/269 - Train Accuracy:  0.857, Validation Accuracy:  0.846, Loss:  0.195\n",
      "Epoch   5 Batch   87/269 - Train Accuracy:  0.848, Validation Accuracy:  0.850, Loss:  0.219\n",
      "Epoch   5 Batch   88/269 - Train Accuracy:  0.843, Validation Accuracy:  0.857, Loss:  0.206\n",
      "Epoch   5 Batch   89/269 - Train Accuracy:  0.867, Validation Accuracy:  0.858, Loss:  0.198\n",
      "Epoch   5 Batch   90/269 - Train Accuracy:  0.849, Validation Accuracy:  0.860, Loss:  0.211\n",
      "Epoch   5 Batch   91/269 - Train Accuracy:  0.872, Validation Accuracy:  0.859, Loss:  0.193\n",
      "Epoch   5 Batch   92/269 - Train Accuracy:  0.880, Validation Accuracy:  0.858, Loss:  0.193\n",
      "Epoch   5 Batch   93/269 - Train Accuracy:  0.871, Validation Accuracy:  0.861, Loss:  0.190\n",
      "Epoch   5 Batch   94/269 - Train Accuracy:  0.864, Validation Accuracy:  0.864, Loss:  0.205\n",
      "Epoch   5 Batch   95/269 - Train Accuracy:  0.855, Validation Accuracy:  0.860, Loss:  0.193\n",
      "Epoch   5 Batch   96/269 - Train Accuracy:  0.857, Validation Accuracy:  0.861, Loss:  0.199\n",
      "Epoch   5 Batch   97/269 - Train Accuracy:  0.864, Validation Accuracy:  0.861, Loss:  0.197\n",
      "Epoch   5 Batch   98/269 - Train Accuracy:  0.873, Validation Accuracy:  0.863, Loss:  0.195\n",
      "Epoch   5 Batch   99/269 - Train Accuracy:  0.860, Validation Accuracy:  0.865, Loss:  0.201\n",
      "Epoch   5 Batch  100/269 - Train Accuracy:  0.876, Validation Accuracy:  0.865, Loss:  0.194\n",
      "Epoch   5 Batch  101/269 - Train Accuracy:  0.860, Validation Accuracy:  0.867, Loss:  0.212\n",
      "Epoch   5 Batch  102/269 - Train Accuracy:  0.866, Validation Accuracy:  0.865, Loss:  0.194\n",
      "Epoch   5 Batch  103/269 - Train Accuracy:  0.884, Validation Accuracy:  0.863, Loss:  0.199\n",
      "Epoch   5 Batch  104/269 - Train Accuracy:  0.857, Validation Accuracy:  0.858, Loss:  0.193\n",
      "Epoch   5 Batch  105/269 - Train Accuracy:  0.864, Validation Accuracy:  0.866, Loss:  0.196\n",
      "Epoch   5 Batch  106/269 - Train Accuracy:  0.870, Validation Accuracy:  0.872, Loss:  0.186\n",
      "Epoch   5 Batch  107/269 - Train Accuracy:  0.866, Validation Accuracy:  0.871, Loss:  0.201\n",
      "Epoch   5 Batch  108/269 - Train Accuracy:  0.865, Validation Accuracy:  0.865, Loss:  0.196\n",
      "Epoch   5 Batch  109/269 - Train Accuracy:  0.848, Validation Accuracy:  0.863, Loss:  0.198\n",
      "Epoch   5 Batch  110/269 - Train Accuracy:  0.856, Validation Accuracy:  0.870, Loss:  0.190\n",
      "Epoch   5 Batch  111/269 - Train Accuracy:  0.856, Validation Accuracy:  0.870, Loss:  0.204\n",
      "Epoch   5 Batch  112/269 - Train Accuracy:  0.862, Validation Accuracy:  0.867, Loss:  0.191\n",
      "Epoch   5 Batch  113/269 - Train Accuracy:  0.858, Validation Accuracy:  0.864, Loss:  0.187\n",
      "Epoch   5 Batch  114/269 - Train Accuracy:  0.872, Validation Accuracy:  0.864, Loss:  0.193\n",
      "Epoch   5 Batch  115/269 - Train Accuracy:  0.871, Validation Accuracy:  0.862, Loss:  0.194\n",
      "Epoch   5 Batch  116/269 - Train Accuracy:  0.876, Validation Accuracy:  0.863, Loss:  0.197\n",
      "Epoch   5 Batch  117/269 - Train Accuracy:  0.876, Validation Accuracy:  0.872, Loss:  0.188\n",
      "Epoch   5 Batch  118/269 - Train Accuracy:  0.878, Validation Accuracy:  0.868, Loss:  0.182\n",
      "Epoch   5 Batch  119/269 - Train Accuracy:  0.858, Validation Accuracy:  0.863, Loss:  0.202\n",
      "Epoch   5 Batch  120/269 - Train Accuracy:  0.860, Validation Accuracy:  0.860, Loss:  0.191\n",
      "Epoch   5 Batch  121/269 - Train Accuracy:  0.867, Validation Accuracy:  0.863, Loss:  0.186\n",
      "Epoch   5 Batch  122/269 - Train Accuracy:  0.871, Validation Accuracy:  0.865, Loss:  0.183\n",
      "Epoch   5 Batch  123/269 - Train Accuracy:  0.867, Validation Accuracy:  0.868, Loss:  0.190\n",
      "Epoch   5 Batch  124/269 - Train Accuracy:  0.866, Validation Accuracy:  0.865, Loss:  0.181\n",
      "Epoch   5 Batch  125/269 - Train Accuracy:  0.873, Validation Accuracy:  0.866, Loss:  0.180\n",
      "Epoch   5 Batch  126/269 - Train Accuracy:  0.860, Validation Accuracy:  0.862, Loss:  0.186\n",
      "Epoch   5 Batch  127/269 - Train Accuracy:  0.864, Validation Accuracy:  0.867, Loss:  0.188\n",
      "Epoch   5 Batch  128/269 - Train Accuracy:  0.869, Validation Accuracy:  0.864, Loss:  0.185\n",
      "Epoch   5 Batch  129/269 - Train Accuracy:  0.869, Validation Accuracy:  0.862, Loss:  0.185\n",
      "Epoch   5 Batch  130/269 - Train Accuracy:  0.872, Validation Accuracy:  0.864, Loss:  0.193\n",
      "Epoch   5 Batch  131/269 - Train Accuracy:  0.863, Validation Accuracy:  0.864, Loss:  0.189\n",
      "Epoch   5 Batch  132/269 - Train Accuracy:  0.859, Validation Accuracy:  0.863, Loss:  0.191\n",
      "Epoch   5 Batch  133/269 - Train Accuracy:  0.887, Validation Accuracy:  0.862, Loss:  0.176\n",
      "Epoch   5 Batch  134/269 - Train Accuracy:  0.866, Validation Accuracy:  0.858, Loss:  0.185\n",
      "Epoch   5 Batch  135/269 - Train Accuracy:  0.871, Validation Accuracy:  0.867, Loss:  0.196\n",
      "Epoch   5 Batch  136/269 - Train Accuracy:  0.853, Validation Accuracy:  0.874, Loss:  0.194\n",
      "Epoch   5 Batch  137/269 - Train Accuracy:  0.849, Validation Accuracy:  0.870, Loss:  0.197\n",
      "Epoch   5 Batch  138/269 - Train Accuracy:  0.875, Validation Accuracy:  0.871, Loss:  0.183\n",
      "Epoch   5 Batch  139/269 - Train Accuracy:  0.874, Validation Accuracy:  0.872, Loss:  0.172\n",
      "Epoch   5 Batch  140/269 - Train Accuracy:  0.869, Validation Accuracy:  0.869, Loss:  0.188\n",
      "Epoch   5 Batch  141/269 - Train Accuracy:  0.869, Validation Accuracy:  0.868, Loss:  0.188\n",
      "Epoch   5 Batch  142/269 - Train Accuracy:  0.873, Validation Accuracy:  0.872, Loss:  0.175\n",
      "Epoch   5 Batch  143/269 - Train Accuracy:  0.873, Validation Accuracy:  0.870, Loss:  0.178\n",
      "Epoch   5 Batch  144/269 - Train Accuracy:  0.900, Validation Accuracy:  0.870, Loss:  0.165\n",
      "Epoch   5 Batch  145/269 - Train Accuracy:  0.873, Validation Accuracy:  0.872, Loss:  0.175\n",
      "Epoch   5 Batch  146/269 - Train Accuracy:  0.860, Validation Accuracy:  0.869, Loss:  0.179\n",
      "Epoch   5 Batch  147/269 - Train Accuracy:  0.879, Validation Accuracy:  0.869, Loss:  0.173\n",
      "Epoch   5 Batch  148/269 - Train Accuracy:  0.862, Validation Accuracy:  0.872, Loss:  0.184\n",
      "Epoch   5 Batch  149/269 - Train Accuracy:  0.852, Validation Accuracy:  0.871, Loss:  0.187\n",
      "Epoch   5 Batch  150/269 - Train Accuracy:  0.871, Validation Accuracy:  0.869, Loss:  0.174\n",
      "Epoch   5 Batch  151/269 - Train Accuracy:  0.872, Validation Accuracy:  0.869, Loss:  0.172\n",
      "Epoch   5 Batch  152/269 - Train Accuracy:  0.876, Validation Accuracy:  0.875, Loss:  0.178\n",
      "Epoch   5 Batch  153/269 - Train Accuracy:  0.881, Validation Accuracy:  0.878, Loss:  0.178\n",
      "Epoch   5 Batch  154/269 - Train Accuracy:  0.891, Validation Accuracy:  0.879, Loss:  0.182\n",
      "Epoch   5 Batch  155/269 - Train Accuracy:  0.870, Validation Accuracy:  0.877, Loss:  0.172\n",
      "Epoch   5 Batch  156/269 - Train Accuracy:  0.865, Validation Accuracy:  0.875, Loss:  0.182\n",
      "Epoch   5 Batch  157/269 - Train Accuracy:  0.878, Validation Accuracy:  0.869, Loss:  0.171\n",
      "Epoch   5 Batch  158/269 - Train Accuracy:  0.870, Validation Accuracy:  0.872, Loss:  0.176\n",
      "Epoch   5 Batch  159/269 - Train Accuracy:  0.872, Validation Accuracy:  0.877, Loss:  0.180\n",
      "Epoch   5 Batch  160/269 - Train Accuracy:  0.883, Validation Accuracy:  0.868, Loss:  0.179\n",
      "Epoch   5 Batch  161/269 - Train Accuracy:  0.873, Validation Accuracy:  0.870, Loss:  0.175\n",
      "Epoch   5 Batch  162/269 - Train Accuracy:  0.894, Validation Accuracy:  0.873, Loss:  0.176\n",
      "Epoch   5 Batch  163/269 - Train Accuracy:  0.882, Validation Accuracy:  0.875, Loss:  0.173\n",
      "Epoch   5 Batch  164/269 - Train Accuracy:  0.878, Validation Accuracy:  0.870, Loss:  0.172\n",
      "Epoch   5 Batch  165/269 - Train Accuracy:  0.881, Validation Accuracy:  0.873, Loss:  0.176\n",
      "Epoch   5 Batch  166/269 - Train Accuracy:  0.877, Validation Accuracy:  0.878, Loss:  0.169\n",
      "Epoch   5 Batch  167/269 - Train Accuracy:  0.884, Validation Accuracy:  0.877, Loss:  0.168\n",
      "Epoch   5 Batch  168/269 - Train Accuracy:  0.881, Validation Accuracy:  0.878, Loss:  0.177\n",
      "Epoch   5 Batch  169/269 - Train Accuracy:  0.874, Validation Accuracy:  0.876, Loss:  0.169\n",
      "Epoch   5 Batch  170/269 - Train Accuracy:  0.874, Validation Accuracy:  0.876, Loss:  0.169\n",
      "Epoch   5 Batch  171/269 - Train Accuracy:  0.884, Validation Accuracy:  0.882, Loss:  0.175\n",
      "Epoch   5 Batch  172/269 - Train Accuracy:  0.870, Validation Accuracy:  0.873, Loss:  0.181\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   5 Batch  173/269 - Train Accuracy:  0.886, Validation Accuracy:  0.868, Loss:  0.163\n",
      "Epoch   5 Batch  174/269 - Train Accuracy:  0.884, Validation Accuracy:  0.867, Loss:  0.169\n",
      "Epoch   5 Batch  175/269 - Train Accuracy:  0.870, Validation Accuracy:  0.866, Loss:  0.185\n",
      "Epoch   5 Batch  176/269 - Train Accuracy:  0.869, Validation Accuracy:  0.866, Loss:  0.181\n",
      "Epoch   5 Batch  177/269 - Train Accuracy:  0.880, Validation Accuracy:  0.870, Loss:  0.163\n",
      "Epoch   5 Batch  178/269 - Train Accuracy:  0.896, Validation Accuracy:  0.875, Loss:  0.167\n",
      "Epoch   5 Batch  179/269 - Train Accuracy:  0.869, Validation Accuracy:  0.883, Loss:  0.169\n",
      "Epoch   5 Batch  180/269 - Train Accuracy:  0.900, Validation Accuracy:  0.883, Loss:  0.164\n",
      "Epoch   5 Batch  181/269 - Train Accuracy:  0.878, Validation Accuracy:  0.888, Loss:  0.170\n",
      "Epoch   5 Batch  182/269 - Train Accuracy:  0.877, Validation Accuracy:  0.890, Loss:  0.169\n",
      "Epoch   5 Batch  183/269 - Train Accuracy:  0.905, Validation Accuracy:  0.883, Loss:  0.149\n",
      "Epoch   5 Batch  184/269 - Train Accuracy:  0.877, Validation Accuracy:  0.880, Loss:  0.167\n",
      "Epoch   5 Batch  185/269 - Train Accuracy:  0.880, Validation Accuracy:  0.877, Loss:  0.164\n",
      "Epoch   5 Batch  186/269 - Train Accuracy:  0.875, Validation Accuracy:  0.876, Loss:  0.168\n",
      "Epoch   5 Batch  187/269 - Train Accuracy:  0.875, Validation Accuracy:  0.882, Loss:  0.162\n",
      "Epoch   5 Batch  188/269 - Train Accuracy:  0.884, Validation Accuracy:  0.882, Loss:  0.159\n",
      "Epoch   5 Batch  189/269 - Train Accuracy:  0.890, Validation Accuracy:  0.880, Loss:  0.156\n",
      "Epoch   5 Batch  190/269 - Train Accuracy:  0.890, Validation Accuracy:  0.876, Loss:  0.159\n",
      "Epoch   5 Batch  191/269 - Train Accuracy:  0.858, Validation Accuracy:  0.881, Loss:  0.166\n",
      "Epoch   5 Batch  192/269 - Train Accuracy:  0.881, Validation Accuracy:  0.882, Loss:  0.169\n",
      "Epoch   5 Batch  193/269 - Train Accuracy:  0.886, Validation Accuracy:  0.881, Loss:  0.163\n",
      "Epoch   5 Batch  194/269 - Train Accuracy:  0.884, Validation Accuracy:  0.879, Loss:  0.164\n",
      "Epoch   5 Batch  195/269 - Train Accuracy:  0.877, Validation Accuracy:  0.883, Loss:  0.160\n",
      "Epoch   5 Batch  196/269 - Train Accuracy:  0.872, Validation Accuracy:  0.888, Loss:  0.158\n",
      "Epoch   5 Batch  197/269 - Train Accuracy:  0.873, Validation Accuracy:  0.888, Loss:  0.168\n",
      "Epoch   5 Batch  198/269 - Train Accuracy:  0.881, Validation Accuracy:  0.893, Loss:  0.169\n",
      "Epoch   5 Batch  199/269 - Train Accuracy:  0.875, Validation Accuracy:  0.887, Loss:  0.169\n",
      "Epoch   5 Batch  200/269 - Train Accuracy:  0.879, Validation Accuracy:  0.891, Loss:  0.166\n",
      "Epoch   5 Batch  201/269 - Train Accuracy:  0.881, Validation Accuracy:  0.887, Loss:  0.164\n",
      "Epoch   5 Batch  202/269 - Train Accuracy:  0.879, Validation Accuracy:  0.887, Loss:  0.160\n",
      "Epoch   5 Batch  203/269 - Train Accuracy:  0.886, Validation Accuracy:  0.880, Loss:  0.174\n",
      "Epoch   5 Batch  204/269 - Train Accuracy:  0.891, Validation Accuracy:  0.880, Loss:  0.168\n",
      "Epoch   5 Batch  205/269 - Train Accuracy:  0.882, Validation Accuracy:  0.884, Loss:  0.155\n",
      "Epoch   5 Batch  206/269 - Train Accuracy:  0.880, Validation Accuracy:  0.885, Loss:  0.169\n",
      "Epoch   5 Batch  207/269 - Train Accuracy:  0.885, Validation Accuracy:  0.885, Loss:  0.156\n",
      "Epoch   5 Batch  208/269 - Train Accuracy:  0.885, Validation Accuracy:  0.878, Loss:  0.167\n",
      "Epoch   5 Batch  209/269 - Train Accuracy:  0.890, Validation Accuracy:  0.884, Loss:  0.158\n",
      "Epoch   5 Batch  210/269 - Train Accuracy:  0.889, Validation Accuracy:  0.886, Loss:  0.151\n",
      "Epoch   5 Batch  211/269 - Train Accuracy:  0.895, Validation Accuracy:  0.886, Loss:  0.159\n",
      "Epoch   5 Batch  212/269 - Train Accuracy:  0.887, Validation Accuracy:  0.886, Loss:  0.162\n",
      "Epoch   5 Batch  213/269 - Train Accuracy:  0.878, Validation Accuracy:  0.886, Loss:  0.157\n",
      "Epoch   5 Batch  214/269 - Train Accuracy:  0.871, Validation Accuracy:  0.887, Loss:  0.160\n",
      "Epoch   5 Batch  215/269 - Train Accuracy:  0.892, Validation Accuracy:  0.880, Loss:  0.148\n",
      "Epoch   5 Batch  216/269 - Train Accuracy:  0.877, Validation Accuracy:  0.881, Loss:  0.169\n",
      "Epoch   5 Batch  217/269 - Train Accuracy:  0.880, Validation Accuracy:  0.881, Loss:  0.161\n",
      "Epoch   5 Batch  218/269 - Train Accuracy:  0.886, Validation Accuracy:  0.885, Loss:  0.159\n",
      "Epoch   5 Batch  219/269 - Train Accuracy:  0.898, Validation Accuracy:  0.891, Loss:  0.164\n",
      "Epoch   5 Batch  220/269 - Train Accuracy:  0.883, Validation Accuracy:  0.892, Loss:  0.147\n",
      "Epoch   5 Batch  221/269 - Train Accuracy:  0.888, Validation Accuracy:  0.896, Loss:  0.159\n",
      "Epoch   5 Batch  222/269 - Train Accuracy:  0.909, Validation Accuracy:  0.894, Loss:  0.147\n",
      "Epoch   5 Batch  223/269 - Train Accuracy:  0.897, Validation Accuracy:  0.892, Loss:  0.150\n",
      "Epoch   5 Batch  224/269 - Train Accuracy:  0.889, Validation Accuracy:  0.891, Loss:  0.164\n",
      "Epoch   5 Batch  225/269 - Train Accuracy:  0.876, Validation Accuracy:  0.886, Loss:  0.150\n",
      "Epoch   5 Batch  226/269 - Train Accuracy:  0.896, Validation Accuracy:  0.885, Loss:  0.157\n",
      "Epoch   5 Batch  227/269 - Train Accuracy:  0.909, Validation Accuracy:  0.885, Loss:  0.146\n",
      "Epoch   5 Batch  228/269 - Train Accuracy:  0.886, Validation Accuracy:  0.889, Loss:  0.152\n",
      "Epoch   5 Batch  229/269 - Train Accuracy:  0.881, Validation Accuracy:  0.889, Loss:  0.152\n",
      "Epoch   5 Batch  230/269 - Train Accuracy:  0.901, Validation Accuracy:  0.892, Loss:  0.150\n",
      "Epoch   5 Batch  231/269 - Train Accuracy:  0.880, Validation Accuracy:  0.890, Loss:  0.157\n",
      "Epoch   5 Batch  232/269 - Train Accuracy:  0.878, Validation Accuracy:  0.885, Loss:  0.156\n",
      "Epoch   5 Batch  233/269 - Train Accuracy:  0.894, Validation Accuracy:  0.888, Loss:  0.155\n",
      "Epoch   5 Batch  234/269 - Train Accuracy:  0.886, Validation Accuracy:  0.888, Loss:  0.147\n",
      "Epoch   5 Batch  235/269 - Train Accuracy:  0.905, Validation Accuracy:  0.896, Loss:  0.143\n",
      "Epoch   5 Batch  236/269 - Train Accuracy:  0.885, Validation Accuracy:  0.891, Loss:  0.141\n",
      "Epoch   5 Batch  237/269 - Train Accuracy:  0.892, Validation Accuracy:  0.883, Loss:  0.150\n",
      "Epoch   5 Batch  238/269 - Train Accuracy:  0.890, Validation Accuracy:  0.885, Loss:  0.149\n",
      "Epoch   5 Batch  239/269 - Train Accuracy:  0.895, Validation Accuracy:  0.886, Loss:  0.150\n",
      "Epoch   5 Batch  240/269 - Train Accuracy:  0.903, Validation Accuracy:  0.883, Loss:  0.140\n",
      "Epoch   5 Batch  241/269 - Train Accuracy:  0.889, Validation Accuracy:  0.887, Loss:  0.158\n",
      "Epoch   5 Batch  242/269 - Train Accuracy:  0.896, Validation Accuracy:  0.892, Loss:  0.148\n",
      "Epoch   5 Batch  243/269 - Train Accuracy:  0.892, Validation Accuracy:  0.895, Loss:  0.139\n",
      "Epoch   5 Batch  244/269 - Train Accuracy:  0.890, Validation Accuracy:  0.891, Loss:  0.153\n",
      "Epoch   5 Batch  245/269 - Train Accuracy:  0.880, Validation Accuracy:  0.894, Loss:  0.156\n",
      "Epoch   5 Batch  246/269 - Train Accuracy:  0.883, Validation Accuracy:  0.890, Loss:  0.146\n",
      "Epoch   5 Batch  247/269 - Train Accuracy:  0.882, Validation Accuracy:  0.891, Loss:  0.149\n",
      "Epoch   5 Batch  248/269 - Train Accuracy:  0.902, Validation Accuracy:  0.884, Loss:  0.143\n",
      "Epoch   5 Batch  249/269 - Train Accuracy:  0.902, Validation Accuracy:  0.883, Loss:  0.138\n",
      "Epoch   5 Batch  250/269 - Train Accuracy:  0.897, Validation Accuracy:  0.894, Loss:  0.149\n",
      "Epoch   5 Batch  251/269 - Train Accuracy:  0.905, Validation Accuracy:  0.899, Loss:  0.143\n",
      "Epoch   5 Batch  252/269 - Train Accuracy:  0.902, Validation Accuracy:  0.903, Loss:  0.139\n",
      "Epoch   5 Batch  253/269 - Train Accuracy:  0.878, Validation Accuracy:  0.903, Loss:  0.152\n",
      "Epoch   5 Batch  254/269 - Train Accuracy:  0.893, Validation Accuracy:  0.904, Loss:  0.144\n",
      "Epoch   5 Batch  255/269 - Train Accuracy:  0.896, Validation Accuracy:  0.902, Loss:  0.141\n",
      "Epoch   5 Batch  256/269 - Train Accuracy:  0.884, Validation Accuracy:  0.898, Loss:  0.142\n",
      "Epoch   5 Batch  257/269 - Train Accuracy:  0.870, Validation Accuracy:  0.895, Loss:  0.152\n",
      "Epoch   5 Batch  258/269 - Train Accuracy:  0.892, Validation Accuracy:  0.892, Loss:  0.146\n",
      "Epoch   5 Batch  259/269 - Train Accuracy:  0.886, Validation Accuracy:  0.891, Loss:  0.141\n",
      "Epoch   5 Batch  260/269 - Train Accuracy:  0.887, Validation Accuracy:  0.892, Loss:  0.148\n",
      "Epoch   5 Batch  261/269 - Train Accuracy:  0.889, Validation Accuracy:  0.888, Loss:  0.145\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   5 Batch  262/269 - Train Accuracy:  0.899, Validation Accuracy:  0.904, Loss:  0.141\n",
      "Epoch   5 Batch  263/269 - Train Accuracy:  0.906, Validation Accuracy:  0.906, Loss:  0.146\n",
      "Epoch   5 Batch  264/269 - Train Accuracy:  0.870, Validation Accuracy:  0.907, Loss:  0.151\n",
      "Epoch   5 Batch  265/269 - Train Accuracy:  0.885, Validation Accuracy:  0.906, Loss:  0.145\n",
      "Epoch   5 Batch  266/269 - Train Accuracy:  0.901, Validation Accuracy:  0.903, Loss:  0.136\n",
      "Epoch   5 Batch  267/269 - Train Accuracy:  0.907, Validation Accuracy:  0.900, Loss:  0.145\n",
      "Epoch   6 Batch    0/269 - Train Accuracy:  0.905, Validation Accuracy:  0.899, Loss:  0.151\n",
      "Epoch   6 Batch    1/269 - Train Accuracy:  0.881, Validation Accuracy:  0.899, Loss:  0.144\n",
      "Epoch   6 Batch    2/269 - Train Accuracy:  0.891, Validation Accuracy:  0.899, Loss:  0.146\n",
      "Epoch   6 Batch    3/269 - Train Accuracy:  0.891, Validation Accuracy:  0.900, Loss:  0.144\n",
      "Epoch   6 Batch    4/269 - Train Accuracy:  0.882, Validation Accuracy:  0.904, Loss:  0.144\n",
      "Epoch   6 Batch    5/269 - Train Accuracy:  0.905, Validation Accuracy:  0.902, Loss:  0.141\n",
      "Epoch   6 Batch    6/269 - Train Accuracy:  0.913, Validation Accuracy:  0.899, Loss:  0.134\n",
      "Epoch   6 Batch    7/269 - Train Accuracy:  0.900, Validation Accuracy:  0.899, Loss:  0.135\n",
      "Epoch   6 Batch    8/269 - Train Accuracy:  0.906, Validation Accuracy:  0.901, Loss:  0.143\n",
      "Epoch   6 Batch    9/269 - Train Accuracy:  0.889, Validation Accuracy:  0.900, Loss:  0.145\n",
      "Epoch   6 Batch   10/269 - Train Accuracy:  0.905, Validation Accuracy:  0.894, Loss:  0.135\n",
      "Epoch   6 Batch   11/269 - Train Accuracy:  0.895, Validation Accuracy:  0.898, Loss:  0.145\n",
      "Epoch   6 Batch   12/269 - Train Accuracy:  0.889, Validation Accuracy:  0.898, Loss:  0.150\n",
      "Epoch   6 Batch   13/269 - Train Accuracy:  0.909, Validation Accuracy:  0.902, Loss:  0.124\n",
      "Epoch   6 Batch   14/269 - Train Accuracy:  0.898, Validation Accuracy:  0.902, Loss:  0.137\n",
      "Epoch   6 Batch   15/269 - Train Accuracy:  0.900, Validation Accuracy:  0.904, Loss:  0.125\n",
      "Epoch   6 Batch   16/269 - Train Accuracy:  0.899, Validation Accuracy:  0.902, Loss:  0.143\n",
      "Epoch   6 Batch   17/269 - Train Accuracy:  0.910, Validation Accuracy:  0.898, Loss:  0.126\n",
      "Epoch   6 Batch   18/269 - Train Accuracy:  0.891, Validation Accuracy:  0.902, Loss:  0.137\n",
      "Epoch   6 Batch   19/269 - Train Accuracy:  0.913, Validation Accuracy:  0.910, Loss:  0.120\n",
      "Epoch   6 Batch   20/269 - Train Accuracy:  0.914, Validation Accuracy:  0.904, Loss:  0.135\n",
      "Epoch   6 Batch   21/269 - Train Accuracy:  0.885, Validation Accuracy:  0.903, Loss:  0.151\n",
      "Epoch   6 Batch   22/269 - Train Accuracy:  0.910, Validation Accuracy:  0.908, Loss:  0.129\n",
      "Epoch   6 Batch   23/269 - Train Accuracy:  0.896, Validation Accuracy:  0.907, Loss:  0.138\n",
      "Epoch   6 Batch   24/269 - Train Accuracy:  0.901, Validation Accuracy:  0.900, Loss:  0.136\n",
      "Epoch   6 Batch   25/269 - Train Accuracy:  0.885, Validation Accuracy:  0.901, Loss:  0.145\n",
      "Epoch   6 Batch   26/269 - Train Accuracy:  0.907, Validation Accuracy:  0.904, Loss:  0.125\n",
      "Epoch   6 Batch   27/269 - Train Accuracy:  0.894, Validation Accuracy:  0.910, Loss:  0.129\n",
      "Epoch   6 Batch   28/269 - Train Accuracy:  0.883, Validation Accuracy:  0.905, Loss:  0.141\n",
      "Epoch   6 Batch   29/269 - Train Accuracy:  0.903, Validation Accuracy:  0.905, Loss:  0.140\n",
      "Epoch   6 Batch   30/269 - Train Accuracy:  0.905, Validation Accuracy:  0.904, Loss:  0.129\n",
      "Epoch   6 Batch   31/269 - Train Accuracy:  0.914, Validation Accuracy:  0.888, Loss:  0.124\n",
      "Epoch   6 Batch   32/269 - Train Accuracy:  0.896, Validation Accuracy:  0.895, Loss:  0.129\n",
      "Epoch   6 Batch   33/269 - Train Accuracy:  0.909, Validation Accuracy:  0.901, Loss:  0.123\n",
      "Epoch   6 Batch   34/269 - Train Accuracy:  0.906, Validation Accuracy:  0.903, Loss:  0.129\n",
      "Epoch   6 Batch   35/269 - Train Accuracy:  0.903, Validation Accuracy:  0.901, Loss:  0.140\n",
      "Epoch   6 Batch   36/269 - Train Accuracy:  0.882, Validation Accuracy:  0.899, Loss:  0.133\n",
      "Epoch   6 Batch   37/269 - Train Accuracy:  0.896, Validation Accuracy:  0.907, Loss:  0.131\n",
      "Epoch   6 Batch   38/269 - Train Accuracy:  0.897, Validation Accuracy:  0.904, Loss:  0.130\n",
      "Epoch   6 Batch   39/269 - Train Accuracy:  0.901, Validation Accuracy:  0.907, Loss:  0.129\n",
      "Epoch   6 Batch   40/269 - Train Accuracy:  0.891, Validation Accuracy:  0.910, Loss:  0.136\n",
      "Epoch   6 Batch   41/269 - Train Accuracy:  0.896, Validation Accuracy:  0.911, Loss:  0.133\n",
      "Epoch   6 Batch   42/269 - Train Accuracy:  0.913, Validation Accuracy:  0.906, Loss:  0.118\n",
      "Epoch   6 Batch   43/269 - Train Accuracy:  0.895, Validation Accuracy:  0.913, Loss:  0.131\n",
      "Epoch   6 Batch   44/269 - Train Accuracy:  0.907, Validation Accuracy:  0.912, Loss:  0.130\n",
      "Epoch   6 Batch   45/269 - Train Accuracy:  0.904, Validation Accuracy:  0.911, Loss:  0.132\n",
      "Epoch   6 Batch   46/269 - Train Accuracy:  0.897, Validation Accuracy:  0.912, Loss:  0.131\n",
      "Epoch   6 Batch   47/269 - Train Accuracy:  0.923, Validation Accuracy:  0.915, Loss:  0.116\n",
      "Epoch   6 Batch   48/269 - Train Accuracy:  0.920, Validation Accuracy:  0.907, Loss:  0.123\n",
      "Epoch   6 Batch   49/269 - Train Accuracy:  0.899, Validation Accuracy:  0.905, Loss:  0.125\n",
      "Epoch   6 Batch   50/269 - Train Accuracy:  0.894, Validation Accuracy:  0.904, Loss:  0.137\n",
      "Epoch   6 Batch   51/269 - Train Accuracy:  0.913, Validation Accuracy:  0.910, Loss:  0.124\n",
      "Epoch   6 Batch   52/269 - Train Accuracy:  0.903, Validation Accuracy:  0.908, Loss:  0.119\n",
      "Epoch   6 Batch   53/269 - Train Accuracy:  0.900, Validation Accuracy:  0.911, Loss:  0.136\n",
      "Epoch   6 Batch   54/269 - Train Accuracy:  0.902, Validation Accuracy:  0.909, Loss:  0.128\n",
      "Epoch   6 Batch   55/269 - Train Accuracy:  0.918, Validation Accuracy:  0.908, Loss:  0.125\n",
      "Epoch   6 Batch   56/269 - Train Accuracy:  0.888, Validation Accuracy:  0.905, Loss:  0.129\n",
      "Epoch   6 Batch   57/269 - Train Accuracy:  0.900, Validation Accuracy:  0.902, Loss:  0.135\n",
      "Epoch   6 Batch   58/269 - Train Accuracy:  0.921, Validation Accuracy:  0.909, Loss:  0.124\n",
      "Epoch   6 Batch   59/269 - Train Accuracy:  0.930, Validation Accuracy:  0.910, Loss:  0.110\n",
      "Epoch   6 Batch   60/269 - Train Accuracy:  0.907, Validation Accuracy:  0.913, Loss:  0.119\n",
      "Epoch   6 Batch   61/269 - Train Accuracy:  0.912, Validation Accuracy:  0.913, Loss:  0.114\n",
      "Epoch   6 Batch   62/269 - Train Accuracy:  0.896, Validation Accuracy:  0.915, Loss:  0.126\n",
      "Epoch   6 Batch   63/269 - Train Accuracy:  0.903, Validation Accuracy:  0.913, Loss:  0.132\n",
      "Epoch   6 Batch   64/269 - Train Accuracy:  0.906, Validation Accuracy:  0.910, Loss:  0.119\n",
      "Epoch   6 Batch   65/269 - Train Accuracy:  0.894, Validation Accuracy:  0.910, Loss:  0.123\n",
      "Epoch   6 Batch   66/269 - Train Accuracy:  0.898, Validation Accuracy:  0.913, Loss:  0.124\n",
      "Epoch   6 Batch   67/269 - Train Accuracy:  0.908, Validation Accuracy:  0.910, Loss:  0.129\n",
      "Epoch   6 Batch   68/269 - Train Accuracy:  0.894, Validation Accuracy:  0.910, Loss:  0.130\n",
      "Epoch   6 Batch   69/269 - Train Accuracy:  0.884, Validation Accuracy:  0.917, Loss:  0.140\n",
      "Epoch   6 Batch   70/269 - Train Accuracy:  0.921, Validation Accuracy:  0.914, Loss:  0.123\n",
      "Epoch   6 Batch   71/269 - Train Accuracy:  0.900, Validation Accuracy:  0.913, Loss:  0.134\n",
      "Epoch   6 Batch   72/269 - Train Accuracy:  0.900, Validation Accuracy:  0.909, Loss:  0.127\n",
      "Epoch   6 Batch   73/269 - Train Accuracy:  0.901, Validation Accuracy:  0.911, Loss:  0.128\n",
      "Epoch   6 Batch   74/269 - Train Accuracy:  0.907, Validation Accuracy:  0.910, Loss:  0.121\n",
      "Epoch   6 Batch   75/269 - Train Accuracy:  0.919, Validation Accuracy:  0.913, Loss:  0.121\n",
      "Epoch   6 Batch   76/269 - Train Accuracy:  0.910, Validation Accuracy:  0.916, Loss:  0.118\n",
      "Epoch   6 Batch   77/269 - Train Accuracy:  0.904, Validation Accuracy:  0.910, Loss:  0.118\n",
      "Epoch   6 Batch   78/269 - Train Accuracy:  0.916, Validation Accuracy:  0.911, Loss:  0.122\n",
      "Epoch   6 Batch   79/269 - Train Accuracy:  0.889, Validation Accuracy:  0.917, Loss:  0.119\n",
      "Epoch   6 Batch   80/269 - Train Accuracy:  0.908, Validation Accuracy:  0.915, Loss:  0.115\n",
      "Epoch   6 Batch   81/269 - Train Accuracy:  0.903, Validation Accuracy:  0.908, Loss:  0.132\n",
      "Epoch   6 Batch   82/269 - Train Accuracy:  0.911, Validation Accuracy:  0.902, Loss:  0.112\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   6 Batch   83/269 - Train Accuracy:  0.902, Validation Accuracy:  0.905, Loss:  0.130\n",
      "Epoch   6 Batch   84/269 - Train Accuracy:  0.915, Validation Accuracy:  0.907, Loss:  0.119\n",
      "Epoch   6 Batch   85/269 - Train Accuracy:  0.908, Validation Accuracy:  0.908, Loss:  0.119\n",
      "Epoch   6 Batch   86/269 - Train Accuracy:  0.908, Validation Accuracy:  0.909, Loss:  0.114\n",
      "Epoch   6 Batch   87/269 - Train Accuracy:  0.889, Validation Accuracy:  0.908, Loss:  0.128\n",
      "Epoch   6 Batch   88/269 - Train Accuracy:  0.895, Validation Accuracy:  0.914, Loss:  0.122\n",
      "Epoch   6 Batch   89/269 - Train Accuracy:  0.916, Validation Accuracy:  0.920, Loss:  0.115\n",
      "Epoch   6 Batch   90/269 - Train Accuracy:  0.905, Validation Accuracy:  0.916, Loss:  0.122\n",
      "Epoch   6 Batch   91/269 - Train Accuracy:  0.916, Validation Accuracy:  0.915, Loss:  0.110\n",
      "Epoch   6 Batch   92/269 - Train Accuracy:  0.926, Validation Accuracy:  0.919, Loss:  0.110\n",
      "Epoch   6 Batch   93/269 - Train Accuracy:  0.923, Validation Accuracy:  0.911, Loss:  0.111\n",
      "Epoch   6 Batch   94/269 - Train Accuracy:  0.914, Validation Accuracy:  0.918, Loss:  0.125\n",
      "Epoch   6 Batch   95/269 - Train Accuracy:  0.904, Validation Accuracy:  0.913, Loss:  0.115\n",
      "Epoch   6 Batch   96/269 - Train Accuracy:  0.902, Validation Accuracy:  0.913, Loss:  0.118\n",
      "Epoch   6 Batch   97/269 - Train Accuracy:  0.904, Validation Accuracy:  0.910, Loss:  0.118\n",
      "Epoch   6 Batch   98/269 - Train Accuracy:  0.910, Validation Accuracy:  0.918, Loss:  0.115\n",
      "Epoch   6 Batch   99/269 - Train Accuracy:  0.918, Validation Accuracy:  0.914, Loss:  0.118\n",
      "Epoch   6 Batch  100/269 - Train Accuracy:  0.920, Validation Accuracy:  0.915, Loss:  0.114\n",
      "Epoch   6 Batch  101/269 - Train Accuracy:  0.909, Validation Accuracy:  0.916, Loss:  0.126\n",
      "Epoch   6 Batch  102/269 - Train Accuracy:  0.911, Validation Accuracy:  0.916, Loss:  0.115\n",
      "Epoch   6 Batch  103/269 - Train Accuracy:  0.910, Validation Accuracy:  0.913, Loss:  0.119\n",
      "Epoch   6 Batch  104/269 - Train Accuracy:  0.908, Validation Accuracy:  0.916, Loss:  0.113\n",
      "Epoch   6 Batch  105/269 - Train Accuracy:  0.905, Validation Accuracy:  0.917, Loss:  0.115\n",
      "Epoch   6 Batch  106/269 - Train Accuracy:  0.926, Validation Accuracy:  0.916, Loss:  0.107\n",
      "Epoch   6 Batch  107/269 - Train Accuracy:  0.918, Validation Accuracy:  0.911, Loss:  0.117\n",
      "Epoch   6 Batch  108/269 - Train Accuracy:  0.915, Validation Accuracy:  0.913, Loss:  0.113\n",
      "Epoch   6 Batch  109/269 - Train Accuracy:  0.896, Validation Accuracy:  0.916, Loss:  0.119\n",
      "Epoch   6 Batch  110/269 - Train Accuracy:  0.900, Validation Accuracy:  0.920, Loss:  0.110\n",
      "Epoch   6 Batch  111/269 - Train Accuracy:  0.901, Validation Accuracy:  0.918, Loss:  0.118\n",
      "Epoch   6 Batch  112/269 - Train Accuracy:  0.907, Validation Accuracy:  0.917, Loss:  0.115\n",
      "Epoch   6 Batch  113/269 - Train Accuracy:  0.904, Validation Accuracy:  0.915, Loss:  0.112\n",
      "Epoch   6 Batch  114/269 - Train Accuracy:  0.913, Validation Accuracy:  0.917, Loss:  0.115\n",
      "Epoch   6 Batch  115/269 - Train Accuracy:  0.905, Validation Accuracy:  0.914, Loss:  0.113\n",
      "Epoch   6 Batch  116/269 - Train Accuracy:  0.921, Validation Accuracy:  0.914, Loss:  0.118\n",
      "Epoch   6 Batch  117/269 - Train Accuracy:  0.915, Validation Accuracy:  0.917, Loss:  0.108\n",
      "Epoch   6 Batch  118/269 - Train Accuracy:  0.921, Validation Accuracy:  0.913, Loss:  0.106\n",
      "Epoch   6 Batch  119/269 - Train Accuracy:  0.896, Validation Accuracy:  0.918, Loss:  0.119\n",
      "Epoch   6 Batch  120/269 - Train Accuracy:  0.924, Validation Accuracy:  0.919, Loss:  0.113\n",
      "Epoch   6 Batch  121/269 - Train Accuracy:  0.908, Validation Accuracy:  0.912, Loss:  0.110\n",
      "Epoch   6 Batch  122/269 - Train Accuracy:  0.917, Validation Accuracy:  0.915, Loss:  0.108\n",
      "Epoch   6 Batch  123/269 - Train Accuracy:  0.904, Validation Accuracy:  0.909, Loss:  0.110\n",
      "Epoch   6 Batch  124/269 - Train Accuracy:  0.915, Validation Accuracy:  0.910, Loss:  0.105\n",
      "Epoch   6 Batch  125/269 - Train Accuracy:  0.916, Validation Accuracy:  0.915, Loss:  0.104\n",
      "Epoch   6 Batch  126/269 - Train Accuracy:  0.897, Validation Accuracy:  0.914, Loss:  0.112\n",
      "Epoch   6 Batch  127/269 - Train Accuracy:  0.896, Validation Accuracy:  0.913, Loss:  0.110\n",
      "Epoch   6 Batch  128/269 - Train Accuracy:  0.907, Validation Accuracy:  0.920, Loss:  0.109\n",
      "Epoch   6 Batch  129/269 - Train Accuracy:  0.908, Validation Accuracy:  0.922, Loss:  0.109\n",
      "Epoch   6 Batch  130/269 - Train Accuracy:  0.914, Validation Accuracy:  0.919, Loss:  0.116\n",
      "Epoch   6 Batch  131/269 - Train Accuracy:  0.893, Validation Accuracy:  0.914, Loss:  0.111\n",
      "Epoch   6 Batch  132/269 - Train Accuracy:  0.899, Validation Accuracy:  0.911, Loss:  0.116\n",
      "Epoch   6 Batch  133/269 - Train Accuracy:  0.921, Validation Accuracy:  0.908, Loss:  0.102\n",
      "Epoch   6 Batch  134/269 - Train Accuracy:  0.904, Validation Accuracy:  0.915, Loss:  0.110\n",
      "Epoch   6 Batch  135/269 - Train Accuracy:  0.912, Validation Accuracy:  0.914, Loss:  0.117\n",
      "Epoch   6 Batch  136/269 - Train Accuracy:  0.895, Validation Accuracy:  0.906, Loss:  0.115\n",
      "Epoch   6 Batch  137/269 - Train Accuracy:  0.905, Validation Accuracy:  0.914, Loss:  0.120\n",
      "Epoch   6 Batch  138/269 - Train Accuracy:  0.914, Validation Accuracy:  0.919, Loss:  0.106\n",
      "Epoch   6 Batch  139/269 - Train Accuracy:  0.911, Validation Accuracy:  0.923, Loss:  0.101\n",
      "Epoch   6 Batch  140/269 - Train Accuracy:  0.902, Validation Accuracy:  0.925, Loss:  0.116\n",
      "Epoch   6 Batch  141/269 - Train Accuracy:  0.907, Validation Accuracy:  0.924, Loss:  0.113\n",
      "Epoch   6 Batch  142/269 - Train Accuracy:  0.913, Validation Accuracy:  0.924, Loss:  0.103\n",
      "Epoch   6 Batch  143/269 - Train Accuracy:  0.921, Validation Accuracy:  0.917, Loss:  0.103\n",
      "Epoch   6 Batch  144/269 - Train Accuracy:  0.934, Validation Accuracy:  0.915, Loss:  0.093\n",
      "Epoch   6 Batch  145/269 - Train Accuracy:  0.909, Validation Accuracy:  0.911, Loss:  0.104\n",
      "Epoch   6 Batch  146/269 - Train Accuracy:  0.907, Validation Accuracy:  0.912, Loss:  0.109\n",
      "Epoch   6 Batch  147/269 - Train Accuracy:  0.917, Validation Accuracy:  0.913, Loss:  0.105\n",
      "Epoch   6 Batch  148/269 - Train Accuracy:  0.914, Validation Accuracy:  0.922, Loss:  0.109\n",
      "Epoch   6 Batch  149/269 - Train Accuracy:  0.899, Validation Accuracy:  0.919, Loss:  0.114\n",
      "Epoch   6 Batch  150/269 - Train Accuracy:  0.911, Validation Accuracy:  0.920, Loss:  0.105\n",
      "Epoch   6 Batch  151/269 - Train Accuracy:  0.907, Validation Accuracy:  0.916, Loss:  0.102\n",
      "Epoch   6 Batch  152/269 - Train Accuracy:  0.908, Validation Accuracy:  0.921, Loss:  0.108\n",
      "Epoch   6 Batch  153/269 - Train Accuracy:  0.919, Validation Accuracy:  0.923, Loss:  0.105\n",
      "Epoch   6 Batch  154/269 - Train Accuracy:  0.926, Validation Accuracy:  0.924, Loss:  0.107\n",
      "Epoch   6 Batch  155/269 - Train Accuracy:  0.910, Validation Accuracy:  0.925, Loss:  0.103\n",
      "Epoch   6 Batch  156/269 - Train Accuracy:  0.911, Validation Accuracy:  0.926, Loss:  0.109\n",
      "Epoch   6 Batch  157/269 - Train Accuracy:  0.901, Validation Accuracy:  0.922, Loss:  0.098\n",
      "Epoch   6 Batch  158/269 - Train Accuracy:  0.915, Validation Accuracy:  0.921, Loss:  0.106\n",
      "Epoch   6 Batch  159/269 - Train Accuracy:  0.912, Validation Accuracy:  0.920, Loss:  0.105\n",
      "Epoch   6 Batch  160/269 - Train Accuracy:  0.919, Validation Accuracy:  0.916, Loss:  0.106\n",
      "Epoch   6 Batch  161/269 - Train Accuracy:  0.910, Validation Accuracy:  0.916, Loss:  0.103\n",
      "Epoch   6 Batch  162/269 - Train Accuracy:  0.931, Validation Accuracy:  0.915, Loss:  0.101\n",
      "Epoch   6 Batch  163/269 - Train Accuracy:  0.915, Validation Accuracy:  0.919, Loss:  0.101\n",
      "Epoch   6 Batch  164/269 - Train Accuracy:  0.914, Validation Accuracy:  0.913, Loss:  0.102\n",
      "Epoch   6 Batch  165/269 - Train Accuracy:  0.927, Validation Accuracy:  0.912, Loss:  0.102\n",
      "Epoch   6 Batch  166/269 - Train Accuracy:  0.906, Validation Accuracy:  0.915, Loss:  0.102\n",
      "Epoch   6 Batch  167/269 - Train Accuracy:  0.919, Validation Accuracy:  0.923, Loss:  0.102\n",
      "Epoch   6 Batch  168/269 - Train Accuracy:  0.918, Validation Accuracy:  0.921, Loss:  0.105\n",
      "Epoch   6 Batch  169/269 - Train Accuracy:  0.906, Validation Accuracy:  0.923, Loss:  0.101\n",
      "Epoch   6 Batch  170/269 - Train Accuracy:  0.910, Validation Accuracy:  0.922, Loss:  0.101\n",
      "Epoch   6 Batch  171/269 - Train Accuracy:  0.918, Validation Accuracy:  0.928, Loss:  0.103\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   6 Batch  172/269 - Train Accuracy:  0.905, Validation Accuracy:  0.922, Loss:  0.111\n",
      "Epoch   6 Batch  173/269 - Train Accuracy:  0.918, Validation Accuracy:  0.914, Loss:  0.094\n",
      "Epoch   6 Batch  174/269 - Train Accuracy:  0.922, Validation Accuracy:  0.919, Loss:  0.100\n",
      "Epoch   6 Batch  175/269 - Train Accuracy:  0.899, Validation Accuracy:  0.919, Loss:  0.118\n",
      "Epoch   6 Batch  176/269 - Train Accuracy:  0.905, Validation Accuracy:  0.922, Loss:  0.110\n",
      "Epoch   6 Batch  177/269 - Train Accuracy:  0.923, Validation Accuracy:  0.926, Loss:  0.095\n",
      "Epoch   6 Batch  178/269 - Train Accuracy:  0.924, Validation Accuracy:  0.927, Loss:  0.098\n",
      "Epoch   6 Batch  179/269 - Train Accuracy:  0.905, Validation Accuracy:  0.929, Loss:  0.100\n",
      "Epoch   6 Batch  180/269 - Train Accuracy:  0.929, Validation Accuracy:  0.931, Loss:  0.098\n",
      "Epoch   6 Batch  181/269 - Train Accuracy:  0.915, Validation Accuracy:  0.930, Loss:  0.103\n",
      "Epoch   6 Batch  182/269 - Train Accuracy:  0.914, Validation Accuracy:  0.930, Loss:  0.101\n",
      "Epoch   6 Batch  183/269 - Train Accuracy:  0.930, Validation Accuracy:  0.928, Loss:  0.087\n",
      "Epoch   6 Batch  184/269 - Train Accuracy:  0.916, Validation Accuracy:  0.930, Loss:  0.097\n",
      "Epoch   6 Batch  185/269 - Train Accuracy:  0.921, Validation Accuracy:  0.926, Loss:  0.099\n",
      "Epoch   6 Batch  186/269 - Train Accuracy:  0.915, Validation Accuracy:  0.924, Loss:  0.098\n",
      "Epoch   6 Batch  187/269 - Train Accuracy:  0.912, Validation Accuracy:  0.926, Loss:  0.095\n",
      "Epoch   6 Batch  188/269 - Train Accuracy:  0.919, Validation Accuracy:  0.924, Loss:  0.096\n",
      "Epoch   6 Batch  189/269 - Train Accuracy:  0.922, Validation Accuracy:  0.929, Loss:  0.094\n",
      "Epoch   6 Batch  190/269 - Train Accuracy:  0.920, Validation Accuracy:  0.925, Loss:  0.095\n",
      "Epoch   6 Batch  191/269 - Train Accuracy:  0.899, Validation Accuracy:  0.923, Loss:  0.101\n",
      "Epoch   6 Batch  192/269 - Train Accuracy:  0.911, Validation Accuracy:  0.921, Loss:  0.103\n",
      "Epoch   6 Batch  193/269 - Train Accuracy:  0.925, Validation Accuracy:  0.927, Loss:  0.098\n",
      "Epoch   6 Batch  194/269 - Train Accuracy:  0.905, Validation Accuracy:  0.925, Loss:  0.101\n",
      "Epoch   6 Batch  195/269 - Train Accuracy:  0.905, Validation Accuracy:  0.925, Loss:  0.096\n",
      "Epoch   6 Batch  196/269 - Train Accuracy:  0.909, Validation Accuracy:  0.924, Loss:  0.095\n",
      "Epoch   6 Batch  197/269 - Train Accuracy:  0.907, Validation Accuracy:  0.927, Loss:  0.103\n",
      "Epoch   6 Batch  198/269 - Train Accuracy:  0.920, Validation Accuracy:  0.930, Loss:  0.100\n",
      "Epoch   6 Batch  199/269 - Train Accuracy:  0.916, Validation Accuracy:  0.924, Loss:  0.104\n",
      "Epoch   6 Batch  200/269 - Train Accuracy:  0.911, Validation Accuracy:  0.926, Loss:  0.100\n",
      "Epoch   6 Batch  201/269 - Train Accuracy:  0.916, Validation Accuracy:  0.931, Loss:  0.100\n",
      "Epoch   6 Batch  202/269 - Train Accuracy:  0.915, Validation Accuracy:  0.919, Loss:  0.095\n",
      "Epoch   6 Batch  203/269 - Train Accuracy:  0.923, Validation Accuracy:  0.925, Loss:  0.106\n",
      "Epoch   6 Batch  204/269 - Train Accuracy:  0.914, Validation Accuracy:  0.931, Loss:  0.103\n",
      "Epoch   6 Batch  205/269 - Train Accuracy:  0.926, Validation Accuracy:  0.925, Loss:  0.092\n",
      "Epoch   6 Batch  206/269 - Train Accuracy:  0.909, Validation Accuracy:  0.927, Loss:  0.103\n",
      "Epoch   6 Batch  207/269 - Train Accuracy:  0.907, Validation Accuracy:  0.923, Loss:  0.096\n",
      "Epoch   6 Batch  208/269 - Train Accuracy:  0.910, Validation Accuracy:  0.924, Loss:  0.102\n",
      "Epoch   6 Batch  209/269 - Train Accuracy:  0.917, Validation Accuracy:  0.924, Loss:  0.093\n",
      "Epoch   6 Batch  210/269 - Train Accuracy:  0.919, Validation Accuracy:  0.922, Loss:  0.090\n",
      "Epoch   6 Batch  211/269 - Train Accuracy:  0.921, Validation Accuracy:  0.925, Loss:  0.098\n",
      "Epoch   6 Batch  212/269 - Train Accuracy:  0.911, Validation Accuracy:  0.923, Loss:  0.102\n",
      "Epoch   6 Batch  213/269 - Train Accuracy:  0.909, Validation Accuracy:  0.921, Loss:  0.097\n",
      "Epoch   6 Batch  214/269 - Train Accuracy:  0.917, Validation Accuracy:  0.914, Loss:  0.097\n",
      "Epoch   6 Batch  215/269 - Train Accuracy:  0.924, Validation Accuracy:  0.913, Loss:  0.091\n",
      "Epoch   6 Batch  216/269 - Train Accuracy:  0.908, Validation Accuracy:  0.917, Loss:  0.107\n",
      "Epoch   6 Batch  217/269 - Train Accuracy:  0.912, Validation Accuracy:  0.921, Loss:  0.100\n",
      "Epoch   6 Batch  218/269 - Train Accuracy:  0.924, Validation Accuracy:  0.924, Loss:  0.094\n",
      "Epoch   6 Batch  219/269 - Train Accuracy:  0.924, Validation Accuracy:  0.923, Loss:  0.099\n",
      "Epoch   6 Batch  220/269 - Train Accuracy:  0.911, Validation Accuracy:  0.925, Loss:  0.090\n",
      "Epoch   6 Batch  221/269 - Train Accuracy:  0.912, Validation Accuracy:  0.925, Loss:  0.099\n",
      "Epoch   6 Batch  222/269 - Train Accuracy:  0.937, Validation Accuracy:  0.922, Loss:  0.089\n",
      "Epoch   6 Batch  223/269 - Train Accuracy:  0.911, Validation Accuracy:  0.927, Loss:  0.091\n",
      "Epoch   6 Batch  224/269 - Train Accuracy:  0.920, Validation Accuracy:  0.923, Loss:  0.103\n",
      "Epoch   6 Batch  225/269 - Train Accuracy:  0.913, Validation Accuracy:  0.925, Loss:  0.089\n",
      "Epoch   6 Batch  226/269 - Train Accuracy:  0.921, Validation Accuracy:  0.917, Loss:  0.098\n",
      "Epoch   6 Batch  227/269 - Train Accuracy:  0.922, Validation Accuracy:  0.919, Loss:  0.096\n",
      "Epoch   6 Batch  228/269 - Train Accuracy:  0.918, Validation Accuracy:  0.924, Loss:  0.091\n",
      "Epoch   6 Batch  229/269 - Train Accuracy:  0.913, Validation Accuracy:  0.922, Loss:  0.093\n",
      "Epoch   6 Batch  230/269 - Train Accuracy:  0.919, Validation Accuracy:  0.926, Loss:  0.094\n",
      "Epoch   6 Batch  231/269 - Train Accuracy:  0.911, Validation Accuracy:  0.926, Loss:  0.096\n",
      "Epoch   6 Batch  232/269 - Train Accuracy:  0.915, Validation Accuracy:  0.921, Loss:  0.095\n",
      "Epoch   6 Batch  233/269 - Train Accuracy:  0.923, Validation Accuracy:  0.916, Loss:  0.097\n",
      "Epoch   6 Batch  234/269 - Train Accuracy:  0.921, Validation Accuracy:  0.919, Loss:  0.091\n",
      "Epoch   6 Batch  235/269 - Train Accuracy:  0.938, Validation Accuracy:  0.915, Loss:  0.085\n",
      "Epoch   6 Batch  236/269 - Train Accuracy:  0.918, Validation Accuracy:  0.918, Loss:  0.084\n",
      "Epoch   6 Batch  237/269 - Train Accuracy:  0.919, Validation Accuracy:  0.921, Loss:  0.091\n",
      "Epoch   6 Batch  238/269 - Train Accuracy:  0.921, Validation Accuracy:  0.923, Loss:  0.092\n",
      "Epoch   6 Batch  239/269 - Train Accuracy:  0.929, Validation Accuracy:  0.923, Loss:  0.091\n",
      "Epoch   6 Batch  240/269 - Train Accuracy:  0.933, Validation Accuracy:  0.920, Loss:  0.085\n",
      "Epoch   6 Batch  241/269 - Train Accuracy:  0.910, Validation Accuracy:  0.919, Loss:  0.102\n",
      "Epoch   6 Batch  242/269 - Train Accuracy:  0.921, Validation Accuracy:  0.920, Loss:  0.093\n",
      "Epoch   6 Batch  243/269 - Train Accuracy:  0.917, Validation Accuracy:  0.926, Loss:  0.084\n",
      "Epoch   6 Batch  244/269 - Train Accuracy:  0.920, Validation Accuracy:  0.928, Loss:  0.094\n",
      "Epoch   6 Batch  245/269 - Train Accuracy:  0.907, Validation Accuracy:  0.926, Loss:  0.096\n",
      "Epoch   6 Batch  246/269 - Train Accuracy:  0.916, Validation Accuracy:  0.924, Loss:  0.093\n",
      "Epoch   6 Batch  247/269 - Train Accuracy:  0.916, Validation Accuracy:  0.923, Loss:  0.090\n",
      "Epoch   6 Batch  248/269 - Train Accuracy:  0.933, Validation Accuracy:  0.924, Loss:  0.087\n",
      "Epoch   6 Batch  249/269 - Train Accuracy:  0.925, Validation Accuracy:  0.926, Loss:  0.085\n",
      "Epoch   6 Batch  250/269 - Train Accuracy:  0.929, Validation Accuracy:  0.927, Loss:  0.092\n",
      "Epoch   6 Batch  251/269 - Train Accuracy:  0.938, Validation Accuracy:  0.930, Loss:  0.087\n",
      "Epoch   6 Batch  252/269 - Train Accuracy:  0.940, Validation Accuracy:  0.922, Loss:  0.079\n",
      "Epoch   6 Batch  253/269 - Train Accuracy:  0.911, Validation Accuracy:  0.923, Loss:  0.093\n",
      "Epoch   6 Batch  254/269 - Train Accuracy:  0.923, Validation Accuracy:  0.927, Loss:  0.090\n",
      "Epoch   6 Batch  255/269 - Train Accuracy:  0.919, Validation Accuracy:  0.925, Loss:  0.089\n",
      "Epoch   6 Batch  256/269 - Train Accuracy:  0.914, Validation Accuracy:  0.926, Loss:  0.086\n",
      "Epoch   6 Batch  257/269 - Train Accuracy:  0.901, Validation Accuracy:  0.917, Loss:  0.098\n",
      "Epoch   6 Batch  258/269 - Train Accuracy:  0.917, Validation Accuracy:  0.918, Loss:  0.090\n",
      "Epoch   6 Batch  259/269 - Train Accuracy:  0.917, Validation Accuracy:  0.923, Loss:  0.089\n",
      "Epoch   6 Batch  260/269 - Train Accuracy:  0.913, Validation Accuracy:  0.929, Loss:  0.092\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   6 Batch  261/269 - Train Accuracy:  0.922, Validation Accuracy:  0.931, Loss:  0.090\n",
      "Epoch   6 Batch  262/269 - Train Accuracy:  0.926, Validation Accuracy:  0.928, Loss:  0.089\n",
      "Epoch   6 Batch  263/269 - Train Accuracy:  0.913, Validation Accuracy:  0.930, Loss:  0.092\n",
      "Epoch   6 Batch  264/269 - Train Accuracy:  0.897, Validation Accuracy:  0.930, Loss:  0.094\n",
      "Epoch   6 Batch  265/269 - Train Accuracy:  0.913, Validation Accuracy:  0.931, Loss:  0.090\n",
      "Epoch   6 Batch  266/269 - Train Accuracy:  0.917, Validation Accuracy:  0.924, Loss:  0.082\n",
      "Epoch   6 Batch  267/269 - Train Accuracy:  0.928, Validation Accuracy:  0.926, Loss:  0.092\n",
      "Epoch   7 Batch    0/269 - Train Accuracy:  0.934, Validation Accuracy:  0.930, Loss:  0.096\n",
      "Epoch   7 Batch    1/269 - Train Accuracy:  0.922, Validation Accuracy:  0.927, Loss:  0.090\n",
      "Epoch   7 Batch    2/269 - Train Accuracy:  0.925, Validation Accuracy:  0.928, Loss:  0.089\n",
      "Epoch   7 Batch    3/269 - Train Accuracy:  0.918, Validation Accuracy:  0.929, Loss:  0.091\n",
      "Epoch   7 Batch    4/269 - Train Accuracy:  0.905, Validation Accuracy:  0.923, Loss:  0.090\n",
      "Epoch   7 Batch    5/269 - Train Accuracy:  0.931, Validation Accuracy:  0.913, Loss:  0.087\n",
      "Epoch   7 Batch    6/269 - Train Accuracy:  0.934, Validation Accuracy:  0.922, Loss:  0.082\n",
      "Epoch   7 Batch    7/269 - Train Accuracy:  0.933, Validation Accuracy:  0.925, Loss:  0.083\n",
      "Epoch   7 Batch    8/269 - Train Accuracy:  0.928, Validation Accuracy:  0.928, Loss:  0.088\n",
      "Epoch   7 Batch    9/269 - Train Accuracy:  0.916, Validation Accuracy:  0.927, Loss:  0.093\n",
      "Epoch   7 Batch   10/269 - Train Accuracy:  0.930, Validation Accuracy:  0.922, Loss:  0.081\n",
      "Epoch   7 Batch   11/269 - Train Accuracy:  0.919, Validation Accuracy:  0.924, Loss:  0.093\n",
      "Epoch   7 Batch   12/269 - Train Accuracy:  0.917, Validation Accuracy:  0.926, Loss:  0.094\n",
      "Epoch   7 Batch   13/269 - Train Accuracy:  0.935, Validation Accuracy:  0.930, Loss:  0.076\n",
      "Epoch   7 Batch   14/269 - Train Accuracy:  0.922, Validation Accuracy:  0.927, Loss:  0.084\n",
      "Epoch   7 Batch   15/269 - Train Accuracy:  0.932, Validation Accuracy:  0.933, Loss:  0.075\n",
      "Epoch   7 Batch   16/269 - Train Accuracy:  0.921, Validation Accuracy:  0.932, Loss:  0.092\n",
      "Epoch   7 Batch   17/269 - Train Accuracy:  0.936, Validation Accuracy:  0.932, Loss:  0.077\n",
      "Epoch   7 Batch   18/269 - Train Accuracy:  0.913, Validation Accuracy:  0.934, Loss:  0.086\n",
      "Epoch   7 Batch   19/269 - Train Accuracy:  0.929, Validation Accuracy:  0.928, Loss:  0.074\n",
      "Epoch   7 Batch   20/269 - Train Accuracy:  0.934, Validation Accuracy:  0.934, Loss:  0.084\n",
      "Epoch   7 Batch   21/269 - Train Accuracy:  0.912, Validation Accuracy:  0.935, Loss:  0.095\n",
      "Epoch   7 Batch   22/269 - Train Accuracy:  0.927, Validation Accuracy:  0.933, Loss:  0.079\n",
      "Epoch   7 Batch   23/269 - Train Accuracy:  0.925, Validation Accuracy:  0.933, Loss:  0.087\n",
      "Epoch   7 Batch   24/269 - Train Accuracy:  0.921, Validation Accuracy:  0.937, Loss:  0.084\n",
      "Epoch   7 Batch   25/269 - Train Accuracy:  0.914, Validation Accuracy:  0.931, Loss:  0.091\n",
      "Epoch   7 Batch   26/269 - Train Accuracy:  0.923, Validation Accuracy:  0.929, Loss:  0.079\n",
      "Epoch   7 Batch   27/269 - Train Accuracy:  0.920, Validation Accuracy:  0.929, Loss:  0.080\n",
      "Epoch   7 Batch   28/269 - Train Accuracy:  0.898, Validation Accuracy:  0.931, Loss:  0.088\n",
      "Epoch   7 Batch   29/269 - Train Accuracy:  0.923, Validation Accuracy:  0.931, Loss:  0.089\n",
      "Epoch   7 Batch   30/269 - Train Accuracy:  0.922, Validation Accuracy:  0.925, Loss:  0.082\n",
      "Epoch   7 Batch   31/269 - Train Accuracy:  0.938, Validation Accuracy:  0.920, Loss:  0.078\n",
      "Epoch   7 Batch   32/269 - Train Accuracy:  0.924, Validation Accuracy:  0.921, Loss:  0.079\n",
      "Epoch   7 Batch   33/269 - Train Accuracy:  0.931, Validation Accuracy:  0.923, Loss:  0.075\n",
      "Epoch   7 Batch   34/269 - Train Accuracy:  0.916, Validation Accuracy:  0.929, Loss:  0.080\n",
      "Epoch   7 Batch   35/269 - Train Accuracy:  0.921, Validation Accuracy:  0.931, Loss:  0.094\n",
      "Epoch   7 Batch   36/269 - Train Accuracy:  0.912, Validation Accuracy:  0.928, Loss:  0.085\n",
      "Epoch   7 Batch   37/269 - Train Accuracy:  0.913, Validation Accuracy:  0.928, Loss:  0.082\n",
      "Epoch   7 Batch   38/269 - Train Accuracy:  0.921, Validation Accuracy:  0.933, Loss:  0.082\n",
      "Epoch   7 Batch   39/269 - Train Accuracy:  0.930, Validation Accuracy:  0.935, Loss:  0.079\n",
      "Epoch   7 Batch   40/269 - Train Accuracy:  0.913, Validation Accuracy:  0.934, Loss:  0.088\n",
      "Epoch   7 Batch   41/269 - Train Accuracy:  0.920, Validation Accuracy:  0.936, Loss:  0.085\n",
      "Epoch   7 Batch   42/269 - Train Accuracy:  0.934, Validation Accuracy:  0.935, Loss:  0.074\n",
      "Epoch   7 Batch   43/269 - Train Accuracy:  0.917, Validation Accuracy:  0.936, Loss:  0.083\n",
      "Epoch   7 Batch   44/269 - Train Accuracy:  0.923, Validation Accuracy:  0.934, Loss:  0.085\n",
      "Epoch   7 Batch   45/269 - Train Accuracy:  0.922, Validation Accuracy:  0.935, Loss:  0.083\n",
      "Epoch   7 Batch   46/269 - Train Accuracy:  0.926, Validation Accuracy:  0.934, Loss:  0.080\n",
      "Epoch   7 Batch   47/269 - Train Accuracy:  0.934, Validation Accuracy:  0.934, Loss:  0.073\n",
      "Epoch   7 Batch   48/269 - Train Accuracy:  0.932, Validation Accuracy:  0.939, Loss:  0.078\n",
      "Epoch   7 Batch   49/269 - Train Accuracy:  0.925, Validation Accuracy:  0.938, Loss:  0.077\n",
      "Epoch   7 Batch   50/269 - Train Accuracy:  0.908, Validation Accuracy:  0.934, Loss:  0.090\n",
      "Epoch   7 Batch   51/269 - Train Accuracy:  0.925, Validation Accuracy:  0.934, Loss:  0.080\n",
      "Epoch   7 Batch   52/269 - Train Accuracy:  0.929, Validation Accuracy:  0.933, Loss:  0.072\n",
      "Epoch   7 Batch   53/269 - Train Accuracy:  0.923, Validation Accuracy:  0.928, Loss:  0.089\n",
      "Epoch   7 Batch   54/269 - Train Accuracy:  0.927, Validation Accuracy:  0.928, Loss:  0.078\n",
      "Epoch   7 Batch   55/269 - Train Accuracy:  0.932, Validation Accuracy:  0.928, Loss:  0.078\n",
      "Epoch   7 Batch   56/269 - Train Accuracy:  0.910, Validation Accuracy:  0.927, Loss:  0.080\n",
      "Epoch   7 Batch   57/269 - Train Accuracy:  0.919, Validation Accuracy:  0.928, Loss:  0.089\n",
      "Epoch   7 Batch   58/269 - Train Accuracy:  0.932, Validation Accuracy:  0.936, Loss:  0.082\n",
      "Epoch   7 Batch   59/269 - Train Accuracy:  0.942, Validation Accuracy:  0.937, Loss:  0.067\n",
      "Epoch   7 Batch   60/269 - Train Accuracy:  0.925, Validation Accuracy:  0.936, Loss:  0.075\n",
      "Epoch   7 Batch   61/269 - Train Accuracy:  0.930, Validation Accuracy:  0.933, Loss:  0.073\n",
      "Epoch   7 Batch   62/269 - Train Accuracy:  0.915, Validation Accuracy:  0.938, Loss:  0.084\n",
      "Epoch   7 Batch   63/269 - Train Accuracy:  0.920, Validation Accuracy:  0.935, Loss:  0.087\n",
      "Epoch   7 Batch   64/269 - Train Accuracy:  0.933, Validation Accuracy:  0.937, Loss:  0.074\n",
      "Epoch   7 Batch   65/269 - Train Accuracy:  0.916, Validation Accuracy:  0.932, Loss:  0.079\n",
      "Epoch   7 Batch   66/269 - Train Accuracy:  0.924, Validation Accuracy:  0.929, Loss:  0.080\n",
      "Epoch   7 Batch   67/269 - Train Accuracy:  0.927, Validation Accuracy:  0.925, Loss:  0.085\n",
      "Epoch   7 Batch   68/269 - Train Accuracy:  0.927, Validation Accuracy:  0.919, Loss:  0.083\n",
      "Epoch   7 Batch   69/269 - Train Accuracy:  0.907, Validation Accuracy:  0.931, Loss:  0.093\n",
      "Epoch   7 Batch   70/269 - Train Accuracy:  0.936, Validation Accuracy:  0.930, Loss:  0.081\n",
      "Epoch   7 Batch   71/269 - Train Accuracy:  0.928, Validation Accuracy:  0.933, Loss:  0.090\n",
      "Epoch   7 Batch   72/269 - Train Accuracy:  0.925, Validation Accuracy:  0.931, Loss:  0.087\n",
      "Epoch   7 Batch   73/269 - Train Accuracy:  0.915, Validation Accuracy:  0.925, Loss:  0.083\n",
      "Epoch   7 Batch   74/269 - Train Accuracy:  0.924, Validation Accuracy:  0.932, Loss:  0.078\n",
      "Epoch   7 Batch   75/269 - Train Accuracy:  0.941, Validation Accuracy:  0.934, Loss:  0.079\n",
      "Epoch   7 Batch   76/269 - Train Accuracy:  0.923, Validation Accuracy:  0.933, Loss:  0.076\n",
      "Epoch   7 Batch   77/269 - Train Accuracy:  0.925, Validation Accuracy:  0.932, Loss:  0.077\n",
      "Epoch   7 Batch   78/269 - Train Accuracy:  0.934, Validation Accuracy:  0.929, Loss:  0.079\n",
      "Epoch   7 Batch   79/269 - Train Accuracy:  0.922, Validation Accuracy:  0.927, Loss:  0.078\n",
      "Epoch   7 Batch   80/269 - Train Accuracy:  0.927, Validation Accuracy:  0.925, Loss:  0.075\n",
      "Epoch   7 Batch   81/269 - Train Accuracy:  0.923, Validation Accuracy:  0.924, Loss:  0.087\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   7 Batch   82/269 - Train Accuracy:  0.940, Validation Accuracy:  0.922, Loss:  0.070\n",
      "Epoch   7 Batch   83/269 - Train Accuracy:  0.909, Validation Accuracy:  0.929, Loss:  0.088\n",
      "Epoch   7 Batch   84/269 - Train Accuracy:  0.929, Validation Accuracy:  0.932, Loss:  0.077\n",
      "Epoch   7 Batch   85/269 - Train Accuracy:  0.926, Validation Accuracy:  0.932, Loss:  0.077\n",
      "Epoch   7 Batch   86/269 - Train Accuracy:  0.933, Validation Accuracy:  0.932, Loss:  0.073\n",
      "Epoch   7 Batch   87/269 - Train Accuracy:  0.917, Validation Accuracy:  0.928, Loss:  0.084\n",
      "Epoch   7 Batch   88/269 - Train Accuracy:  0.912, Validation Accuracy:  0.928, Loss:  0.080\n",
      "Epoch   7 Batch   89/269 - Train Accuracy:  0.938, Validation Accuracy:  0.932, Loss:  0.075\n",
      "Epoch   7 Batch   90/269 - Train Accuracy:  0.931, Validation Accuracy:  0.932, Loss:  0.080\n",
      "Epoch   7 Batch   91/269 - Train Accuracy:  0.940, Validation Accuracy:  0.932, Loss:  0.070\n",
      "Epoch   7 Batch   92/269 - Train Accuracy:  0.938, Validation Accuracy:  0.933, Loss:  0.070\n",
      "Epoch   7 Batch   93/269 - Train Accuracy:  0.935, Validation Accuracy:  0.933, Loss:  0.073\n",
      "Epoch   7 Batch   94/269 - Train Accuracy:  0.923, Validation Accuracy:  0.933, Loss:  0.083\n",
      "Epoch   7 Batch   95/269 - Train Accuracy:  0.931, Validation Accuracy:  0.934, Loss:  0.076\n",
      "Epoch   7 Batch   96/269 - Train Accuracy:  0.914, Validation Accuracy:  0.930, Loss:  0.078\n",
      "Epoch   7 Batch   97/269 - Train Accuracy:  0.927, Validation Accuracy:  0.927, Loss:  0.080\n",
      "Epoch   7 Batch   98/269 - Train Accuracy:  0.927, Validation Accuracy:  0.926, Loss:  0.074\n",
      "Epoch   7 Batch   99/269 - Train Accuracy:  0.931, Validation Accuracy:  0.930, Loss:  0.078\n",
      "Epoch   7 Batch  100/269 - Train Accuracy:  0.925, Validation Accuracy:  0.936, Loss:  0.075\n",
      "Epoch   7 Batch  101/269 - Train Accuracy:  0.926, Validation Accuracy:  0.939, Loss:  0.087\n",
      "Epoch   7 Batch  102/269 - Train Accuracy:  0.933, Validation Accuracy:  0.932, Loss:  0.075\n",
      "Epoch   7 Batch  103/269 - Train Accuracy:  0.931, Validation Accuracy:  0.936, Loss:  0.082\n",
      "Epoch   7 Batch  104/269 - Train Accuracy:  0.924, Validation Accuracy:  0.933, Loss:  0.075\n",
      "Epoch   7 Batch  105/269 - Train Accuracy:  0.927, Validation Accuracy:  0.934, Loss:  0.076\n",
      "Epoch   7 Batch  106/269 - Train Accuracy:  0.937, Validation Accuracy:  0.937, Loss:  0.070\n",
      "Epoch   7 Batch  107/269 - Train Accuracy:  0.934, Validation Accuracy:  0.936, Loss:  0.078\n",
      "Epoch   7 Batch  108/269 - Train Accuracy:  0.943, Validation Accuracy:  0.935, Loss:  0.075\n",
      "Epoch   7 Batch  109/269 - Train Accuracy:  0.914, Validation Accuracy:  0.932, Loss:  0.081\n",
      "Epoch   7 Batch  110/269 - Train Accuracy:  0.937, Validation Accuracy:  0.933, Loss:  0.070\n",
      "Epoch   7 Batch  111/269 - Train Accuracy:  0.928, Validation Accuracy:  0.934, Loss:  0.079\n",
      "Epoch   7 Batch  112/269 - Train Accuracy:  0.926, Validation Accuracy:  0.929, Loss:  0.078\n",
      "Epoch   7 Batch  113/269 - Train Accuracy:  0.923, Validation Accuracy:  0.931, Loss:  0.075\n",
      "Epoch   7 Batch  114/269 - Train Accuracy:  0.925, Validation Accuracy:  0.934, Loss:  0.076\n",
      "Epoch   7 Batch  115/269 - Train Accuracy:  0.931, Validation Accuracy:  0.936, Loss:  0.073\n",
      "Epoch   7 Batch  116/269 - Train Accuracy:  0.941, Validation Accuracy:  0.933, Loss:  0.078\n",
      "Epoch   7 Batch  117/269 - Train Accuracy:  0.934, Validation Accuracy:  0.937, Loss:  0.069\n",
      "Epoch   7 Batch  118/269 - Train Accuracy:  0.929, Validation Accuracy:  0.936, Loss:  0.068\n",
      "Epoch   7 Batch  119/269 - Train Accuracy:  0.918, Validation Accuracy:  0.935, Loss:  0.080\n",
      "Epoch   7 Batch  120/269 - Train Accuracy:  0.940, Validation Accuracy:  0.934, Loss:  0.074\n",
      "Epoch   7 Batch  121/269 - Train Accuracy:  0.935, Validation Accuracy:  0.931, Loss:  0.071\n",
      "Epoch   7 Batch  122/269 - Train Accuracy:  0.936, Validation Accuracy:  0.929, Loss:  0.070\n",
      "Epoch   7 Batch  123/269 - Train Accuracy:  0.926, Validation Accuracy:  0.932, Loss:  0.072\n",
      "Epoch   7 Batch  124/269 - Train Accuracy:  0.935, Validation Accuracy:  0.929, Loss:  0.067\n",
      "Epoch   7 Batch  125/269 - Train Accuracy:  0.938, Validation Accuracy:  0.936, Loss:  0.067\n",
      "Epoch   7 Batch  126/269 - Train Accuracy:  0.910, Validation Accuracy:  0.937, Loss:  0.074\n",
      "Epoch   7 Batch  127/269 - Train Accuracy:  0.928, Validation Accuracy:  0.936, Loss:  0.073\n",
      "Epoch   7 Batch  128/269 - Train Accuracy:  0.936, Validation Accuracy:  0.930, Loss:  0.070\n",
      "Epoch   7 Batch  129/269 - Train Accuracy:  0.924, Validation Accuracy:  0.929, Loss:  0.073\n",
      "Epoch   7 Batch  130/269 - Train Accuracy:  0.933, Validation Accuracy:  0.935, Loss:  0.079\n",
      "Epoch   7 Batch  131/269 - Train Accuracy:  0.921, Validation Accuracy:  0.935, Loss:  0.074\n",
      "Epoch   7 Batch  132/269 - Train Accuracy:  0.911, Validation Accuracy:  0.929, Loss:  0.079\n",
      "Epoch   7 Batch  133/269 - Train Accuracy:  0.937, Validation Accuracy:  0.926, Loss:  0.068\n",
      "Epoch   7 Batch  134/269 - Train Accuracy:  0.923, Validation Accuracy:  0.928, Loss:  0.072\n",
      "Epoch   7 Batch  135/269 - Train Accuracy:  0.921, Validation Accuracy:  0.929, Loss:  0.079\n",
      "Epoch   7 Batch  136/269 - Train Accuracy:  0.909, Validation Accuracy:  0.928, Loss:  0.079\n",
      "Epoch   7 Batch  137/269 - Train Accuracy:  0.920, Validation Accuracy:  0.933, Loss:  0.081\n",
      "Epoch   7 Batch  138/269 - Train Accuracy:  0.931, Validation Accuracy:  0.933, Loss:  0.069\n",
      "Epoch   7 Batch  139/269 - Train Accuracy:  0.936, Validation Accuracy:  0.935, Loss:  0.066\n",
      "Epoch   7 Batch  140/269 - Train Accuracy:  0.929, Validation Accuracy:  0.935, Loss:  0.078\n",
      "Epoch   7 Batch  141/269 - Train Accuracy:  0.924, Validation Accuracy:  0.936, Loss:  0.076\n",
      "Epoch   7 Batch  142/269 - Train Accuracy:  0.927, Validation Accuracy:  0.936, Loss:  0.069\n",
      "Epoch   7 Batch  143/269 - Train Accuracy:  0.936, Validation Accuracy:  0.936, Loss:  0.067\n",
      "Epoch   7 Batch  144/269 - Train Accuracy:  0.947, Validation Accuracy:  0.934, Loss:  0.060\n",
      "Epoch   7 Batch  145/269 - Train Accuracy:  0.933, Validation Accuracy:  0.941, Loss:  0.068\n",
      "Epoch   7 Batch  146/269 - Train Accuracy:  0.926, Validation Accuracy:  0.934, Loss:  0.073\n",
      "Epoch   7 Batch  147/269 - Train Accuracy:  0.939, Validation Accuracy:  0.930, Loss:  0.074\n",
      "Epoch   7 Batch  148/269 - Train Accuracy:  0.932, Validation Accuracy:  0.926, Loss:  0.074\n",
      "Epoch   7 Batch  149/269 - Train Accuracy:  0.916, Validation Accuracy:  0.930, Loss:  0.080\n",
      "Epoch   7 Batch  150/269 - Train Accuracy:  0.931, Validation Accuracy:  0.937, Loss:  0.072\n",
      "Epoch   7 Batch  151/269 - Train Accuracy:  0.929, Validation Accuracy:  0.937, Loss:  0.070\n",
      "Epoch   7 Batch  152/269 - Train Accuracy:  0.918, Validation Accuracy:  0.935, Loss:  0.074\n",
      "Epoch   7 Batch  153/269 - Train Accuracy:  0.942, Validation Accuracy:  0.933, Loss:  0.071\n",
      "Epoch   7 Batch  154/269 - Train Accuracy:  0.941, Validation Accuracy:  0.935, Loss:  0.072\n",
      "Epoch   7 Batch  155/269 - Train Accuracy:  0.928, Validation Accuracy:  0.941, Loss:  0.070\n",
      "Epoch   7 Batch  156/269 - Train Accuracy:  0.930, Validation Accuracy:  0.941, Loss:  0.072\n",
      "Epoch   7 Batch  157/269 - Train Accuracy:  0.919, Validation Accuracy:  0.937, Loss:  0.065\n",
      "Epoch   7 Batch  158/269 - Train Accuracy:  0.932, Validation Accuracy:  0.938, Loss:  0.071\n",
      "Epoch   7 Batch  159/269 - Train Accuracy:  0.931, Validation Accuracy:  0.927, Loss:  0.074\n",
      "Epoch   7 Batch  160/269 - Train Accuracy:  0.935, Validation Accuracy:  0.930, Loss:  0.071\n",
      "Epoch   7 Batch  161/269 - Train Accuracy:  0.929, Validation Accuracy:  0.938, Loss:  0.069\n",
      "Epoch   7 Batch  162/269 - Train Accuracy:  0.941, Validation Accuracy:  0.940, Loss:  0.068\n",
      "Epoch   7 Batch  163/269 - Train Accuracy:  0.940, Validation Accuracy:  0.938, Loss:  0.066\n",
      "Epoch   7 Batch  164/269 - Train Accuracy:  0.938, Validation Accuracy:  0.937, Loss:  0.069\n",
      "Epoch   7 Batch  165/269 - Train Accuracy:  0.937, Validation Accuracy:  0.929, Loss:  0.067\n",
      "Epoch   7 Batch  166/269 - Train Accuracy:  0.933, Validation Accuracy:  0.925, Loss:  0.071\n",
      "Epoch   7 Batch  167/269 - Train Accuracy:  0.937, Validation Accuracy:  0.934, Loss:  0.068\n",
      "Epoch   7 Batch  168/269 - Train Accuracy:  0.941, Validation Accuracy:  0.934, Loss:  0.071\n",
      "Epoch   7 Batch  169/269 - Train Accuracy:  0.921, Validation Accuracy:  0.937, Loss:  0.068\n",
      "Epoch   7 Batch  170/269 - Train Accuracy:  0.928, Validation Accuracy:  0.937, Loss:  0.066\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   7 Batch  171/269 - Train Accuracy:  0.933, Validation Accuracy:  0.937, Loss:  0.070\n",
      "Epoch   7 Batch  172/269 - Train Accuracy:  0.928, Validation Accuracy:  0.942, Loss:  0.077\n",
      "Epoch   7 Batch  173/269 - Train Accuracy:  0.934, Validation Accuracy:  0.943, Loss:  0.064\n",
      "Epoch   7 Batch  174/269 - Train Accuracy:  0.939, Validation Accuracy:  0.934, Loss:  0.068\n",
      "Epoch   7 Batch  175/269 - Train Accuracy:  0.923, Validation Accuracy:  0.939, Loss:  0.084\n",
      "Epoch   7 Batch  176/269 - Train Accuracy:  0.918, Validation Accuracy:  0.927, Loss:  0.076\n",
      "Epoch   7 Batch  177/269 - Train Accuracy:  0.935, Validation Accuracy:  0.929, Loss:  0.064\n",
      "Epoch   7 Batch  178/269 - Train Accuracy:  0.947, Validation Accuracy:  0.938, Loss:  0.065\n",
      "Epoch   7 Batch  179/269 - Train Accuracy:  0.922, Validation Accuracy:  0.940, Loss:  0.068\n",
      "Epoch   7 Batch  180/269 - Train Accuracy:  0.939, Validation Accuracy:  0.942, Loss:  0.066\n",
      "Epoch   7 Batch  181/269 - Train Accuracy:  0.927, Validation Accuracy:  0.944, Loss:  0.071\n",
      "Epoch   7 Batch  182/269 - Train Accuracy:  0.929, Validation Accuracy:  0.934, Loss:  0.068\n",
      "Epoch   7 Batch  183/269 - Train Accuracy:  0.934, Validation Accuracy:  0.935, Loss:  0.059\n",
      "Epoch   7 Batch  184/269 - Train Accuracy:  0.930, Validation Accuracy:  0.940, Loss:  0.065\n",
      "Epoch   7 Batch  185/269 - Train Accuracy:  0.937, Validation Accuracy:  0.943, Loss:  0.069\n",
      "Epoch   7 Batch  186/269 - Train Accuracy:  0.938, Validation Accuracy:  0.939, Loss:  0.066\n",
      "Epoch   7 Batch  187/269 - Train Accuracy:  0.933, Validation Accuracy:  0.936, Loss:  0.066\n",
      "Epoch   7 Batch  188/269 - Train Accuracy:  0.925, Validation Accuracy:  0.941, Loss:  0.065\n",
      "Epoch   7 Batch  189/269 - Train Accuracy:  0.940, Validation Accuracy:  0.937, Loss:  0.064\n",
      "Epoch   7 Batch  190/269 - Train Accuracy:  0.938, Validation Accuracy:  0.937, Loss:  0.064\n",
      "Epoch   7 Batch  191/269 - Train Accuracy:  0.922, Validation Accuracy:  0.941, Loss:  0.067\n",
      "Epoch   7 Batch  192/269 - Train Accuracy:  0.929, Validation Accuracy:  0.937, Loss:  0.070\n",
      "Epoch   7 Batch  193/269 - Train Accuracy:  0.935, Validation Accuracy:  0.944, Loss:  0.066\n",
      "Epoch   7 Batch  194/269 - Train Accuracy:  0.922, Validation Accuracy:  0.948, Loss:  0.069\n",
      "Epoch   7 Batch  195/269 - Train Accuracy:  0.929, Validation Accuracy:  0.939, Loss:  0.065\n",
      "Epoch   7 Batch  196/269 - Train Accuracy:  0.923, Validation Accuracy:  0.935, Loss:  0.064\n",
      "Epoch   7 Batch  197/269 - Train Accuracy:  0.921, Validation Accuracy:  0.938, Loss:  0.070\n",
      "Epoch   7 Batch  198/269 - Train Accuracy:  0.939, Validation Accuracy:  0.941, Loss:  0.070\n",
      "Epoch   7 Batch  199/269 - Train Accuracy:  0.924, Validation Accuracy:  0.938, Loss:  0.071\n",
      "Epoch   7 Batch  200/269 - Train Accuracy:  0.936, Validation Accuracy:  0.939, Loss:  0.070\n",
      "Epoch   7 Batch  201/269 - Train Accuracy:  0.936, Validation Accuracy:  0.936, Loss:  0.068\n",
      "Epoch   7 Batch  202/269 - Train Accuracy:  0.934, Validation Accuracy:  0.932, Loss:  0.066\n",
      "Epoch   7 Batch  203/269 - Train Accuracy:  0.929, Validation Accuracy:  0.931, Loss:  0.072\n",
      "Epoch   7 Batch  204/269 - Train Accuracy:  0.935, Validation Accuracy:  0.939, Loss:  0.071\n",
      "Epoch   7 Batch  205/269 - Train Accuracy:  0.947, Validation Accuracy:  0.944, Loss:  0.061\n",
      "Epoch   7 Batch  206/269 - Train Accuracy:  0.930, Validation Accuracy:  0.939, Loss:  0.071\n",
      "Epoch   7 Batch  207/269 - Train Accuracy:  0.934, Validation Accuracy:  0.940, Loss:  0.068\n",
      "Epoch   7 Batch  208/269 - Train Accuracy:  0.937, Validation Accuracy:  0.938, Loss:  0.069\n",
      "Epoch   7 Batch  209/269 - Train Accuracy:  0.944, Validation Accuracy:  0.937, Loss:  0.062\n",
      "Epoch   7 Batch  210/269 - Train Accuracy:  0.941, Validation Accuracy:  0.936, Loss:  0.062\n",
      "Epoch   7 Batch  211/269 - Train Accuracy:  0.935, Validation Accuracy:  0.938, Loss:  0.069\n",
      "Epoch   7 Batch  212/269 - Train Accuracy:  0.925, Validation Accuracy:  0.940, Loss:  0.072\n",
      "Epoch   7 Batch  213/269 - Train Accuracy:  0.920, Validation Accuracy:  0.939, Loss:  0.067\n",
      "Epoch   7 Batch  214/269 - Train Accuracy:  0.935, Validation Accuracy:  0.937, Loss:  0.067\n",
      "Epoch   7 Batch  215/269 - Train Accuracy:  0.938, Validation Accuracy:  0.938, Loss:  0.062\n",
      "Epoch   7 Batch  216/269 - Train Accuracy:  0.918, Validation Accuracy:  0.936, Loss:  0.076\n",
      "Epoch   7 Batch  217/269 - Train Accuracy:  0.928, Validation Accuracy:  0.936, Loss:  0.068\n",
      "Epoch   7 Batch  218/269 - Train Accuracy:  0.938, Validation Accuracy:  0.943, Loss:  0.064\n",
      "Epoch   7 Batch  219/269 - Train Accuracy:  0.935, Validation Accuracy:  0.943, Loss:  0.067\n",
      "Epoch   7 Batch  220/269 - Train Accuracy:  0.926, Validation Accuracy:  0.939, Loss:  0.063\n",
      "Epoch   7 Batch  221/269 - Train Accuracy:  0.935, Validation Accuracy:  0.936, Loss:  0.070\n",
      "Epoch   7 Batch  222/269 - Train Accuracy:  0.952, Validation Accuracy:  0.929, Loss:  0.062\n",
      "Epoch   7 Batch  223/269 - Train Accuracy:  0.933, Validation Accuracy:  0.928, Loss:  0.063\n",
      "Epoch   7 Batch  224/269 - Train Accuracy:  0.935, Validation Accuracy:  0.940, Loss:  0.074\n",
      "Epoch   7 Batch  225/269 - Train Accuracy:  0.923, Validation Accuracy:  0.944, Loss:  0.061\n",
      "Epoch   7 Batch  226/269 - Train Accuracy:  0.935, Validation Accuracy:  0.938, Loss:  0.069\n",
      "Epoch   7 Batch  227/269 - Train Accuracy:  0.947, Validation Accuracy:  0.931, Loss:  0.070\n",
      "Epoch   7 Batch  228/269 - Train Accuracy:  0.931, Validation Accuracy:  0.938, Loss:  0.062\n",
      "Epoch   7 Batch  229/269 - Train Accuracy:  0.929, Validation Accuracy:  0.945, Loss:  0.065\n",
      "Epoch   7 Batch  230/269 - Train Accuracy:  0.940, Validation Accuracy:  0.945, Loss:  0.064\n",
      "Epoch   7 Batch  231/269 - Train Accuracy:  0.927, Validation Accuracy:  0.945, Loss:  0.066\n",
      "Epoch   7 Batch  232/269 - Train Accuracy:  0.931, Validation Accuracy:  0.943, Loss:  0.065\n",
      "Epoch   7 Batch  233/269 - Train Accuracy:  0.938, Validation Accuracy:  0.939, Loss:  0.069\n",
      "Epoch   7 Batch  234/269 - Train Accuracy:  0.938, Validation Accuracy:  0.943, Loss:  0.064\n",
      "Epoch   7 Batch  235/269 - Train Accuracy:  0.951, Validation Accuracy:  0.934, Loss:  0.058\n",
      "Epoch   7 Batch  236/269 - Train Accuracy:  0.943, Validation Accuracy:  0.936, Loss:  0.060\n",
      "Epoch   7 Batch  237/269 - Train Accuracy:  0.937, Validation Accuracy:  0.937, Loss:  0.062\n",
      "Epoch   7 Batch  238/269 - Train Accuracy:  0.942, Validation Accuracy:  0.940, Loss:  0.064\n",
      "Epoch   7 Batch  239/269 - Train Accuracy:  0.933, Validation Accuracy:  0.938, Loss:  0.064\n",
      "Epoch   7 Batch  240/269 - Train Accuracy:  0.943, Validation Accuracy:  0.937, Loss:  0.059\n",
      "Epoch   7 Batch  241/269 - Train Accuracy:  0.931, Validation Accuracy:  0.936, Loss:  0.074\n",
      "Epoch   7 Batch  242/269 - Train Accuracy:  0.942, Validation Accuracy:  0.936, Loss:  0.063\n",
      "Epoch   7 Batch  243/269 - Train Accuracy:  0.942, Validation Accuracy:  0.940, Loss:  0.056\n",
      "Epoch   7 Batch  244/269 - Train Accuracy:  0.932, Validation Accuracy:  0.942, Loss:  0.067\n",
      "Epoch   7 Batch  245/269 - Train Accuracy:  0.924, Validation Accuracy:  0.944, Loss:  0.066\n",
      "Epoch   7 Batch  246/269 - Train Accuracy:  0.932, Validation Accuracy:  0.940, Loss:  0.065\n",
      "Epoch   7 Batch  247/269 - Train Accuracy:  0.941, Validation Accuracy:  0.939, Loss:  0.062\n",
      "Epoch   7 Batch  248/269 - Train Accuracy:  0.948, Validation Accuracy:  0.933, Loss:  0.058\n",
      "Epoch   7 Batch  249/269 - Train Accuracy:  0.939, Validation Accuracy:  0.929, Loss:  0.058\n",
      "Epoch   7 Batch  250/269 - Train Accuracy:  0.941, Validation Accuracy:  0.942, Loss:  0.064\n",
      "Epoch   7 Batch  251/269 - Train Accuracy:  0.952, Validation Accuracy:  0.941, Loss:  0.059\n",
      "Epoch   7 Batch  252/269 - Train Accuracy:  0.955, Validation Accuracy:  0.941, Loss:  0.053\n",
      "Epoch   7 Batch  253/269 - Train Accuracy:  0.927, Validation Accuracy:  0.940, Loss:  0.067\n",
      "Epoch   7 Batch  254/269 - Train Accuracy:  0.932, Validation Accuracy:  0.933, Loss:  0.063\n",
      "Epoch   7 Batch  255/269 - Train Accuracy:  0.933, Validation Accuracy:  0.944, Loss:  0.063\n",
      "Epoch   7 Batch  256/269 - Train Accuracy:  0.932, Validation Accuracy:  0.942, Loss:  0.060\n",
      "Epoch   7 Batch  257/269 - Train Accuracy:  0.924, Validation Accuracy:  0.937, Loss:  0.069\n",
      "Epoch   7 Batch  258/269 - Train Accuracy:  0.933, Validation Accuracy:  0.933, Loss:  0.064\n",
      "Epoch   7 Batch  259/269 - Train Accuracy:  0.934, Validation Accuracy:  0.932, Loss:  0.062\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   7 Batch  260/269 - Train Accuracy:  0.930, Validation Accuracy:  0.936, Loss:  0.065\n",
      "Epoch   7 Batch  261/269 - Train Accuracy:  0.942, Validation Accuracy:  0.945, Loss:  0.062\n",
      "Epoch   7 Batch  262/269 - Train Accuracy:  0.936, Validation Accuracy:  0.943, Loss:  0.063\n",
      "Epoch   7 Batch  263/269 - Train Accuracy:  0.934, Validation Accuracy:  0.940, Loss:  0.064\n",
      "Epoch   7 Batch  264/269 - Train Accuracy:  0.918, Validation Accuracy:  0.940, Loss:  0.067\n",
      "Epoch   7 Batch  265/269 - Train Accuracy:  0.926, Validation Accuracy:  0.939, Loss:  0.064\n",
      "Epoch   7 Batch  266/269 - Train Accuracy:  0.935, Validation Accuracy:  0.944, Loss:  0.056\n",
      "Epoch   7 Batch  267/269 - Train Accuracy:  0.945, Validation Accuracy:  0.947, Loss:  0.066\n",
      "Epoch   8 Batch    0/269 - Train Accuracy:  0.943, Validation Accuracy:  0.949, Loss:  0.068\n",
      "Epoch   8 Batch    1/269 - Train Accuracy:  0.934, Validation Accuracy:  0.943, Loss:  0.062\n",
      "Epoch   8 Batch    2/269 - Train Accuracy:  0.939, Validation Accuracy:  0.943, Loss:  0.062\n",
      "Epoch   8 Batch    3/269 - Train Accuracy:  0.936, Validation Accuracy:  0.944, Loss:  0.062\n",
      "Epoch   8 Batch    4/269 - Train Accuracy:  0.924, Validation Accuracy:  0.944, Loss:  0.062\n",
      "Epoch   8 Batch    5/269 - Train Accuracy:  0.951, Validation Accuracy:  0.935, Loss:  0.059\n",
      "Epoch   8 Batch    6/269 - Train Accuracy:  0.943, Validation Accuracy:  0.930, Loss:  0.056\n",
      "Epoch   8 Batch    7/269 - Train Accuracy:  0.939, Validation Accuracy:  0.936, Loss:  0.059\n",
      "Epoch   8 Batch    8/269 - Train Accuracy:  0.943, Validation Accuracy:  0.944, Loss:  0.062\n",
      "Epoch   8 Batch    9/269 - Train Accuracy:  0.929, Validation Accuracy:  0.939, Loss:  0.066\n",
      "Epoch   8 Batch   10/269 - Train Accuracy:  0.942, Validation Accuracy:  0.938, Loss:  0.056\n",
      "Epoch   8 Batch   11/269 - Train Accuracy:  0.938, Validation Accuracy:  0.940, Loss:  0.065\n",
      "Epoch   8 Batch   12/269 - Train Accuracy:  0.931, Validation Accuracy:  0.936, Loss:  0.066\n",
      "Epoch   8 Batch   13/269 - Train Accuracy:  0.946, Validation Accuracy:  0.938, Loss:  0.053\n",
      "Epoch   8 Batch   14/269 - Train Accuracy:  0.935, Validation Accuracy:  0.941, Loss:  0.059\n",
      "Epoch   8 Batch   15/269 - Train Accuracy:  0.946, Validation Accuracy:  0.941, Loss:  0.051\n",
      "Epoch   8 Batch   16/269 - Train Accuracy:  0.936, Validation Accuracy:  0.945, Loss:  0.066\n",
      "Epoch   8 Batch   17/269 - Train Accuracy:  0.943, Validation Accuracy:  0.940, Loss:  0.053\n",
      "Epoch   8 Batch   18/269 - Train Accuracy:  0.935, Validation Accuracy:  0.934, Loss:  0.062\n",
      "Epoch   8 Batch   19/269 - Train Accuracy:  0.941, Validation Accuracy:  0.932, Loss:  0.052\n",
      "Epoch   8 Batch   20/269 - Train Accuracy:  0.948, Validation Accuracy:  0.941, Loss:  0.059\n",
      "Epoch   8 Batch   21/269 - Train Accuracy:  0.925, Validation Accuracy:  0.945, Loss:  0.067\n",
      "Epoch   8 Batch   22/269 - Train Accuracy:  0.937, Validation Accuracy:  0.942, Loss:  0.056\n",
      "Epoch   8 Batch   23/269 - Train Accuracy:  0.932, Validation Accuracy:  0.944, Loss:  0.063\n",
      "Epoch   8 Batch   24/269 - Train Accuracy:  0.934, Validation Accuracy:  0.944, Loss:  0.061\n",
      "Epoch   8 Batch   25/269 - Train Accuracy:  0.932, Validation Accuracy:  0.932, Loss:  0.065\n",
      "Epoch   8 Batch   26/269 - Train Accuracy:  0.930, Validation Accuracy:  0.934, Loss:  0.057\n",
      "Epoch   8 Batch   27/269 - Train Accuracy:  0.937, Validation Accuracy:  0.944, Loss:  0.057\n",
      "Epoch   8 Batch   28/269 - Train Accuracy:  0.914, Validation Accuracy:  0.934, Loss:  0.064\n",
      "Epoch   8 Batch   29/269 - Train Accuracy:  0.931, Validation Accuracy:  0.935, Loss:  0.063\n",
      "Epoch   8 Batch   30/269 - Train Accuracy:  0.943, Validation Accuracy:  0.936, Loss:  0.060\n",
      "Epoch   8 Batch   31/269 - Train Accuracy:  0.954, Validation Accuracy:  0.930, Loss:  0.056\n",
      "Epoch   8 Batch   32/269 - Train Accuracy:  0.941, Validation Accuracy:  0.919, Loss:  0.056\n",
      "Epoch   8 Batch   33/269 - Train Accuracy:  0.941, Validation Accuracy:  0.936, Loss:  0.055\n",
      "Epoch   8 Batch   34/269 - Train Accuracy:  0.935, Validation Accuracy:  0.946, Loss:  0.057\n",
      "Epoch   8 Batch   35/269 - Train Accuracy:  0.934, Validation Accuracy:  0.945, Loss:  0.069\n",
      "Epoch   8 Batch   36/269 - Train Accuracy:  0.929, Validation Accuracy:  0.940, Loss:  0.061\n",
      "Epoch   8 Batch   37/269 - Train Accuracy:  0.925, Validation Accuracy:  0.939, Loss:  0.062\n",
      "Epoch   8 Batch   38/269 - Train Accuracy:  0.934, Validation Accuracy:  0.941, Loss:  0.059\n",
      "Epoch   8 Batch   39/269 - Train Accuracy:  0.938, Validation Accuracy:  0.942, Loss:  0.057\n",
      "Epoch   8 Batch   40/269 - Train Accuracy:  0.927, Validation Accuracy:  0.942, Loss:  0.063\n",
      "Epoch   8 Batch   41/269 - Train Accuracy:  0.929, Validation Accuracy:  0.943, Loss:  0.061\n",
      "Epoch   8 Batch   42/269 - Train Accuracy:  0.947, Validation Accuracy:  0.945, Loss:  0.053\n",
      "Epoch   8 Batch   43/269 - Train Accuracy:  0.931, Validation Accuracy:  0.940, Loss:  0.059\n",
      "Epoch   8 Batch   44/269 - Train Accuracy:  0.940, Validation Accuracy:  0.937, Loss:  0.060\n",
      "Epoch   8 Batch   45/269 - Train Accuracy:  0.930, Validation Accuracy:  0.941, Loss:  0.060\n",
      "Epoch   8 Batch   46/269 - Train Accuracy:  0.938, Validation Accuracy:  0.942, Loss:  0.056\n",
      "Epoch   8 Batch   47/269 - Train Accuracy:  0.946, Validation Accuracy:  0.945, Loss:  0.052\n",
      "Epoch   8 Batch   48/269 - Train Accuracy:  0.942, Validation Accuracy:  0.947, Loss:  0.055\n",
      "Epoch   8 Batch   49/269 - Train Accuracy:  0.944, Validation Accuracy:  0.943, Loss:  0.055\n",
      "Epoch   8 Batch   50/269 - Train Accuracy:  0.925, Validation Accuracy:  0.941, Loss:  0.067\n",
      "Epoch   8 Batch   51/269 - Train Accuracy:  0.935, Validation Accuracy:  0.942, Loss:  0.056\n",
      "Epoch   8 Batch   52/269 - Train Accuracy:  0.948, Validation Accuracy:  0.942, Loss:  0.049\n",
      "Epoch   8 Batch   53/269 - Train Accuracy:  0.940, Validation Accuracy:  0.946, Loss:  0.063\n",
      "Epoch   8 Batch   54/269 - Train Accuracy:  0.939, Validation Accuracy:  0.946, Loss:  0.055\n",
      "Epoch   8 Batch   55/269 - Train Accuracy:  0.940, Validation Accuracy:  0.943, Loss:  0.056\n",
      "Epoch   8 Batch   56/269 - Train Accuracy:  0.926, Validation Accuracy:  0.939, Loss:  0.058\n",
      "Epoch   8 Batch   57/269 - Train Accuracy:  0.939, Validation Accuracy:  0.936, Loss:  0.063\n",
      "Epoch   8 Batch   58/269 - Train Accuracy:  0.939, Validation Accuracy:  0.934, Loss:  0.060\n",
      "Epoch   8 Batch   59/269 - Train Accuracy:  0.953, Validation Accuracy:  0.946, Loss:  0.047\n",
      "Epoch   8 Batch   60/269 - Train Accuracy:  0.939, Validation Accuracy:  0.946, Loss:  0.055\n",
      "Epoch   8 Batch   61/269 - Train Accuracy:  0.941, Validation Accuracy:  0.946, Loss:  0.053\n",
      "Epoch   8 Batch   62/269 - Train Accuracy:  0.928, Validation Accuracy:  0.948, Loss:  0.060\n",
      "Epoch   8 Batch   63/269 - Train Accuracy:  0.933, Validation Accuracy:  0.946, Loss:  0.064\n",
      "Epoch   8 Batch   64/269 - Train Accuracy:  0.941, Validation Accuracy:  0.945, Loss:  0.053\n",
      "Epoch   8 Batch   65/269 - Train Accuracy:  0.930, Validation Accuracy:  0.944, Loss:  0.057\n",
      "Epoch   8 Batch   66/269 - Train Accuracy:  0.934, Validation Accuracy:  0.942, Loss:  0.058\n",
      "Epoch   8 Batch   67/269 - Train Accuracy:  0.933, Validation Accuracy:  0.936, Loss:  0.064\n",
      "Epoch   8 Batch   68/269 - Train Accuracy:  0.940, Validation Accuracy:  0.937, Loss:  0.061\n",
      "Epoch   8 Batch   69/269 - Train Accuracy:  0.925, Validation Accuracy:  0.933, Loss:  0.068\n",
      "Epoch   8 Batch   70/269 - Train Accuracy:  0.939, Validation Accuracy:  0.939, Loss:  0.059\n",
      "Epoch   8 Batch   71/269 - Train Accuracy:  0.944, Validation Accuracy:  0.942, Loss:  0.065\n",
      "Epoch   8 Batch   72/269 - Train Accuracy:  0.935, Validation Accuracy:  0.944, Loss:  0.065\n",
      "Epoch   8 Batch   73/269 - Train Accuracy:  0.932, Validation Accuracy:  0.944, Loss:  0.060\n",
      "Epoch   8 Batch   74/269 - Train Accuracy:  0.948, Validation Accuracy:  0.940, Loss:  0.056\n",
      "Epoch   8 Batch   75/269 - Train Accuracy:  0.944, Validation Accuracy:  0.938, Loss:  0.059\n",
      "Epoch   8 Batch   76/269 - Train Accuracy:  0.926, Validation Accuracy:  0.938, Loss:  0.056\n",
      "Epoch   8 Batch   77/269 - Train Accuracy:  0.942, Validation Accuracy:  0.945, Loss:  0.057\n",
      "Epoch   8 Batch   78/269 - Train Accuracy:  0.941, Validation Accuracy:  0.945, Loss:  0.057\n",
      "Epoch   8 Batch   79/269 - Train Accuracy:  0.932, Validation Accuracy:  0.942, Loss:  0.057\n",
      "Epoch   8 Batch   80/269 - Train Accuracy:  0.940, Validation Accuracy:  0.942, Loss:  0.054\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   8 Batch   81/269 - Train Accuracy:  0.940, Validation Accuracy:  0.934, Loss:  0.064\n",
      "Epoch   8 Batch   82/269 - Train Accuracy:  0.946, Validation Accuracy:  0.930, Loss:  0.051\n",
      "Epoch   8 Batch   83/269 - Train Accuracy:  0.925, Validation Accuracy:  0.931, Loss:  0.068\n",
      "Epoch   8 Batch   84/269 - Train Accuracy:  0.943, Validation Accuracy:  0.941, Loss:  0.056\n",
      "Epoch   8 Batch   85/269 - Train Accuracy:  0.939, Validation Accuracy:  0.936, Loss:  0.055\n",
      "Epoch   8 Batch   86/269 - Train Accuracy:  0.945, Validation Accuracy:  0.940, Loss:  0.051\n",
      "Epoch   8 Batch   87/269 - Train Accuracy:  0.940, Validation Accuracy:  0.934, Loss:  0.062\n",
      "Epoch   8 Batch   88/269 - Train Accuracy:  0.933, Validation Accuracy:  0.931, Loss:  0.060\n",
      "Epoch   8 Batch   89/269 - Train Accuracy:  0.945, Validation Accuracy:  0.938, Loss:  0.054\n",
      "Epoch   8 Batch   90/269 - Train Accuracy:  0.946, Validation Accuracy:  0.951, Loss:  0.058\n",
      "Epoch   8 Batch   91/269 - Train Accuracy:  0.953, Validation Accuracy:  0.947, Loss:  0.050\n",
      "Epoch   8 Batch   92/269 - Train Accuracy:  0.953, Validation Accuracy:  0.950, Loss:  0.050\n",
      "Epoch   8 Batch   93/269 - Train Accuracy:  0.948, Validation Accuracy:  0.941, Loss:  0.053\n",
      "Epoch   8 Batch   94/269 - Train Accuracy:  0.940, Validation Accuracy:  0.941, Loss:  0.062\n",
      "Epoch   8 Batch   95/269 - Train Accuracy:  0.942, Validation Accuracy:  0.940, Loss:  0.055\n",
      "Epoch   8 Batch   96/269 - Train Accuracy:  0.927, Validation Accuracy:  0.945, Loss:  0.058\n",
      "Epoch   8 Batch   97/269 - Train Accuracy:  0.941, Validation Accuracy:  0.949, Loss:  0.058\n",
      "Epoch   8 Batch   98/269 - Train Accuracy:  0.940, Validation Accuracy:  0.947, Loss:  0.055\n",
      "Epoch   8 Batch   99/269 - Train Accuracy:  0.942, Validation Accuracy:  0.946, Loss:  0.057\n",
      "Epoch   8 Batch  100/269 - Train Accuracy:  0.943, Validation Accuracy:  0.948, Loss:  0.057\n",
      "Epoch   8 Batch  101/269 - Train Accuracy:  0.940, Validation Accuracy:  0.947, Loss:  0.064\n",
      "Epoch   8 Batch  102/269 - Train Accuracy:  0.939, Validation Accuracy:  0.949, Loss:  0.054\n",
      "Epoch   8 Batch  103/269 - Train Accuracy:  0.949, Validation Accuracy:  0.946, Loss:  0.062\n",
      "Epoch   8 Batch  104/269 - Train Accuracy:  0.939, Validation Accuracy:  0.945, Loss:  0.055\n",
      "Epoch   8 Batch  105/269 - Train Accuracy:  0.933, Validation Accuracy:  0.948, Loss:  0.056\n",
      "Epoch   8 Batch  106/269 - Train Accuracy:  0.950, Validation Accuracy:  0.950, Loss:  0.050\n",
      "Epoch   8 Batch  107/269 - Train Accuracy:  0.946, Validation Accuracy:  0.949, Loss:  0.056\n",
      "Epoch   8 Batch  108/269 - Train Accuracy:  0.957, Validation Accuracy:  0.950, Loss:  0.055\n",
      "Epoch   8 Batch  109/269 - Train Accuracy:  0.929, Validation Accuracy:  0.946, Loss:  0.059\n",
      "Epoch   8 Batch  110/269 - Train Accuracy:  0.942, Validation Accuracy:  0.944, Loss:  0.051\n",
      "Epoch   8 Batch  111/269 - Train Accuracy:  0.938, Validation Accuracy:  0.942, Loss:  0.058\n",
      "Epoch   8 Batch  112/269 - Train Accuracy:  0.943, Validation Accuracy:  0.947, Loss:  0.058\n",
      "Epoch   8 Batch  113/269 - Train Accuracy:  0.932, Validation Accuracy:  0.948, Loss:  0.055\n",
      "Epoch   8 Batch  114/269 - Train Accuracy:  0.934, Validation Accuracy:  0.948, Loss:  0.056\n",
      "Epoch   8 Batch  115/269 - Train Accuracy:  0.944, Validation Accuracy:  0.947, Loss:  0.054\n",
      "Epoch   8 Batch  116/269 - Train Accuracy:  0.952, Validation Accuracy:  0.938, Loss:  0.058\n",
      "Epoch   8 Batch  117/269 - Train Accuracy:  0.948, Validation Accuracy:  0.939, Loss:  0.050\n",
      "Epoch   8 Batch  118/269 - Train Accuracy:  0.941, Validation Accuracy:  0.940, Loss:  0.051\n",
      "Epoch   8 Batch  119/269 - Train Accuracy:  0.929, Validation Accuracy:  0.945, Loss:  0.058\n",
      "Epoch   8 Batch  120/269 - Train Accuracy:  0.944, Validation Accuracy:  0.945, Loss:  0.056\n",
      "Epoch   8 Batch  121/269 - Train Accuracy:  0.944, Validation Accuracy:  0.941, Loss:  0.053\n",
      "Epoch   8 Batch  122/269 - Train Accuracy:  0.944, Validation Accuracy:  0.943, Loss:  0.053\n",
      "Epoch   8 Batch  123/269 - Train Accuracy:  0.935, Validation Accuracy:  0.938, Loss:  0.052\n",
      "Epoch   8 Batch  124/269 - Train Accuracy:  0.939, Validation Accuracy:  0.940, Loss:  0.050\n",
      "Epoch   8 Batch  125/269 - Train Accuracy:  0.946, Validation Accuracy:  0.947, Loss:  0.049\n",
      "Epoch   8 Batch  126/269 - Train Accuracy:  0.932, Validation Accuracy:  0.945, Loss:  0.054\n",
      "Epoch   8 Batch  127/269 - Train Accuracy:  0.946, Validation Accuracy:  0.943, Loss:  0.054\n",
      "Epoch   8 Batch  128/269 - Train Accuracy:  0.944, Validation Accuracy:  0.947, Loss:  0.054\n",
      "Epoch   8 Batch  129/269 - Train Accuracy:  0.938, Validation Accuracy:  0.940, Loss:  0.053\n",
      "Epoch   8 Batch  130/269 - Train Accuracy:  0.934, Validation Accuracy:  0.935, Loss:  0.061\n",
      "Epoch   8 Batch  131/269 - Train Accuracy:  0.932, Validation Accuracy:  0.945, Loss:  0.056\n",
      "Epoch   8 Batch  132/269 - Train Accuracy:  0.924, Validation Accuracy:  0.948, Loss:  0.059\n",
      "Epoch   8 Batch  133/269 - Train Accuracy:  0.951, Validation Accuracy:  0.943, Loss:  0.048\n",
      "Epoch   8 Batch  134/269 - Train Accuracy:  0.929, Validation Accuracy:  0.943, Loss:  0.055\n",
      "Epoch   8 Batch  135/269 - Train Accuracy:  0.941, Validation Accuracy:  0.944, Loss:  0.056\n",
      "Epoch   8 Batch  136/269 - Train Accuracy:  0.929, Validation Accuracy:  0.935, Loss:  0.058\n",
      "Epoch   8 Batch  137/269 - Train Accuracy:  0.930, Validation Accuracy:  0.939, Loss:  0.060\n",
      "Epoch   8 Batch  138/269 - Train Accuracy:  0.943, Validation Accuracy:  0.949, Loss:  0.050\n",
      "Epoch   8 Batch  139/269 - Train Accuracy:  0.950, Validation Accuracy:  0.949, Loss:  0.048\n",
      "Epoch   8 Batch  140/269 - Train Accuracy:  0.945, Validation Accuracy:  0.951, Loss:  0.060\n",
      "Epoch   8 Batch  141/269 - Train Accuracy:  0.941, Validation Accuracy:  0.949, Loss:  0.056\n",
      "Epoch   8 Batch  142/269 - Train Accuracy:  0.941, Validation Accuracy:  0.945, Loss:  0.052\n",
      "Epoch   8 Batch  143/269 - Train Accuracy:  0.950, Validation Accuracy:  0.939, Loss:  0.047\n",
      "Epoch   8 Batch  144/269 - Train Accuracy:  0.951, Validation Accuracy:  0.943, Loss:  0.044\n",
      "Epoch   8 Batch  145/269 - Train Accuracy:  0.945, Validation Accuracy:  0.945, Loss:  0.051\n",
      "Epoch   8 Batch  146/269 - Train Accuracy:  0.941, Validation Accuracy:  0.946, Loss:  0.055\n",
      "Epoch   8 Batch  147/269 - Train Accuracy:  0.951, Validation Accuracy:  0.942, Loss:  0.056\n",
      "Epoch   8 Batch  148/269 - Train Accuracy:  0.947, Validation Accuracy:  0.941, Loss:  0.054\n",
      "Epoch   8 Batch  149/269 - Train Accuracy:  0.935, Validation Accuracy:  0.939, Loss:  0.061\n",
      "Epoch   8 Batch  150/269 - Train Accuracy:  0.941, Validation Accuracy:  0.946, Loss:  0.055\n",
      "Epoch   8 Batch  151/269 - Train Accuracy:  0.945, Validation Accuracy:  0.952, Loss:  0.052\n",
      "Epoch   8 Batch  152/269 - Train Accuracy:  0.941, Validation Accuracy:  0.953, Loss:  0.055\n",
      "Epoch   8 Batch  153/269 - Train Accuracy:  0.949, Validation Accuracy:  0.950, Loss:  0.053\n",
      "Epoch   8 Batch  154/269 - Train Accuracy:  0.953, Validation Accuracy:  0.948, Loss:  0.053\n",
      "Epoch   8 Batch  155/269 - Train Accuracy:  0.937, Validation Accuracy:  0.946, Loss:  0.051\n",
      "Epoch   8 Batch  156/269 - Train Accuracy:  0.946, Validation Accuracy:  0.949, Loss:  0.053\n",
      "Epoch   8 Batch  157/269 - Train Accuracy:  0.938, Validation Accuracy:  0.951, Loss:  0.046\n",
      "Epoch   8 Batch  158/269 - Train Accuracy:  0.938, Validation Accuracy:  0.949, Loss:  0.054\n",
      "Epoch   8 Batch  159/269 - Train Accuracy:  0.949, Validation Accuracy:  0.950, Loss:  0.054\n",
      "Epoch   8 Batch  160/269 - Train Accuracy:  0.948, Validation Accuracy:  0.947, Loss:  0.052\n",
      "Epoch   8 Batch  161/269 - Train Accuracy:  0.945, Validation Accuracy:  0.949, Loss:  0.051\n",
      "Epoch   8 Batch  162/269 - Train Accuracy:  0.953, Validation Accuracy:  0.949, Loss:  0.051\n",
      "Epoch   8 Batch  163/269 - Train Accuracy:  0.950, Validation Accuracy:  0.947, Loss:  0.049\n",
      "Epoch   8 Batch  164/269 - Train Accuracy:  0.953, Validation Accuracy:  0.948, Loss:  0.051\n",
      "Epoch   8 Batch  165/269 - Train Accuracy:  0.942, Validation Accuracy:  0.946, Loss:  0.050\n",
      "Epoch   8 Batch  166/269 - Train Accuracy:  0.942, Validation Accuracy:  0.944, Loss:  0.053\n",
      "Epoch   8 Batch  167/269 - Train Accuracy:  0.944, Validation Accuracy:  0.944, Loss:  0.050\n",
      "Epoch   8 Batch  168/269 - Train Accuracy:  0.956, Validation Accuracy:  0.947, Loss:  0.052\n",
      "Epoch   8 Batch  169/269 - Train Accuracy:  0.942, Validation Accuracy:  0.949, Loss:  0.050\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   8 Batch  170/269 - Train Accuracy:  0.940, Validation Accuracy:  0.950, Loss:  0.049\n",
      "Epoch   8 Batch  171/269 - Train Accuracy:  0.951, Validation Accuracy:  0.950, Loss:  0.052\n",
      "Epoch   8 Batch  172/269 - Train Accuracy:  0.940, Validation Accuracy:  0.949, Loss:  0.057\n",
      "Epoch   8 Batch  173/269 - Train Accuracy:  0.946, Validation Accuracy:  0.951, Loss:  0.047\n",
      "Epoch   8 Batch  174/269 - Train Accuracy:  0.946, Validation Accuracy:  0.950, Loss:  0.052\n",
      "Epoch   8 Batch  175/269 - Train Accuracy:  0.937, Validation Accuracy:  0.953, Loss:  0.066\n",
      "Epoch   8 Batch  176/269 - Train Accuracy:  0.929, Validation Accuracy:  0.954, Loss:  0.059\n",
      "Epoch   8 Batch  177/269 - Train Accuracy:  0.950, Validation Accuracy:  0.952, Loss:  0.048\n",
      "Epoch   8 Batch  178/269 - Train Accuracy:  0.955, Validation Accuracy:  0.948, Loss:  0.046\n",
      "Epoch   8 Batch  179/269 - Train Accuracy:  0.930, Validation Accuracy:  0.948, Loss:  0.052\n",
      "Epoch   8 Batch  180/269 - Train Accuracy:  0.950, Validation Accuracy:  0.953, Loss:  0.051\n",
      "Epoch   8 Batch  181/269 - Train Accuracy:  0.941, Validation Accuracy:  0.952, Loss:  0.055\n",
      "Epoch   8 Batch  182/269 - Train Accuracy:  0.940, Validation Accuracy:  0.947, Loss:  0.051\n",
      "Epoch   8 Batch  183/269 - Train Accuracy:  0.949, Validation Accuracy:  0.948, Loss:  0.043\n",
      "Epoch   8 Batch  184/269 - Train Accuracy:  0.943, Validation Accuracy:  0.952, Loss:  0.048\n",
      "Epoch   8 Batch  185/269 - Train Accuracy:  0.952, Validation Accuracy:  0.954, Loss:  0.051\n",
      "Epoch   8 Batch  186/269 - Train Accuracy:  0.943, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   8 Batch  187/269 - Train Accuracy:  0.947, Validation Accuracy:  0.954, Loss:  0.048\n",
      "Epoch   8 Batch  188/269 - Train Accuracy:  0.941, Validation Accuracy:  0.956, Loss:  0.048\n",
      "Epoch   8 Batch  189/269 - Train Accuracy:  0.949, Validation Accuracy:  0.954, Loss:  0.048\n",
      "Epoch   8 Batch  190/269 - Train Accuracy:  0.951, Validation Accuracy:  0.953, Loss:  0.050\n",
      "Epoch   8 Batch  191/269 - Train Accuracy:  0.936, Validation Accuracy:  0.955, Loss:  0.051\n",
      "Epoch   8 Batch  192/269 - Train Accuracy:  0.937, Validation Accuracy:  0.957, Loss:  0.054\n",
      "Epoch   8 Batch  193/269 - Train Accuracy:  0.949, Validation Accuracy:  0.956, Loss:  0.048\n",
      "Epoch   8 Batch  194/269 - Train Accuracy:  0.940, Validation Accuracy:  0.957, Loss:  0.052\n",
      "Epoch   8 Batch  195/269 - Train Accuracy:  0.944, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   8 Batch  196/269 - Train Accuracy:  0.934, Validation Accuracy:  0.949, Loss:  0.049\n",
      "Epoch   8 Batch  197/269 - Train Accuracy:  0.934, Validation Accuracy:  0.947, Loss:  0.052\n",
      "Epoch   8 Batch  198/269 - Train Accuracy:  0.950, Validation Accuracy:  0.955, Loss:  0.052\n",
      "Epoch   8 Batch  199/269 - Train Accuracy:  0.941, Validation Accuracy:  0.947, Loss:  0.053\n",
      "Epoch   8 Batch  200/269 - Train Accuracy:  0.946, Validation Accuracy:  0.946, Loss:  0.051\n",
      "Epoch   8 Batch  201/269 - Train Accuracy:  0.941, Validation Accuracy:  0.946, Loss:  0.053\n",
      "Epoch   8 Batch  202/269 - Train Accuracy:  0.941, Validation Accuracy:  0.943, Loss:  0.049\n",
      "Epoch   8 Batch  203/269 - Train Accuracy:  0.938, Validation Accuracy:  0.942, Loss:  0.055\n",
      "Epoch   8 Batch  204/269 - Train Accuracy:  0.941, Validation Accuracy:  0.948, Loss:  0.053\n",
      "Epoch   8 Batch  205/269 - Train Accuracy:  0.955, Validation Accuracy:  0.951, Loss:  0.047\n",
      "Epoch   8 Batch  206/269 - Train Accuracy:  0.941, Validation Accuracy:  0.945, Loss:  0.055\n",
      "Epoch   8 Batch  207/269 - Train Accuracy:  0.944, Validation Accuracy:  0.949, Loss:  0.050\n",
      "Epoch   8 Batch  208/269 - Train Accuracy:  0.956, Validation Accuracy:  0.949, Loss:  0.052\n",
      "Epoch   8 Batch  209/269 - Train Accuracy:  0.953, Validation Accuracy:  0.943, Loss:  0.048\n",
      "Epoch   8 Batch  210/269 - Train Accuracy:  0.942, Validation Accuracy:  0.942, Loss:  0.048\n",
      "Epoch   8 Batch  211/269 - Train Accuracy:  0.941, Validation Accuracy:  0.952, Loss:  0.052\n",
      "Epoch   8 Batch  212/269 - Train Accuracy:  0.942, Validation Accuracy:  0.954, Loss:  0.058\n",
      "Epoch   8 Batch  213/269 - Train Accuracy:  0.934, Validation Accuracy:  0.952, Loss:  0.050\n",
      "Epoch   8 Batch  214/269 - Train Accuracy:  0.945, Validation Accuracy:  0.951, Loss:  0.051\n",
      "Epoch   8 Batch  215/269 - Train Accuracy:  0.948, Validation Accuracy:  0.948, Loss:  0.048\n",
      "Epoch   8 Batch  216/269 - Train Accuracy:  0.939, Validation Accuracy:  0.948, Loss:  0.059\n",
      "Epoch   8 Batch  217/269 - Train Accuracy:  0.937, Validation Accuracy:  0.945, Loss:  0.054\n",
      "Epoch   8 Batch  218/269 - Train Accuracy:  0.951, Validation Accuracy:  0.946, Loss:  0.049\n",
      "Epoch   8 Batch  219/269 - Train Accuracy:  0.950, Validation Accuracy:  0.951, Loss:  0.052\n",
      "Epoch   8 Batch  220/269 - Train Accuracy:  0.939, Validation Accuracy:  0.947, Loss:  0.048\n",
      "Epoch   8 Batch  221/269 - Train Accuracy:  0.935, Validation Accuracy:  0.944, Loss:  0.053\n",
      "Epoch   8 Batch  222/269 - Train Accuracy:  0.959, Validation Accuracy:  0.944, Loss:  0.046\n",
      "Epoch   8 Batch  223/269 - Train Accuracy:  0.939, Validation Accuracy:  0.940, Loss:  0.047\n",
      "Epoch   8 Batch  224/269 - Train Accuracy:  0.943, Validation Accuracy:  0.942, Loss:  0.057\n",
      "Epoch   8 Batch  225/269 - Train Accuracy:  0.947, Validation Accuracy:  0.949, Loss:  0.045\n",
      "Epoch   8 Batch  226/269 - Train Accuracy:  0.950, Validation Accuracy:  0.947, Loss:  0.053\n",
      "Epoch   8 Batch  227/269 - Train Accuracy:  0.953, Validation Accuracy:  0.948, Loss:  0.056\n",
      "Epoch   8 Batch  228/269 - Train Accuracy:  0.946, Validation Accuracy:  0.946, Loss:  0.047\n",
      "Epoch   8 Batch  229/269 - Train Accuracy:  0.942, Validation Accuracy:  0.953, Loss:  0.049\n",
      "Epoch   8 Batch  230/269 - Train Accuracy:  0.954, Validation Accuracy:  0.955, Loss:  0.049\n",
      "Epoch   8 Batch  231/269 - Train Accuracy:  0.937, Validation Accuracy:  0.956, Loss:  0.050\n",
      "Epoch   8 Batch  232/269 - Train Accuracy:  0.947, Validation Accuracy:  0.956, Loss:  0.048\n",
      "Epoch   8 Batch  233/269 - Train Accuracy:  0.952, Validation Accuracy:  0.956, Loss:  0.054\n",
      "Epoch   8 Batch  234/269 - Train Accuracy:  0.947, Validation Accuracy:  0.954, Loss:  0.047\n",
      "Epoch   8 Batch  235/269 - Train Accuracy:  0.960, Validation Accuracy:  0.952, Loss:  0.042\n",
      "Epoch   8 Batch  236/269 - Train Accuracy:  0.953, Validation Accuracy:  0.947, Loss:  0.044\n",
      "Epoch   8 Batch  237/269 - Train Accuracy:  0.948, Validation Accuracy:  0.947, Loss:  0.048\n",
      "Epoch   8 Batch  238/269 - Train Accuracy:  0.949, Validation Accuracy:  0.947, Loss:  0.049\n",
      "Epoch   8 Batch  239/269 - Train Accuracy:  0.947, Validation Accuracy:  0.945, Loss:  0.049\n",
      "Epoch   8 Batch  240/269 - Train Accuracy:  0.952, Validation Accuracy:  0.950, Loss:  0.045\n",
      "Epoch   8 Batch  241/269 - Train Accuracy:  0.936, Validation Accuracy:  0.950, Loss:  0.057\n",
      "Epoch   8 Batch  242/269 - Train Accuracy:  0.951, Validation Accuracy:  0.953, Loss:  0.050\n",
      "Epoch   8 Batch  243/269 - Train Accuracy:  0.955, Validation Accuracy:  0.952, Loss:  0.041\n",
      "Epoch   8 Batch  244/269 - Train Accuracy:  0.940, Validation Accuracy:  0.955, Loss:  0.049\n",
      "Epoch   8 Batch  245/269 - Train Accuracy:  0.947, Validation Accuracy:  0.954, Loss:  0.052\n",
      "Epoch   8 Batch  246/269 - Train Accuracy:  0.944, Validation Accuracy:  0.947, Loss:  0.051\n",
      "Epoch   8 Batch  247/269 - Train Accuracy:  0.946, Validation Accuracy:  0.948, Loss:  0.047\n",
      "Epoch   8 Batch  248/269 - Train Accuracy:  0.957, Validation Accuracy:  0.941, Loss:  0.045\n",
      "Epoch   8 Batch  249/269 - Train Accuracy:  0.953, Validation Accuracy:  0.947, Loss:  0.043\n",
      "Epoch   8 Batch  250/269 - Train Accuracy:  0.947, Validation Accuracy:  0.949, Loss:  0.049\n",
      "Epoch   8 Batch  251/269 - Train Accuracy:  0.957, Validation Accuracy:  0.949, Loss:  0.045\n",
      "Epoch   8 Batch  252/269 - Train Accuracy:  0.964, Validation Accuracy:  0.948, Loss:  0.040\n",
      "Epoch   8 Batch  253/269 - Train Accuracy:  0.932, Validation Accuracy:  0.948, Loss:  0.051\n",
      "Epoch   8 Batch  254/269 - Train Accuracy:  0.948, Validation Accuracy:  0.943, Loss:  0.049\n",
      "Epoch   8 Batch  255/269 - Train Accuracy:  0.940, Validation Accuracy:  0.949, Loss:  0.049\n",
      "Epoch   8 Batch  256/269 - Train Accuracy:  0.942, Validation Accuracy:  0.951, Loss:  0.047\n",
      "Epoch   8 Batch  257/269 - Train Accuracy:  0.936, Validation Accuracy:  0.948, Loss:  0.054\n",
      "Epoch   8 Batch  258/269 - Train Accuracy:  0.944, Validation Accuracy:  0.944, Loss:  0.048\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   8 Batch  259/269 - Train Accuracy:  0.943, Validation Accuracy:  0.946, Loss:  0.048\n",
      "Epoch   8 Batch  260/269 - Train Accuracy:  0.945, Validation Accuracy:  0.950, Loss:  0.049\n",
      "Epoch   8 Batch  261/269 - Train Accuracy:  0.951, Validation Accuracy:  0.953, Loss:  0.047\n",
      "Epoch   8 Batch  262/269 - Train Accuracy:  0.946, Validation Accuracy:  0.956, Loss:  0.048\n",
      "Epoch   8 Batch  263/269 - Train Accuracy:  0.947, Validation Accuracy:  0.956, Loss:  0.050\n",
      "Epoch   8 Batch  264/269 - Train Accuracy:  0.930, Validation Accuracy:  0.956, Loss:  0.051\n",
      "Epoch   8 Batch  265/269 - Train Accuracy:  0.930, Validation Accuracy:  0.956, Loss:  0.049\n",
      "Epoch   8 Batch  266/269 - Train Accuracy:  0.947, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   8 Batch  267/269 - Train Accuracy:  0.956, Validation Accuracy:  0.951, Loss:  0.052\n",
      "Epoch   9 Batch    0/269 - Train Accuracy:  0.954, Validation Accuracy:  0.954, Loss:  0.053\n",
      "Epoch   9 Batch    1/269 - Train Accuracy:  0.950, Validation Accuracy:  0.950, Loss:  0.048\n",
      "Epoch   9 Batch    2/269 - Train Accuracy:  0.949, Validation Accuracy:  0.952, Loss:  0.047\n",
      "Epoch   9 Batch    3/269 - Train Accuracy:  0.950, Validation Accuracy:  0.955, Loss:  0.048\n",
      "Epoch   9 Batch    4/269 - Train Accuracy:  0.931, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   9 Batch    5/269 - Train Accuracy:  0.956, Validation Accuracy:  0.952, Loss:  0.046\n",
      "Epoch   9 Batch    6/269 - Train Accuracy:  0.951, Validation Accuracy:  0.949, Loss:  0.043\n",
      "Epoch   9 Batch    7/269 - Train Accuracy:  0.943, Validation Accuracy:  0.950, Loss:  0.045\n",
      "Epoch   9 Batch    8/269 - Train Accuracy:  0.951, Validation Accuracy:  0.949, Loss:  0.051\n",
      "Epoch   9 Batch    9/269 - Train Accuracy:  0.935, Validation Accuracy:  0.951, Loss:  0.051\n",
      "Epoch   9 Batch   10/269 - Train Accuracy:  0.955, Validation Accuracy:  0.952, Loss:  0.043\n",
      "Epoch   9 Batch   11/269 - Train Accuracy:  0.947, Validation Accuracy:  0.950, Loss:  0.051\n",
      "Epoch   9 Batch   12/269 - Train Accuracy:  0.942, Validation Accuracy:  0.951, Loss:  0.053\n",
      "Epoch   9 Batch   13/269 - Train Accuracy:  0.954, Validation Accuracy:  0.952, Loss:  0.040\n",
      "Epoch   9 Batch   14/269 - Train Accuracy:  0.949, Validation Accuracy:  0.954, Loss:  0.045\n",
      "Epoch   9 Batch   15/269 - Train Accuracy:  0.960, Validation Accuracy:  0.954, Loss:  0.038\n",
      "Epoch   9 Batch   16/269 - Train Accuracy:  0.949, Validation Accuracy:  0.951, Loss:  0.050\n",
      "Epoch   9 Batch   17/269 - Train Accuracy:  0.955, Validation Accuracy:  0.953, Loss:  0.041\n",
      "Epoch   9 Batch   18/269 - Train Accuracy:  0.948, Validation Accuracy:  0.949, Loss:  0.047\n",
      "Epoch   9 Batch   19/269 - Train Accuracy:  0.951, Validation Accuracy:  0.950, Loss:  0.040\n",
      "Epoch   9 Batch   20/269 - Train Accuracy:  0.956, Validation Accuracy:  0.952, Loss:  0.045\n",
      "Epoch   9 Batch   21/269 - Train Accuracy:  0.936, Validation Accuracy:  0.951, Loss:  0.051\n",
      "Epoch   9 Batch   22/269 - Train Accuracy:  0.949, Validation Accuracy:  0.945, Loss:  0.043\n",
      "Epoch   9 Batch   23/269 - Train Accuracy:  0.945, Validation Accuracy:  0.948, Loss:  0.048\n",
      "Epoch   9 Batch   24/269 - Train Accuracy:  0.943, Validation Accuracy:  0.956, Loss:  0.045\n",
      "Epoch   9 Batch   25/269 - Train Accuracy:  0.942, Validation Accuracy:  0.940, Loss:  0.050\n",
      "Epoch   9 Batch   26/269 - Train Accuracy:  0.935, Validation Accuracy:  0.940, Loss:  0.044\n",
      "Epoch   9 Batch   27/269 - Train Accuracy:  0.950, Validation Accuracy:  0.952, Loss:  0.043\n",
      "Epoch   9 Batch   28/269 - Train Accuracy:  0.933, Validation Accuracy:  0.956, Loss:  0.048\n",
      "Epoch   9 Batch   29/269 - Train Accuracy:  0.949, Validation Accuracy:  0.950, Loss:  0.047\n",
      "Epoch   9 Batch   30/269 - Train Accuracy:  0.944, Validation Accuracy:  0.947, Loss:  0.047\n",
      "Epoch   9 Batch   31/269 - Train Accuracy:  0.962, Validation Accuracy:  0.942, Loss:  0.043\n",
      "Epoch   9 Batch   32/269 - Train Accuracy:  0.952, Validation Accuracy:  0.937, Loss:  0.043\n",
      "Epoch   9 Batch   33/269 - Train Accuracy:  0.957, Validation Accuracy:  0.941, Loss:  0.041\n",
      "Epoch   9 Batch   34/269 - Train Accuracy:  0.946, Validation Accuracy:  0.955, Loss:  0.044\n",
      "Epoch   9 Batch   35/269 - Train Accuracy:  0.944, Validation Accuracy:  0.952, Loss:  0.054\n",
      "Epoch   9 Batch   36/269 - Train Accuracy:  0.941, Validation Accuracy:  0.954, Loss:  0.047\n",
      "Epoch   9 Batch   37/269 - Train Accuracy:  0.936, Validation Accuracy:  0.956, Loss:  0.047\n",
      "Epoch   9 Batch   38/269 - Train Accuracy:  0.945, Validation Accuracy:  0.959, Loss:  0.045\n",
      "Epoch   9 Batch   39/269 - Train Accuracy:  0.953, Validation Accuracy:  0.954, Loss:  0.044\n",
      "Epoch   9 Batch   40/269 - Train Accuracy:  0.939, Validation Accuracy:  0.952, Loss:  0.048\n",
      "Epoch   9 Batch   41/269 - Train Accuracy:  0.948, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   9 Batch   42/269 - Train Accuracy:  0.953, Validation Accuracy:  0.948, Loss:  0.040\n",
      "Epoch   9 Batch   43/269 - Train Accuracy:  0.947, Validation Accuracy:  0.949, Loss:  0.046\n",
      "Epoch   9 Batch   44/269 - Train Accuracy:  0.946, Validation Accuracy:  0.949, Loss:  0.047\n",
      "Epoch   9 Batch   45/269 - Train Accuracy:  0.945, Validation Accuracy:  0.952, Loss:  0.047\n",
      "Epoch   9 Batch   46/269 - Train Accuracy:  0.942, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   9 Batch   47/269 - Train Accuracy:  0.949, Validation Accuracy:  0.959, Loss:  0.040\n",
      "Epoch   9 Batch   48/269 - Train Accuracy:  0.952, Validation Accuracy:  0.958, Loss:  0.042\n",
      "Epoch   9 Batch   49/269 - Train Accuracy:  0.959, Validation Accuracy:  0.957, Loss:  0.042\n",
      "Epoch   9 Batch   50/269 - Train Accuracy:  0.937, Validation Accuracy:  0.953, Loss:  0.054\n",
      "Epoch   9 Batch   51/269 - Train Accuracy:  0.949, Validation Accuracy:  0.951, Loss:  0.044\n",
      "Epoch   9 Batch   52/269 - Train Accuracy:  0.954, Validation Accuracy:  0.953, Loss:  0.038\n",
      "Epoch   9 Batch   53/269 - Train Accuracy:  0.948, Validation Accuracy:  0.948, Loss:  0.048\n",
      "Epoch   9 Batch   54/269 - Train Accuracy:  0.953, Validation Accuracy:  0.954, Loss:  0.043\n",
      "Epoch   9 Batch   55/269 - Train Accuracy:  0.949, Validation Accuracy:  0.951, Loss:  0.044\n",
      "Epoch   9 Batch   56/269 - Train Accuracy:  0.940, Validation Accuracy:  0.948, Loss:  0.045\n",
      "Epoch   9 Batch   57/269 - Train Accuracy:  0.951, Validation Accuracy:  0.949, Loss:  0.051\n",
      "Epoch   9 Batch   58/269 - Train Accuracy:  0.953, Validation Accuracy:  0.949, Loss:  0.047\n",
      "Epoch   9 Batch   59/269 - Train Accuracy:  0.962, Validation Accuracy:  0.953, Loss:  0.036\n",
      "Epoch   9 Batch   60/269 - Train Accuracy:  0.950, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   9 Batch   61/269 - Train Accuracy:  0.949, Validation Accuracy:  0.957, Loss:  0.041\n",
      "Epoch   9 Batch   62/269 - Train Accuracy:  0.938, Validation Accuracy:  0.957, Loss:  0.048\n",
      "Epoch   9 Batch   63/269 - Train Accuracy:  0.945, Validation Accuracy:  0.954, Loss:  0.052\n",
      "Epoch   9 Batch   64/269 - Train Accuracy:  0.950, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   9 Batch   65/269 - Train Accuracy:  0.945, Validation Accuracy:  0.958, Loss:  0.044\n",
      "Epoch   9 Batch   66/269 - Train Accuracy:  0.943, Validation Accuracy:  0.952, Loss:  0.046\n",
      "Epoch   9 Batch   67/269 - Train Accuracy:  0.947, Validation Accuracy:  0.950, Loss:  0.050\n",
      "Epoch   9 Batch   68/269 - Train Accuracy:  0.946, Validation Accuracy:  0.944, Loss:  0.048\n",
      "Epoch   9 Batch   69/269 - Train Accuracy:  0.936, Validation Accuracy:  0.940, Loss:  0.055\n",
      "Epoch   9 Batch   70/269 - Train Accuracy:  0.948, Validation Accuracy:  0.946, Loss:  0.047\n",
      "Epoch   9 Batch   71/269 - Train Accuracy:  0.954, Validation Accuracy:  0.951, Loss:  0.052\n",
      "Epoch   9 Batch   72/269 - Train Accuracy:  0.946, Validation Accuracy:  0.952, Loss:  0.052\n",
      "Epoch   9 Batch   73/269 - Train Accuracy:  0.943, Validation Accuracy:  0.955, Loss:  0.047\n",
      "Epoch   9 Batch   74/269 - Train Accuracy:  0.952, Validation Accuracy:  0.952, Loss:  0.044\n",
      "Epoch   9 Batch   75/269 - Train Accuracy:  0.951, Validation Accuracy:  0.945, Loss:  0.046\n",
      "Epoch   9 Batch   76/269 - Train Accuracy:  0.945, Validation Accuracy:  0.945, Loss:  0.045\n",
      "Epoch   9 Batch   77/269 - Train Accuracy:  0.948, Validation Accuracy:  0.953, Loss:  0.045\n",
      "Epoch   9 Batch   78/269 - Train Accuracy:  0.957, Validation Accuracy:  0.955, Loss:  0.045\n",
      "Epoch   9 Batch   79/269 - Train Accuracy:  0.935, Validation Accuracy:  0.957, Loss:  0.045\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   9 Batch   80/269 - Train Accuracy:  0.944, Validation Accuracy:  0.957, Loss:  0.043\n",
      "Epoch   9 Batch   81/269 - Train Accuracy:  0.946, Validation Accuracy:  0.947, Loss:  0.051\n",
      "Epoch   9 Batch   82/269 - Train Accuracy:  0.956, Validation Accuracy:  0.937, Loss:  0.039\n",
      "Epoch   9 Batch   83/269 - Train Accuracy:  0.938, Validation Accuracy:  0.946, Loss:  0.055\n",
      "Epoch   9 Batch   84/269 - Train Accuracy:  0.954, Validation Accuracy:  0.954, Loss:  0.045\n",
      "Epoch   9 Batch   85/269 - Train Accuracy:  0.945, Validation Accuracy:  0.950, Loss:  0.043\n",
      "Epoch   9 Batch   86/269 - Train Accuracy:  0.954, Validation Accuracy:  0.951, Loss:  0.040\n",
      "Epoch   9 Batch   87/269 - Train Accuracy:  0.943, Validation Accuracy:  0.948, Loss:  0.049\n",
      "Epoch   9 Batch   88/269 - Train Accuracy:  0.943, Validation Accuracy:  0.941, Loss:  0.047\n",
      "Epoch   9 Batch   89/269 - Train Accuracy:  0.947, Validation Accuracy:  0.944, Loss:  0.041\n",
      "Epoch   9 Batch   90/269 - Train Accuracy:  0.946, Validation Accuracy:  0.952, Loss:  0.046\n",
      "Epoch   9 Batch   91/269 - Train Accuracy:  0.962, Validation Accuracy:  0.959, Loss:  0.039\n",
      "Epoch   9 Batch   92/269 - Train Accuracy:  0.954, Validation Accuracy:  0.960, Loss:  0.040\n",
      "Epoch   9 Batch   93/269 - Train Accuracy:  0.957, Validation Accuracy:  0.955, Loss:  0.043\n",
      "Epoch   9 Batch   94/269 - Train Accuracy:  0.946, Validation Accuracy:  0.953, Loss:  0.049\n",
      "Epoch   9 Batch   95/269 - Train Accuracy:  0.949, Validation Accuracy:  0.944, Loss:  0.043\n",
      "Epoch   9 Batch   96/269 - Train Accuracy:  0.941, Validation Accuracy:  0.944, Loss:  0.047\n",
      "Epoch   9 Batch   97/269 - Train Accuracy:  0.950, Validation Accuracy:  0.952, Loss:  0.047\n",
      "Epoch   9 Batch   98/269 - Train Accuracy:  0.947, Validation Accuracy:  0.953, Loss:  0.043\n",
      "Epoch   9 Batch   99/269 - Train Accuracy:  0.947, Validation Accuracy:  0.953, Loss:  0.043\n",
      "Epoch   9 Batch  100/269 - Train Accuracy:  0.958, Validation Accuracy:  0.949, Loss:  0.045\n",
      "Epoch   9 Batch  101/269 - Train Accuracy:  0.943, Validation Accuracy:  0.951, Loss:  0.054\n",
      "Epoch   9 Batch  102/269 - Train Accuracy:  0.956, Validation Accuracy:  0.947, Loss:  0.042\n",
      "Epoch   9 Batch  103/269 - Train Accuracy:  0.952, Validation Accuracy:  0.944, Loss:  0.048\n",
      "Epoch   9 Batch  104/269 - Train Accuracy:  0.953, Validation Accuracy:  0.949, Loss:  0.043\n",
      "Epoch   9 Batch  105/269 - Train Accuracy:  0.941, Validation Accuracy:  0.951, Loss:  0.044\n",
      "Epoch   9 Batch  106/269 - Train Accuracy:  0.960, Validation Accuracy:  0.959, Loss:  0.038\n",
      "Epoch   9 Batch  107/269 - Train Accuracy:  0.952, Validation Accuracy:  0.960, Loss:  0.045\n",
      "Epoch   9 Batch  108/269 - Train Accuracy:  0.959, Validation Accuracy:  0.958, Loss:  0.044\n",
      "Epoch   9 Batch  109/269 - Train Accuracy:  0.936, Validation Accuracy:  0.955, Loss:  0.046\n",
      "Epoch   9 Batch  110/269 - Train Accuracy:  0.953, Validation Accuracy:  0.951, Loss:  0.039\n",
      "Epoch   9 Batch  111/269 - Train Accuracy:  0.949, Validation Accuracy:  0.953, Loss:  0.045\n",
      "Epoch   9 Batch  112/269 - Train Accuracy:  0.949, Validation Accuracy:  0.956, Loss:  0.047\n",
      "Epoch   9 Batch  113/269 - Train Accuracy:  0.944, Validation Accuracy:  0.956, Loss:  0.043\n",
      "Epoch   9 Batch  114/269 - Train Accuracy:  0.940, Validation Accuracy:  0.956, Loss:  0.045\n",
      "Epoch   9 Batch  115/269 - Train Accuracy:  0.953, Validation Accuracy:  0.959, Loss:  0.042\n",
      "Epoch   9 Batch  116/269 - Train Accuracy:  0.958, Validation Accuracy:  0.944, Loss:  0.046\n",
      "Epoch   9 Batch  117/269 - Train Accuracy:  0.952, Validation Accuracy:  0.945, Loss:  0.039\n",
      "Epoch   9 Batch  118/269 - Train Accuracy:  0.952, Validation Accuracy:  0.947, Loss:  0.040\n",
      "Epoch   9 Batch  119/269 - Train Accuracy:  0.941, Validation Accuracy:  0.953, Loss:  0.046\n",
      "Epoch   9 Batch  120/269 - Train Accuracy:  0.951, Validation Accuracy:  0.954, Loss:  0.045\n",
      "Epoch   9 Batch  121/269 - Train Accuracy:  0.949, Validation Accuracy:  0.954, Loss:  0.042\n",
      "Epoch   9 Batch  122/269 - Train Accuracy:  0.952, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   9 Batch  123/269 - Train Accuracy:  0.948, Validation Accuracy:  0.950, Loss:  0.041\n",
      "Epoch   9 Batch  124/269 - Train Accuracy:  0.952, Validation Accuracy:  0.945, Loss:  0.038\n",
      "Epoch   9 Batch  125/269 - Train Accuracy:  0.960, Validation Accuracy:  0.953, Loss:  0.039\n",
      "Epoch   9 Batch  126/269 - Train Accuracy:  0.938, Validation Accuracy:  0.956, Loss:  0.043\n",
      "Epoch   9 Batch  127/269 - Train Accuracy:  0.953, Validation Accuracy:  0.958, Loss:  0.042\n",
      "Epoch   9 Batch  128/269 - Train Accuracy:  0.950, Validation Accuracy:  0.956, Loss:  0.041\n",
      "Epoch   9 Batch  129/269 - Train Accuracy:  0.946, Validation Accuracy:  0.952, Loss:  0.042\n",
      "Epoch   9 Batch  130/269 - Train Accuracy:  0.950, Validation Accuracy:  0.949, Loss:  0.048\n",
      "Epoch   9 Batch  131/269 - Train Accuracy:  0.944, Validation Accuracy:  0.952, Loss:  0.044\n",
      "Epoch   9 Batch  132/269 - Train Accuracy:  0.935, Validation Accuracy:  0.957, Loss:  0.047\n",
      "Epoch   9 Batch  133/269 - Train Accuracy:  0.963, Validation Accuracy:  0.955, Loss:  0.037\n",
      "Epoch   9 Batch  134/269 - Train Accuracy:  0.941, Validation Accuracy:  0.954, Loss:  0.042\n",
      "Epoch   9 Batch  135/269 - Train Accuracy:  0.954, Validation Accuracy:  0.952, Loss:  0.044\n",
      "Epoch   9 Batch  136/269 - Train Accuracy:  0.941, Validation Accuracy:  0.943, Loss:  0.046\n",
      "Epoch   9 Batch  137/269 - Train Accuracy:  0.940, Validation Accuracy:  0.939, Loss:  0.048\n",
      "Epoch   9 Batch  138/269 - Train Accuracy:  0.957, Validation Accuracy:  0.942, Loss:  0.040\n",
      "Epoch   9 Batch  139/269 - Train Accuracy:  0.962, Validation Accuracy:  0.948, Loss:  0.038\n",
      "Epoch   9 Batch  140/269 - Train Accuracy:  0.952, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   9 Batch  141/269 - Train Accuracy:  0.941, Validation Accuracy:  0.954, Loss:  0.044\n",
      "Epoch   9 Batch  142/269 - Train Accuracy:  0.954, Validation Accuracy:  0.956, Loss:  0.041\n",
      "Epoch   9 Batch  143/269 - Train Accuracy:  0.955, Validation Accuracy:  0.948, Loss:  0.037\n",
      "Epoch   9 Batch  144/269 - Train Accuracy:  0.956, Validation Accuracy:  0.946, Loss:  0.034\n",
      "Epoch   9 Batch  145/269 - Train Accuracy:  0.953, Validation Accuracy:  0.951, Loss:  0.040\n",
      "Epoch   9 Batch  146/269 - Train Accuracy:  0.950, Validation Accuracy:  0.959, Loss:  0.043\n",
      "Epoch   9 Batch  147/269 - Train Accuracy:  0.960, Validation Accuracy:  0.961, Loss:  0.046\n",
      "Epoch   9 Batch  148/269 - Train Accuracy:  0.951, Validation Accuracy:  0.957, Loss:  0.042\n",
      "Epoch   9 Batch  149/269 - Train Accuracy:  0.937, Validation Accuracy:  0.953, Loss:  0.050\n",
      "Epoch   9 Batch  150/269 - Train Accuracy:  0.953, Validation Accuracy:  0.952, Loss:  0.044\n",
      "Epoch   9 Batch  151/269 - Train Accuracy:  0.950, Validation Accuracy:  0.949, Loss:  0.043\n",
      "Epoch   9 Batch  152/269 - Train Accuracy:  0.946, Validation Accuracy:  0.954, Loss:  0.044\n",
      "Epoch   9 Batch  153/269 - Train Accuracy:  0.956, Validation Accuracy:  0.957, Loss:  0.041\n",
      "Epoch   9 Batch  154/269 - Train Accuracy:  0.962, Validation Accuracy:  0.958, Loss:  0.043\n",
      "Epoch   9 Batch  155/269 - Train Accuracy:  0.947, Validation Accuracy:  0.960, Loss:  0.040\n",
      "Epoch   9 Batch  156/269 - Train Accuracy:  0.949, Validation Accuracy:  0.960, Loss:  0.042\n",
      "Epoch   9 Batch  157/269 - Train Accuracy:  0.947, Validation Accuracy:  0.962, Loss:  0.037\n",
      "Epoch   9 Batch  158/269 - Train Accuracy:  0.949, Validation Accuracy:  0.954, Loss:  0.044\n",
      "Epoch   9 Batch  159/269 - Train Accuracy:  0.953, Validation Accuracy:  0.954, Loss:  0.043\n",
      "Epoch   9 Batch  160/269 - Train Accuracy:  0.953, Validation Accuracy:  0.959, Loss:  0.041\n",
      "Epoch   9 Batch  161/269 - Train Accuracy:  0.954, Validation Accuracy:  0.963, Loss:  0.042\n",
      "Epoch   9 Batch  162/269 - Train Accuracy:  0.957, Validation Accuracy:  0.962, Loss:  0.041\n",
      "Epoch   9 Batch  163/269 - Train Accuracy:  0.960, Validation Accuracy:  0.960, Loss:  0.039\n",
      "Epoch   9 Batch  164/269 - Train Accuracy:  0.962, Validation Accuracy:  0.960, Loss:  0.041\n",
      "Epoch   9 Batch  165/269 - Train Accuracy:  0.944, Validation Accuracy:  0.953, Loss:  0.040\n",
      "Epoch   9 Batch  166/269 - Train Accuracy:  0.949, Validation Accuracy:  0.954, Loss:  0.042\n",
      "Epoch   9 Batch  167/269 - Train Accuracy:  0.950, Validation Accuracy:  0.953, Loss:  0.042\n",
      "Epoch   9 Batch  168/269 - Train Accuracy:  0.960, Validation Accuracy:  0.956, Loss:  0.042\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   9 Batch  169/269 - Train Accuracy:  0.947, Validation Accuracy:  0.961, Loss:  0.040\n",
      "Epoch   9 Batch  170/269 - Train Accuracy:  0.952, Validation Accuracy:  0.963, Loss:  0.039\n",
      "Epoch   9 Batch  171/269 - Train Accuracy:  0.957, Validation Accuracy:  0.959, Loss:  0.042\n",
      "Epoch   9 Batch  172/269 - Train Accuracy:  0.952, Validation Accuracy:  0.958, Loss:  0.046\n",
      "Epoch   9 Batch  173/269 - Train Accuracy:  0.951, Validation Accuracy:  0.959, Loss:  0.037\n",
      "Epoch   9 Batch  174/269 - Train Accuracy:  0.952, Validation Accuracy:  0.956, Loss:  0.041\n",
      "Epoch   9 Batch  175/269 - Train Accuracy:  0.942, Validation Accuracy:  0.957, Loss:  0.054\n",
      "Epoch   9 Batch  176/269 - Train Accuracy:  0.933, Validation Accuracy:  0.957, Loss:  0.047\n",
      "Epoch   9 Batch  177/269 - Train Accuracy:  0.957, Validation Accuracy:  0.958, Loss:  0.038\n",
      "Epoch   9 Batch  178/269 - Train Accuracy:  0.962, Validation Accuracy:  0.956, Loss:  0.037\n",
      "Epoch   9 Batch  179/269 - Train Accuracy:  0.937, Validation Accuracy:  0.955, Loss:  0.042\n",
      "Epoch   9 Batch  180/269 - Train Accuracy:  0.954, Validation Accuracy:  0.958, Loss:  0.040\n",
      "Epoch   9 Batch  181/269 - Train Accuracy:  0.946, Validation Accuracy:  0.959, Loss:  0.044\n",
      "Epoch   9 Batch  182/269 - Train Accuracy:  0.949, Validation Accuracy:  0.953, Loss:  0.041\n",
      "Epoch   9 Batch  183/269 - Train Accuracy:  0.957, Validation Accuracy:  0.953, Loss:  0.034\n",
      "Epoch   9 Batch  184/269 - Train Accuracy:  0.948, Validation Accuracy:  0.953, Loss:  0.038\n",
      "Epoch   9 Batch  185/269 - Train Accuracy:  0.957, Validation Accuracy:  0.960, Loss:  0.041\n",
      "Epoch   9 Batch  186/269 - Train Accuracy:  0.953, Validation Accuracy:  0.958, Loss:  0.038\n",
      "Epoch   9 Batch  187/269 - Train Accuracy:  0.947, Validation Accuracy:  0.958, Loss:  0.038\n",
      "Epoch   9 Batch  188/269 - Train Accuracy:  0.952, Validation Accuracy:  0.958, Loss:  0.038\n",
      "Epoch   9 Batch  189/269 - Train Accuracy:  0.958, Validation Accuracy:  0.960, Loss:  0.038\n",
      "Epoch   9 Batch  190/269 - Train Accuracy:  0.958, Validation Accuracy:  0.961, Loss:  0.039\n",
      "Epoch   9 Batch  191/269 - Train Accuracy:  0.941, Validation Accuracy:  0.961, Loss:  0.041\n",
      "Epoch   9 Batch  192/269 - Train Accuracy:  0.947, Validation Accuracy:  0.962, Loss:  0.043\n",
      "Epoch   9 Batch  193/269 - Train Accuracy:  0.953, Validation Accuracy:  0.962, Loss:  0.038\n",
      "Epoch   9 Batch  194/269 - Train Accuracy:  0.947, Validation Accuracy:  0.958, Loss:  0.042\n",
      "Epoch   9 Batch  195/269 - Train Accuracy:  0.958, Validation Accuracy:  0.961, Loss:  0.038\n",
      "Epoch   9 Batch  196/269 - Train Accuracy:  0.946, Validation Accuracy:  0.950, Loss:  0.039\n",
      "Epoch   9 Batch  197/269 - Train Accuracy:  0.942, Validation Accuracy:  0.946, Loss:  0.042\n",
      "Epoch   9 Batch  198/269 - Train Accuracy:  0.954, Validation Accuracy:  0.957, Loss:  0.043\n",
      "Epoch   9 Batch  199/269 - Train Accuracy:  0.951, Validation Accuracy:  0.958, Loss:  0.042\n",
      "Epoch   9 Batch  200/269 - Train Accuracy:  0.953, Validation Accuracy:  0.955, Loss:  0.039\n",
      "Epoch   9 Batch  201/269 - Train Accuracy:  0.950, Validation Accuracy:  0.957, Loss:  0.043\n",
      "Epoch   9 Batch  202/269 - Train Accuracy:  0.949, Validation Accuracy:  0.960, Loss:  0.040\n",
      "Epoch   9 Batch  203/269 - Train Accuracy:  0.945, Validation Accuracy:  0.951, Loss:  0.043\n",
      "Epoch   9 Batch  204/269 - Train Accuracy:  0.945, Validation Accuracy:  0.954, Loss:  0.043\n",
      "Epoch   9 Batch  205/269 - Train Accuracy:  0.961, Validation Accuracy:  0.953, Loss:  0.038\n",
      "Epoch   9 Batch  206/269 - Train Accuracy:  0.950, Validation Accuracy:  0.960, Loss:  0.045\n",
      "Epoch   9 Batch  207/269 - Train Accuracy:  0.950, Validation Accuracy:  0.959, Loss:  0.040\n",
      "Epoch   9 Batch  208/269 - Train Accuracy:  0.956, Validation Accuracy:  0.956, Loss:  0.042\n",
      "Epoch   9 Batch  209/269 - Train Accuracy:  0.953, Validation Accuracy:  0.958, Loss:  0.039\n",
      "Epoch   9 Batch  210/269 - Train Accuracy:  0.947, Validation Accuracy:  0.955, Loss:  0.038\n",
      "Epoch   9 Batch  211/269 - Train Accuracy:  0.948, Validation Accuracy:  0.952, Loss:  0.043\n",
      "Epoch   9 Batch  212/269 - Train Accuracy:  0.951, Validation Accuracy:  0.958, Loss:  0.048\n",
      "Epoch   9 Batch  213/269 - Train Accuracy:  0.947, Validation Accuracy:  0.962, Loss:  0.041\n",
      "Epoch   9 Batch  214/269 - Train Accuracy:  0.951, Validation Accuracy:  0.959, Loss:  0.040\n",
      "Epoch   9 Batch  215/269 - Train Accuracy:  0.951, Validation Accuracy:  0.956, Loss:  0.038\n",
      "Epoch   9 Batch  216/269 - Train Accuracy:  0.946, Validation Accuracy:  0.960, Loss:  0.050\n",
      "Epoch   9 Batch  217/269 - Train Accuracy:  0.943, Validation Accuracy:  0.952, Loss:  0.042\n",
      "Epoch   9 Batch  218/269 - Train Accuracy:  0.955, Validation Accuracy:  0.947, Loss:  0.038\n",
      "Epoch   9 Batch  219/269 - Train Accuracy:  0.959, Validation Accuracy:  0.949, Loss:  0.040\n",
      "Epoch   9 Batch  220/269 - Train Accuracy:  0.950, Validation Accuracy:  0.952, Loss:  0.039\n",
      "Epoch   9 Batch  221/269 - Train Accuracy:  0.948, Validation Accuracy:  0.956, Loss:  0.042\n",
      "Epoch   9 Batch  222/269 - Train Accuracy:  0.960, Validation Accuracy:  0.957, Loss:  0.037\n",
      "Epoch   9 Batch  223/269 - Train Accuracy:  0.951, Validation Accuracy:  0.953, Loss:  0.038\n",
      "Epoch   9 Batch  224/269 - Train Accuracy:  0.951, Validation Accuracy:  0.948, Loss:  0.047\n",
      "Epoch   9 Batch  225/269 - Train Accuracy:  0.957, Validation Accuracy:  0.945, Loss:  0.036\n",
      "Epoch   9 Batch  226/269 - Train Accuracy:  0.956, Validation Accuracy:  0.951, Loss:  0.043\n",
      "Epoch   9 Batch  227/269 - Train Accuracy:  0.959, Validation Accuracy:  0.953, Loss:  0.047\n",
      "Epoch   9 Batch  228/269 - Train Accuracy:  0.956, Validation Accuracy:  0.957, Loss:  0.037\n",
      "Epoch   9 Batch  229/269 - Train Accuracy:  0.955, Validation Accuracy:  0.960, Loss:  0.038\n",
      "Epoch   9 Batch  230/269 - Train Accuracy:  0.959, Validation Accuracy:  0.957, Loss:  0.039\n",
      "Epoch   9 Batch  231/269 - Train Accuracy:  0.946, Validation Accuracy:  0.959, Loss:  0.041\n",
      "Epoch   9 Batch  232/269 - Train Accuracy:  0.954, Validation Accuracy:  0.958, Loss:  0.038\n",
      "Epoch   9 Batch  233/269 - Train Accuracy:  0.963, Validation Accuracy:  0.960, Loss:  0.045\n",
      "Epoch   9 Batch  234/269 - Train Accuracy:  0.952, Validation Accuracy:  0.958, Loss:  0.037\n",
      "Epoch   9 Batch  235/269 - Train Accuracy:  0.972, Validation Accuracy:  0.958, Loss:  0.033\n",
      "Epoch   9 Batch  236/269 - Train Accuracy:  0.961, Validation Accuracy:  0.961, Loss:  0.035\n",
      "Epoch   9 Batch  237/269 - Train Accuracy:  0.961, Validation Accuracy:  0.960, Loss:  0.038\n",
      "Epoch   9 Batch  238/269 - Train Accuracy:  0.957, Validation Accuracy:  0.953, Loss:  0.039\n",
      "Epoch   9 Batch  239/269 - Train Accuracy:  0.949, Validation Accuracy:  0.951, Loss:  0.039\n",
      "Epoch   9 Batch  240/269 - Train Accuracy:  0.956, Validation Accuracy:  0.954, Loss:  0.037\n",
      "Epoch   9 Batch  241/269 - Train Accuracy:  0.944, Validation Accuracy:  0.953, Loss:  0.048\n",
      "Epoch   9 Batch  242/269 - Train Accuracy:  0.962, Validation Accuracy:  0.953, Loss:  0.039\n",
      "Epoch   9 Batch  243/269 - Train Accuracy:  0.966, Validation Accuracy:  0.955, Loss:  0.034\n",
      "Epoch   9 Batch  244/269 - Train Accuracy:  0.956, Validation Accuracy:  0.959, Loss:  0.040\n",
      "Epoch   9 Batch  245/269 - Train Accuracy:  0.953, Validation Accuracy:  0.959, Loss:  0.041\n",
      "Epoch   9 Batch  246/269 - Train Accuracy:  0.943, Validation Accuracy:  0.956, Loss:  0.042\n",
      "Epoch   9 Batch  247/269 - Train Accuracy:  0.950, Validation Accuracy:  0.948, Loss:  0.037\n",
      "Epoch   9 Batch  248/269 - Train Accuracy:  0.965, Validation Accuracy:  0.949, Loss:  0.035\n",
      "Epoch   9 Batch  249/269 - Train Accuracy:  0.958, Validation Accuracy:  0.948, Loss:  0.035\n",
      "Epoch   9 Batch  250/269 - Train Accuracy:  0.951, Validation Accuracy:  0.953, Loss:  0.039\n",
      "Epoch   9 Batch  251/269 - Train Accuracy:  0.966, Validation Accuracy:  0.960, Loss:  0.037\n",
      "Epoch   9 Batch  252/269 - Train Accuracy:  0.971, Validation Accuracy:  0.954, Loss:  0.032\n",
      "Epoch   9 Batch  253/269 - Train Accuracy:  0.943, Validation Accuracy:  0.948, Loss:  0.042\n",
      "Epoch   9 Batch  254/269 - Train Accuracy:  0.953, Validation Accuracy:  0.947, Loss:  0.039\n",
      "Epoch   9 Batch  255/269 - Train Accuracy:  0.940, Validation Accuracy:  0.951, Loss:  0.041\n",
      "Epoch   9 Batch  256/269 - Train Accuracy:  0.947, Validation Accuracy:  0.959, Loss:  0.037\n",
      "Epoch   9 Batch  257/269 - Train Accuracy:  0.941, Validation Accuracy:  0.955, Loss:  0.044\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   9 Batch  258/269 - Train Accuracy:  0.948, Validation Accuracy:  0.956, Loss:  0.040\n",
      "Epoch   9 Batch  259/269 - Train Accuracy:  0.953, Validation Accuracy:  0.955, Loss:  0.038\n",
      "Epoch   9 Batch  260/269 - Train Accuracy:  0.955, Validation Accuracy:  0.954, Loss:  0.039\n",
      "Epoch   9 Batch  261/269 - Train Accuracy:  0.958, Validation Accuracy:  0.956, Loss:  0.037\n",
      "Epoch   9 Batch  262/269 - Train Accuracy:  0.950, Validation Accuracy:  0.956, Loss:  0.039\n",
      "Epoch   9 Batch  263/269 - Train Accuracy:  0.954, Validation Accuracy:  0.959, Loss:  0.041\n",
      "Epoch   9 Batch  264/269 - Train Accuracy:  0.938, Validation Accuracy:  0.961, Loss:  0.043\n",
      "Epoch   9 Batch  265/269 - Train Accuracy:  0.947, Validation Accuracy:  0.960, Loss:  0.040\n",
      "Epoch   9 Batch  266/269 - Train Accuracy:  0.956, Validation Accuracy:  0.960, Loss:  0.034\n",
      "Epoch   9 Batch  267/269 - Train Accuracy:  0.961, Validation Accuracy:  0.962, Loss:  0.043\n",
      "Model Trained and Saved\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import time\n",
    "\n",
    "def get_accuracy(target, logits):\n",
    "    \"\"\"\n",
    "    Calculate accuracy\n",
    "    \"\"\"\n",
    "    max_seq = max(target.shape[1], logits.shape[1])\n",
    "    if max_seq - target.shape[1]:\n",
    "        target = np.pad(\n",
    "            target,\n",
    "            [(0,0),(0,max_seq - target.shape[1])],\n",
    "            'constant')\n",
    "    if max_seq - logits.shape[1]:\n",
    "        logits = np.pad(\n",
    "            logits,\n",
    "            [(0,0),(0,max_seq - logits.shape[1]), (0,0)],\n",
    "            'constant')\n",
    "\n",
    "    return np.mean(np.equal(target, np.argmax(logits, 2)))\n",
    "\n",
    "train_source = source_int_text[batch_size:]\n",
    "train_target = target_int_text[batch_size:]\n",
    "\n",
    "valid_source = helper.pad_sentence_batch(source_int_text[:batch_size])\n",
    "valid_target = helper.pad_sentence_batch(target_int_text[:batch_size])\n",
    "\n",
    "with tf.Session(graph=train_graph) as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    for epoch_i in range(epochs):\n",
    "        for batch_i, (source_batch, target_batch) in enumerate(\n",
    "                helper.batch_data(train_source, train_target, batch_size)):\n",
    "            start_time = time.time()\n",
    "            \n",
    "            _, loss = sess.run(\n",
    "                [train_op, cost],\n",
    "                {input_data: source_batch,\n",
    "                 targets: target_batch,\n",
    "                 lr: learning_rate,\n",
    "                 sequence_length: target_batch.shape[1],\n",
    "                 keep_prob: keep_probability})\n",
    "            \n",
    "            batch_train_logits = sess.run(\n",
    "                inference_logits,\n",
    "                {input_data: source_batch, keep_prob: 1.0})\n",
    "            batch_valid_logits = sess.run(\n",
    "                inference_logits,\n",
    "                {input_data: valid_source, keep_prob: 1.0})\n",
    "                \n",
    "            train_acc = get_accuracy(target_batch, batch_train_logits)\n",
    "            valid_acc = get_accuracy(np.array(valid_target), batch_valid_logits)\n",
    "            end_time = time.time()\n",
    "            print('Epoch {:>3} Batch {:>4}/{} - Train Accuracy: {:>6.3f}, Validation Accuracy: {:>6.3f}, Loss: {:>6.3f}'\n",
    "                  .format(epoch_i, batch_i, len(source_int_text) // batch_size, train_acc, valid_acc, loss))\n",
    "\n",
    "    # Save Model\n",
    "    saver = tf.train.Saver()\n",
    "    saver.save(sess, save_path)\n",
    "    print('Model Trained and Saved')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save Parameters\n",
    "Save the `batch_size` and `save_path` parameters for inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "# Save parameters for checkpoint\n",
    "helper.save_params(save_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Checkpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import helper\n",
    "import problem_unittests as tests\n",
    "\n",
    "_, (source_vocab_to_int, target_vocab_to_int), (source_int_to_vocab, target_int_to_vocab) = helper.load_preprocess()\n",
    "load_path = helper.load_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sentence to Sequence\n",
    "To feed a sentence into the model for translation, you first need to preprocess it.  Implement the function `sentence_to_seq()` to preprocess new sentences.\n",
    "\n",
    "- Convert the sentence to lowercase\n",
    "- Convert words into ids using `vocab_to_int`\n",
    " - Convert words not in the vocabulary, to the `<UNK>` word id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def sentence_to_seq(sentence, vocab_to_int):\n",
    "    \"\"\"\n",
    "    Convert a sentence to a sequence of ids\n",
    "    :param sentence: String\n",
    "    :param vocab_to_int: Dictionary to go from the words to an id\n",
    "    :return: List of word ids\n",
    "    \"\"\"\n",
    "    word_ids = []\n",
    "    \n",
    "    for word in sentence.lower().split():\n",
    "        try:\n",
    "            word_ids.append(vocab_to_int[word])\n",
    "        except KeyError:\n",
    "            word_ids.append(vocab_to_int['<UNK>'])\n",
    "    return word_ids\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_sentence_to_seq(sentence_to_seq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Translate\n",
    "This will translate `translate_sentence` from English to French."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input\n",
      "  Word Ids:      [120, 99, 179, 83, 66, 164, 93]\n",
      "  English Words: ['he', 'saw', 'a', 'old', 'yellow', 'truck', '.']\n",
      "\n",
      "Prediction\n",
      "  Word Ids:      [100, 337, 112, 283, 53, 210, 1]\n",
      "  French Words: ['il', 'vu', 'un', 'vieux', 'camion', '.', '<EOS>']\n"
     ]
    }
   ],
   "source": [
    "translate_sentence = 'he saw a old yellow truck .'\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "translate_sentence = sentence_to_seq(translate_sentence, source_vocab_to_int)\n",
    "\n",
    "loaded_graph = tf.Graph()\n",
    "with tf.Session(graph=loaded_graph) as sess:\n",
    "    # Load saved model\n",
    "    loader = tf.train.import_meta_graph(load_path + '.meta')\n",
    "    loader.restore(sess, load_path)\n",
    "\n",
    "    input_data = loaded_graph.get_tensor_by_name('input:0')\n",
    "    logits = loaded_graph.get_tensor_by_name('logits:0')\n",
    "    keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')\n",
    "\n",
    "    translate_logits = sess.run(logits, {input_data: [translate_sentence], keep_prob: 1.0})[0]\n",
    "\n",
    "print('Input')\n",
    "print('  Word Ids:      {}'.format([i for i in translate_sentence]))\n",
    "print('  English Words: {}'.format([source_int_to_vocab[i] for i in translate_sentence]))\n",
    "\n",
    "print('\\nPrediction')\n",
    "print('  Word Ids:      {}'.format([i for i in np.argmax(translate_logits, 1)]))\n",
    "print('  French Words: {}'.format([target_int_to_vocab[i] for i in np.argmax(translate_logits, 1)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imperfect Translation\n",
    "You might notice that some sentences translate better than others.  Since the dataset you're using only has a vocabulary of 227 English words of the thousands that you use, you're only going to see good results using these words.  For this project, you don't need a perfect translation. However, if you want to create a better translation model, you'll need better data.\n",
    "\n",
    "You can train on the [WMT10 French-English corpus](http://www.statmt.org/wmt10/training-giga-fren.tar).  This dataset has more vocabulary and richer in topics discussed.  However, this will take you days to train, so make sure you've a GPU and the neural network is performing well on dataset we provided.  Just make sure you play with the WMT10 corpus after you've submitted this project.\n",
    "## Submitting This Project\n",
    "When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as \"dlnd_language_translation.ipynb\" and save it as a HTML file under \"File\" -> \"Download as\". Include the \"helper.py\" and \"problem_unittests.py\" files in your submission."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}