understanding-deep-convolutional-neural-networks-with-a-practical-use-case-in-tensorflow-and-keras.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook contains the code of the article: Understanding deep Convolutional Neural Networks with a practical use-case in Tensorflow and Keras as well as portions of the post's text. This will help understanding the code.\n",
    "\n",
    "Link <a href=\"https://ahmedbesbes.com/understanding-deep-convolutional-neural-networks-with-a-practical-use-case-in-tensorflow-and-keras.html\"> here </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is what the convolution layer does: given a filter, it scans the input and generates a feature map.\n",
    "\n",
    "> ** But what does this convolution operation really represent? How to interpret the resulting feature map? **\n",
    "\n",
    "I started by stating that convolution layers capture visual patterns within an image. Let me illustrate this to convince you.\n",
    "\n",
    "I'm going to load a cat image from the dataset. I'll then apply different convolutions on it, changing the kernel each time, and visualizing the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from scipy.signal import convolve2d\n",
    "import numpy as np \n",
    "import cv2\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "image = cv2.imread('./data/train/cats/cat.46.jpg')\n",
    "# converting the image to grayscale\n",
    "image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'll define a function that takes a kenel as an input, performs a convolution on the image and then plots the original image and the convolved one next to it.\n",
    "\n",
    "The kernel is a small square matrix (gray square in the figures above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def show_differences(kernel):\n",
    "    convolved = convolve2d(image, kernel)\n",
    "    fig = plt.figure(figsize=(15, 15))\n",
    "    plt.subplot(121)\n",
    "    plt.title('Original image')\n",
    "    plt.axis('off')\n",
    "    plt.imshow(image, cmap='gray')\n",
    "    \n",
    "    plt.subplot(122)\n",
    "    plt.title('Convolved image')\n",
    "    plt.axis('off')\n",
    "    plt.imshow(convolved, cmap='gray')\n",
    "    return convolved"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by this filter:\n",
    "$$ \\frac{1}{9} \\left[ \n",
    "\\begin{array}{cccc}\n",
    "1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 \\\\ \\end{array} \\right]$$\n",
    "\n",
    "It's called box blur. When this filter is applied to a pixel value in the input image,  it basically takes this pixel and its 8 neighbors (that's why we have 1 everywhere) and compute their average pixel value (that's why we devide by 9).\n",
    "\n",
    "Mathematically, this is a simple average. Visually, it results in smoothing abrupt contrast transitions that appear in the image.\n",
    "\n",
    "Box blur is highly used in **noise removal**.\n",
    "\n",
    "Let's apply it on a cat image and see what it does."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kernel = np.array([[1, 1, 1], [1, 1, 1], [1, 1, 1]])/9\n",
    "output = show_differences(kernel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/flou1.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you closely look at the convolved image, you'll notice that it's smoother, with less white pixels (noise) sprinkled on it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's look at a more agressive bluring filter:\n",
    "$$\\frac{1}{64}\\left[ \n",
    "\\begin{array}{cccc}\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1  \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\\n",
    "1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\\\ \\end{array} \\right]$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kernel = np.ones((8,8), np.float32)/64\n",
    "dx = show_differences(kernel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/flou2.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some filters are used to capture intrinsic image details like **edges**.\n",
    "\n",
    "Here is one example that computes an approximation of the vertical changes in the image A.\n",
    "\n",
    "$$G_x= A * \\left[ \n",
    "\\begin{array}{cccc}\n",
    "-1 & 0 & 1\\\\\n",
    "-2 & 0 & 2 \\\\\n",
    "-1 & 0 & 1 \\\\ \\end{array} \\right]$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kernel = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], np.float32)\n",
    "dx = show_differences(kernel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/dx.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The white areas are the ones that better respond to the filter, indicating the presence of a vertical edge. Look closely at the cat's left ear for example and notice how its edge is captured. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool, right? Here is a second filter that does the same operation but for the horizontal changes.\n",
    "\n",
    "\n",
    "$$G_y = A * \\left[ \n",
    "\\begin{array}{cccc}\n",
    "1 & 2 & 1\\\\\n",
    "0 & 0 & 0 \\\\\n",
    "-1 & -2 & -1 \\\\ \\end{array} \\right]$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kernel = np.array([[1, 2, 1], [0, 0, 0], [-1, -2, -1]], np.float32)\n",
    "dy = show_differences(kernel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/dy.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the whiskers are detected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two previous filters are gradient operators. They allow, to some extent, to reveal an inherent structure in the image with respect to a direction. \n",
    "\n",
    "However, when G_x and G_y are combined in the following formula:\n",
    "\n",
    "$$G = \\sqrt{G_x^2 + G_y^2}$$\n",
    "\n",
    "they allow an even better edge detection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mag = np.hypot(dx, dy)  # magnitude\n",
    "mag *= 255.0 / np.max(mag)  # normalize (Q&D)\n",
    "\n",
    "fig = plt.figure(figsize=(15, 15))\n",
    "plt.subplot(121)\n",
    "plt.title('Original image')\n",
    "plt.axis('off')\n",
    "plt.imshow(image, cmap='gray')\n",
    "\n",
    "plt.subplot(122)\n",
    "plt.title('Convoluted image with highlighted edges')\n",
    "plt.axis('off')\n",
    "plt.imshow(mag, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/edges.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 - 1 Builiding a convnet from scratch\n",
    "\n",
    "In this first section, we'll setup an end-to-end pipeline to train a CNN. We'll go through data preparation and augmentation, architecture design, training and validation. We'll plot the loss and accuracy metrics on both the train and validation set: this will allow us to assess the improvement of the model over the training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to do before starting is to download and unzip the <a href=\"https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition/download/train.zip\" style=\"text-decoration:none;\">train</a> dataset from Kaggle.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "import os \n",
    "import cv2\n",
    "from tqdm import tqdm_notebook\n",
    "from random import shuffle\n",
    "import shutil\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We must now organize the data so that it's easily processed by keras.\n",
    "\n",
    "We'll create a data folder in which we'll have in two subfolders:\n",
    "\n",
    "+ train\n",
    "+ validation\n",
    "\n",
    "Each of them will have 2 folders: \n",
    "\n",
    "+ cats \n",
    "+ dogs\n",
    "\n",
    "At the end we'll have the following structure:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "```javascript\n",
    "data/\n",
    "    train/\n",
    "        dogs/\n",
    "            dog001.jpg\n",
    "            dog002.jpg\n",
    "            ...\n",
    "        cats/\n",
    "            cat001.jpg\n",
    "            cat002.jpg\n",
    "            ...\n",
    "    validation/\n",
    "        dogs/\n",
    "            dog001.jpg\n",
    "            dog002.jpg\n",
    "            ...\n",
    "        cats/\n",
    "            cat001.jpg\n",
    "            cat002.jpg\n",
    "```            \n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "This structrue will allow our model to know from which folder to fetch the images as well as their labels for either training or validation. \n",
    "\n",
    "Here's a function that allows you to construct this file tree. It has two parameters: the total number of images **n** and the ratio of validation set **r**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def organize_datasets(path_to_data, n=4000, ratio=0.2):\n",
    "    files = os.listdir(path_to_data)\n",
    "    files = [os.path.join(path_to_data, f) for f in files]\n",
    "    shuffle(files)\n",
    "    files = files[:n]\n",
    "    \n",
    "    n = int(len(files) * ratio)\n",
    "    val, train = files[:n], files[n:]\n",
    "    \n",
    "\n",
    "    shutil.rmtree('./data/')\n",
    "    print('/data/ removed')\n",
    "\n",
    "    for c in ['dogs', 'cats']: \n",
    "        os.makedirs('./data/train/{0}/'.format(c))\n",
    "        os.makedirs('./data/validation/{0}/'.format(c))\n",
    "\n",
    "    print('folders created !')\n",
    "\n",
    "    for t in tqdm_notebook(train):\n",
    "        if 'cat' in t:\n",
    "            shutil.copy2(t, os.path.join('.', 'data', 'train', 'cats'))\n",
    "        else:\n",
    "            shutil.copy2(t, os.path.join('.', 'data', 'train', 'dogs'))\n",
    "     \n",
    "    for v in tqdm_notebook(val):\n",
    "        if 'cat' in v:\n",
    "            shutil.copy2(v, os.path.join('.', 'data', 'validation', 'cats'))\n",
    "        else:\n",
    "            shutil.copy2(v, os.path.join('.', 'data', 'validation', 'dogs'))\n",
    "            \n",
    "    print('Data copied!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I used:\n",
    "\n",
    "+ n : 25000 (the entire dataset)\n",
    "+ r : 0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "ratio = 0.2\n",
    "n = 25000\n",
    "organize_datasets(path_to_data='./train/', n=n, ratio=ratio)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load Keras and its dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras_tqdm import TQDMNotebookCallback\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense\n",
    "from keras.layers import Dropout\n",
    "from keras.layers import Flatten\n",
    "from keras.constraints import maxnorm\n",
    "from keras.optimizers import SGD\n",
    "from keras.layers.convolutional import Conv2D\n",
    "from keras.layers.convolutional import MaxPooling2D\n",
    "from keras.utils import np_utils\n",
    "from keras.callbacks import Callback"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Image generators and data augmentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When training a model, we're not going to load the entire dataset in memory. This won't be efficient, especially if you use your local machine.\n",
    "\n",
    "We're going to use the ImageDataGenerator class. It allows you to indefinitely stream the images by batches from the train and validation folders. Every batch flows through the network, makes a forward-prop, a back-prop then a parameter updates (along with the test on the validation data). Then comes the next batch and does the same thing, etc.\n",
    "\n",
    "Inside the ImageDataGenerator object, we're going to introduce random modifications on each batch. It's a process we call **data augmentation**. It allows to generate more data so that our model would never see twice the exact same picture. This helps prevent overfitting and makes the model generalize better. \n",
    "\n",
    "We'll create two ImageDataGenerator objects.\n",
    "\n",
    "**train_datagen** for the training set and **val_datagen** one for the validation set. The two of them will apply a rescaling on the image, but the train_datagen will introduce more modifications."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "batch_size = 32"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "train_datagen = ImageDataGenerator(rescale=1/255.,\n",
    "                                    shear_range=0.2,\n",
    "                                    zoom_range=0.2,\n",
    "                                    horizontal_flip=True\n",
    "                                    )\n",
    "val_datagen = ImageDataGenerator(rescale=1/255.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the two previous objects we're going to create two file generators:\n",
    "\n",
    "+ train_generator\n",
    "+ validation_generator\n",
    "\n",
    "Each one generates, from its directory, batches of tensor image data with real-time data augmentation. The data will be looped over (in batches) indefinitely."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 20000 images belonging to 2 classes.\n",
      "Found 5000 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "train_generator = train_datagen.flow_from_directory(\n",
    "        './data/train/',\n",
    "        target_size=(150, 150),\n",
    "        batch_size=batch_size,\n",
    "        class_mode='categorical')\n",
    "\n",
    "validation_generator = val_datagen.flow_from_directory(\n",
    "        './data/validation/',\n",
    "        target_size=(150, 150),\n",
    "        batch_size=batch_size,\n",
    "        class_mode='categorical')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'll use a CNN with three convolution/pooling layers and two fully connected layers.\n",
    "\n",
    "The three conv layers will use respectively 32, 32 and 64 3x3 filters.\n",
    "\n",
    "I used dropout on the two fully connected layers to prevent overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(32, (3, 3), input_shape=(150, 150, 3), padding='same', activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Conv2D(64, (3, 3), activation='relu', padding='same'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "\n",
    "model.add(Dropout(0.25))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(64, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(2, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I used the stochastic gradient descent optimizer with learning rate of 0.01 and momentum of 0.9. \n",
    "\n",
    "I used the **binary crossentropy** loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "epochs = 50\n",
    "lrate = 0.01\n",
    "decay = lrate/epochs\n",
    "sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)\n",
    "model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keras provides a convenient method to display a summary of the model. For each layer, this shows up the output shape and the number of trainable parameters.\n",
    "\n",
    "This a sanity check before starting fitting the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 150, 150, 32)      896       \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 75, 75, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 75, 75, 32)        9248      \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 37, 37, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 37, 37, 64)        18496     \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 18, 18, 64)        0         \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 18, 18, 64)        0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 20736)             0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                1327168   \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 64)                0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 2)                 130       \n",
      "=================================================================\n",
      "Total params: 1,355,938\n",
      "Trainable params: 1,355,938\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the network architecture:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing the architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<img src=\"./images/model.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Training the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before training the model, I defined two callback functions that will be called while training.\n",
    "\n",
    "+ One for **early stopping** the training when the loss function stops improving on the validation data  \n",
    "+ One for storing the validation loss and accuracy of each epoch: this allows to plot the training error\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "## Callback for loss logging per epoch\n",
    "class LossHistory(Callback):\n",
    "    def on_train_begin(self, logs={}):\n",
    "        self.losses = []\n",
    "        self.val_losses = []\n",
    "        \n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "        self.losses.append(logs.get('loss'))\n",
    "        self.val_losses.append(logs.get('val_loss'))\n",
    "        \n",
    "history = LossHistory()\n",
    "\n",
    "## Callback for early stopping the training\n",
    "early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss',\n",
    "                              min_delta=0,\n",
    "                              patience=2,\n",
    "                              verbose=0, mode='auto')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I also used <a  href=\"https://github.com/bstriner/keras-tqdm\"> keras-tqdm</a> which is an **awesome progress-bar ** that perfectly integrates with Keras. \n",
    "\n",
    "It allows you to monitor the training of you models very easily.\n",
    "\n",
    "What you need to do is simply load the TQDMNotebookCallback class from keras_tqdm then pass it as a third callback functions.\n",
    "\n",
    "```python\n",
    "from keras_tqdm import TQDMNotebookCallback\n",
    "```\n",
    "\n",
    "Here's what keras-tqdm looks like on simple example:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/keras-tqdm.png\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A few words about the training:\n",
    "\n",
    "- We'll use the fit_generator method which is a variant (of the standard fit method) that takes a generator as input.\n",
    "\n",
    "- We'll train the model over 50 epochs: over one epoch 20000 unique and augmented images will flow by batch of 32 to the network, performing a forward and back propagation and adjusting the weights with SGD. The idea of using multiple epochs is to prevent overfitting.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true
   },
   "source": [
    "<hr>\n",
    "```python\n",
    "fitted_model = model.fit_generator(\n",
    "        train_generator,\n",
    "        steps_per_epoch= int(n * (1-ratio)) // batch_size,\n",
    "        epochs=50,\n",
    "        validation_data=validation_generator,\n",
    "        validation_steps= int(n * ratio) // batch_size,\n",
    "        callbacks=[TQDMNotebookCallback(leave_inner=True, leave_outer=True), early_stopping, history],\n",
    "        verbose=0)\n",
    "```\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a **heavy computation**: \n",
    "\n",
    "- If you're on your laptop this may take about 15 minutes per epoch\n",
    "- If you're using an the p2.xlarge EC2 instance like me, this takes about 2 minutes or so per epoch\n",
    "\n",
    "tqdm allows you to monitor the validation loss and accuracy on each epochs. This is useful to check the quality of your model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classification results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We reached **89.4% accuracy** (on the validation data) in 34 epochs (training/validation error and accuracy displayed below) \n",
    "\n",
    "This is a good result given that I didn't invest too much time designing the architecture of the network.\n",
    "\n",
    "Now let's save the model for later use."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<hr>\n",
    "```python\n",
    "model.save('./models/model4.h5')\n",
    "```\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot the train and validation losses on the same graph:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "```python\n",
    "losses, val_losses = history.losses, history.val_losses\n",
    "fig = plt.figure(figsize=(15, 5))\n",
    "plt.plot(fitted_model.history['loss'], 'g', label=\"train losses\")\n",
    "plt.plot(fitted_model.history['val_loss'], 'r', label=\"val losses\")\n",
    "plt.grid(True)\n",
    "plt.title('Training loss vs. Validation loss')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "```\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/loss.png\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We interrupt the training when the validation loss doesn't improve on two successive epochs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now plot the accuracy on both the training set and validation set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "```python\n",
    "losses, val_losses = history.losses, history.val_losses\n",
    "fig = plt.figure(figsize=(15, 5))\n",
    "plt.plot(fitted_model.history['acc'], 'g', label=\"accuracy on train set\")\n",
    "plt.plot(fitted_model.history['val_acc'], 'r', label=\"accuracy on validation set\")\n",
    "plt.grid(True)\n",
    "plt.title('Training Accuracy vs. Validation Accuracy')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()\n",
    "```\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/accuracy.png\"></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These two metrics keep increasing before reaching a plateau where the model eventually starts overfitting (from epoch 34)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 - 2 Loading a pre-trained model "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far so good: we designed a custom convnet that performs reasonably well on the valiation data with ~ 89% accuracy.\n",
    " \n",
    "There is however a way to get a better score: loading the weights of a pre-trained convnet on a large dataset that includes images of cats and dogs among 1000 classes. Such a network would have learnt relevant features that are relevant to our classification.\n",
    "\n",
    "I'll load the weights of the VGG16 network: more specifically, I'm going to load the network weights up to the last conv layer. This network part acts as a feature detector to which we're going to add fully connected layers for our classification task.\n",
    " \n",
    "VGG16 is a very large network in comparison to LeNet5. It has 16 layers with trainable weights and around 140 millions parameters. To learn more about VGG16 please refer to this <a href=\"https://arxiv.org/pdf/1409.1556.pdf\" style=\"text-decoration:none;\"> pdf link.</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first start by loading the VGG16 weights (trained on ImageNet) by specifying that we're not interested in the last three FC layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from keras import applications\n",
    "# include_top: whether to include the 3 fully-connected layers at the top of the network.\n",
    "model = applications.VGG16(include_top=False, weights='imagenet')\n",
    "datagen = ImageDataGenerator(rescale=1. / 255)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we pass the images through the network to get a feature representation that we'll input to a neural network classifier.\n",
    "\n",
    "We do this for the training set and the validation set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 20000 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "generator = datagen.flow_from_directory('./data/train/',\n",
    "                                        target_size=(150, 150),\n",
    "                                        batch_size=batch_size,\n",
    "                                        class_mode=None,\n",
    "                                        shuffle=False)\n",
    "\n",
    "bottleneck_features_train = model.predict_generator(generator, int(n * (1 - ratio)) // batch_size)\n",
    "np.save(open('./features/bottleneck_features_train.npy', 'wb'), bottleneck_features_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 5000 images belonging to 2 classes.\n"
     ]
    }
   ],
   "source": [
    "generator = datagen.flow_from_directory('./data/validation/',\n",
    "                                        target_size=(150, 150),\n",
    "                                        batch_size=batch_size,\n",
    "                                        class_mode=None,\n",
    "                                        shuffle=False)\n",
    "\n",
    "bottleneck_features_validation = model.predict_generator(generator, int(n * ratio) // batch_size,)\n",
    "np.save('./features/bottleneck_features_validation.npy', bottleneck_features_validation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the images are passed to the network, they are in the right orders. So we associate the labels to them easily."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "train_data = np.load('./features/bottleneck_features_train.npy')\n",
    "train_labels = np.array([0] * (int((1-ratio) * n) // 2) + [1] * (int((1 - ratio) * n) // 2))\n",
    "\n",
    "validation_data = np.load('./features/bottleneck_features_validation.npy')\n",
    "validation_labels = np.array([0] * (int(ratio * n) // 2) + [1] * (int(ratio * n) // 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we design a small fully connected network that plugs in to the features extracted from the VGG16 and acts as the classification part of a CNN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "model = Sequential()\n",
    "model.add(Flatten(input_shape=train_data.shape[1:]))\n",
    "model.add(Dense(512, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(Dense(256, activation='relu'))\n",
    "model.add(Dropout(0.2))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='binary_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r"
     ]
    }
   ],
   "source": [
    "fitted_model = model.fit(train_data, train_labels,\n",
    "          epochs=15,\n",
    "          batch_size=batch_size,\n",
    "          validation_data=(validation_data, validation_labels[:validation_data.shape[0]]),\n",
    "          verbose=0,\n",
    "          callbacks=[TQDMNotebookCallback(leave_inner=True, leave_outer=False), history])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We reached **90.7% accuracy** on 15 epochs only. Quite not bad.\n",
    "\n",
    "Note that each epoch takes around 1 minute on my personal laptop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x174ddb00>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3sAAAFNCAYAAAC5cXZ6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VOX9/vH3JwkJ+xYgIAEyKqLsmwgqkBxbi7vYWrVa\nl1ap1rZal2+t/bW1tYsWW5dWRWpbt7pWrUtRaAsBxI3FgFCQfQfZZAlbtuf3xzMhQwgkkExOJrlf\n13WuZM6cOfOZeRKYO89yzDmHiIiIiIiI1C9JYRcgIiIiIiIiNU9hT0REREREpB5S2BMREREREamH\nFPZERERERETqIYU9ERERERGRekhhT0REREREpB5S2BMRqcPMLNnM8s2sa00eewx1/MrMnqrp8yYK\nM7vezHKj3x/xfY499hifa5KZXXmsjz/CeZ8zs3tq+rwiIlJ3KeyJiNSgaAgo3UrMbG/M7aP+AO+c\nK3bONXfOra7JYxsSM2tqZjvNbEQF9/3RzF48mvPV5PtcUYh2zp3tnPt7dc9dVx1Ne5jZlWY208x2\nm9kmM/vQzG4s95ghZjbBzLZHtwVmdq+ZtY7e39nM3jKzDWbmzCyzguc928w+iT7PGjO7JB6vXUSk\ntinsiYjUoGgIaO6caw6sBi6I2XfIB3gzS6n9KhsW59we4BXg6tj9ZtYIuBx4Ooy6GqqqtoeZ/Qj4\nPfBbICO6fRcYaWbJ0WOGA5OBXOAk51xr4NzoKXtHv5YAE4CvVVSPmfUBngXuAloBA4C86r9SEZHw\nKeyJiNSiaE/OS2b2gpntAq4ys2HRHovt0d6HR6IffDGzlGhvRFb09nPR+98xs11m9oGZRY722Oj9\n55jZYjPbEe1RmWFm11bxdYyO9qBsN7PJZtYj5r67zWx9tPdmkZllR/cPNbM50f2fm9nYw5x7iZmN\nirmdambbzKxvtFfoeTPbGn3uj82sXRVKfhr4mpk1jtl3DlAETIo+z/8zs+XR92qBmV14mPrKv8/t\nzezt6Ov6EIiUO/5PZrY2ev9MMzs9uv984P+AK6M9v7Oj+98rbQczSzKzn5nZqmjP1lNm1jJ634nR\nOq6Onn+zmd1VhfeitK4bzWxp9L38p5l1innOR6LPt8PM5plZz9KazWxh9D1aa2Y/rOrzlXPE9jCz\nNsA9wHecc6855/KdN8c5d4Vzrjj6mLHAn51zv3PObQJwzq1yzv3UOfde9PYG59zjwOzD1PJT4DHn\n3ETnXJFzbotzbvkxvi4RkTpFYU9EpPaNBp7H9yK8hP+AewvQDjgDGAV85wiP/wb+A2pbfO/hvUd7\nrJl1AF4G7ow+7wpgSFWKN7NT8D0h3wfaA/8B3jSzRmbWK1r7QOdcS/wH+NLhjn8Exkb3nwj84zBP\n8QJwRcztc4D1zrl5wHVAUyATSMf39OyrQtnTgW3AxTH7vgn8PSY4LMa//62AXwPPm1lGFc79OLAL\n6AiMAb5V7v6PgL74NvgH8IqZpTnn3gZ+F62huXNuUAXnvh64CsgGTgDaAA+XO+Z0/Pv5FeAXZta9\nsoLN7Gzgl/jers7AeqC05/kcYCjQPfp8l+PfO4C/Ad92zrWIvqaplT3XYVTWHmfgP6O8dYTX0BL/\nM/vqMdZQaiiQZGbzo39seSYaNkVEEp7CnohI7XvPOfeWc67EObfXOTfTOfdRtFdhOTAeGHmEx//D\nOTfLOVeI/4De/xiOPR/Ic869Eb3vQWBLFeu/HHjTOTc5+tj78AHpNHxwbQz0MrMU59yKmF6SQqC7\nmaU753Y55z46zPmfBy6O6fX5RnRf6TnaASdG587Ncs7lV1awc84BzxAdOmh+PtcFxAzhdM69HO0F\nKnHOPQ+sBAYf6bzRHtiLgZ865/ZEA+mz5Z77WefcNudcET7clYbdqrgSeCD6Pu4C7ga+YWax/3/f\n45zb55ybAywA+lXxvE865/Kcc/vwQxhHmp/PVhit8eRo/f9zzm2MPq4Q6GlmLaKvaU4VX8dBqtAe\n7YDNzrmS0sdEe3G3m58Hezo+PBuwMeaYP0SP2V2VXk4zM3zYvQrfjidFX/tDx/K6RETqGoU9EZHa\ntyb2hpmdbGb/MrONZrYT3+NypKGJG2O+3wM0P4Zjj4utI/rhe20Vai997KqYx5ZEH9vZOfcZcDv+\nNWwyP1y1Y/TQ64CewGfRD+7nUgHn3CJgGXCemTXHB9PSsPcUvifxZTNbZ2b3WdXnPT4DfDnaW/d1\nYKFz7tPSO83sWjObGw0L2/Fhp7IhohlAMge36arYA8zs/6LDWXcAXwDNqnDeUge919HvU/E9qgDE\nBDGo/OehwvM653ZGa+vsnJsEjMP3WH5uZuPMrEX00NHAhcBqM8s1s9MqOrmZfWZlCxMNO0wNR2qP\nrUCH2FDrnBsSnZO3A//5ZRvggE4xx9wWPeYtoNKfi+jP/T7gr865pdFA/VvK5v2JiCQ0hT0Rkdrn\nyt1+ApiP761qCfwM32MRTxvwQyGBg3o4qmI90C3msUnRc60DcM4955w7Az93LRn/4Rnn3GfOucuB\nDviFN14tN2crVulQztH4HsiV0XMUOOfucc6dApwZvb9Kq5xGexg/iB7/TWJ69czseHy4uQlIjwaG\nRVTeDp/jFwDpErPvwCUZzCwHuA34KtAaPywyP+a85X8WyjvovY6euwDYXMnjKlO+DVtEayttw4ec\ncwPxi5z0jL4Goj3QF+Lb8G2gwpVMnXM9YhYm+uAwxxy2PYAZQDG+t69C0YA6C6juypnzOLgdKmsT\nEZGEobAnIhK+Fvjeit3R+XBHmq9XU94GBprZBdGesVuI6S2qxMvAhWaWHR3GeCd+ztpHZnaKmeWY\nWRqwN7qVAJjZN82sXbQncAf+Q3VJxU/BC/i5Y2Mo69XDzAIz6x0NmDvxwwoPd46KPB19rafFnhff\nG+bwIcrM7AaiwxiPJDqM9Z/4uXJNzKw3PriUaoEf2roFaIRfdKRZzP2fA1nRsF2RF4DbzCwrGsh+\nDbwQO7zxGL0AfNv8ojdp+EA+3Tm31vylDIZEfy5248NlSfT1fcPMWkZf9y6O7r2vSIXt4Zzbhp9f\nOs7MLjGz5tGFYwYATWIefycwJtp72h7AzLpwcEAm+keFtOjNtOhrLvW36HuRZWZNgR/hfz9ERBKe\nwp6ISPhuB67Bf3h+Ar9oS1w55z4HLgP+gB8ydwLwCbC/Co9dgK/3cXw4GgVcGA0Aafh5aVvwQ0jb\nAD+JPvRcYKH5VUgfAC5zzhUc5jnW4ntthuLDZanjgNfwQW8Bfkjn8wBm9qSZ/amS8l/BD6GcWLp6\nY/T55uEXkPkY3+vZA7+wSlXcFH2dnwN/wYeHUhOiNS7BzwHcGT1/qZfwwzK3mdnHFZz7z9FjpgPL\n8T8jt1SxrsNyzr2LH2r7erSerpT1kLaOvo7t0Zo34H9OwLf7quhw42/j57pVR4XtEa3xN/jgdTew\nCf/z9Dj+9+Wj6DFTgS8BOcDS6PDbd/Dv+WNw4PIme6OvB2ApPsSW+jM+/M7CD23dDRzrKqMiInWK\n+eHqIiLSkJm/btl64GvOuelh1yMiIiLVp549EZEGysxGmVnr6JC2n+KHRFbUuyQiIiIJSGFPRKTh\nOhM/NHAz/hpto51zlQ7jFBERkcSgYZwiIiIiIiL1kHr2RERERERE6iGFPRERERERkXooJewCjla7\ndu1cVlZW2GUcYvfu3TRr1qzyA6XOUJslHrVZYlF7JR61WeJRmyUWtVfiqattNnv27C3OuUqvj5tw\nYS8rK4tZs2aFXcYhcnNzyc7ODrsMOQpqs8SjNkssaq/EozZLPGqzxKL2Sjx1tc3MbFVVjtMwThER\nERERkXpIYU9ERERERKQeUtgTERERERGphxJuzp6IiIiIiISjsLCQtWvXsm/fvrBLqRWtWrVi4cKF\noT1/48aNyczMpFGjRsf0eIU9ERERERGpkrVr19KiRQuysrIws7DLibtdu3bRokWLUJ7bOcfWrVtZ\nu3YtkUjkmM6hYZwiIiIiIlIl+/btIz09vUEEvbCZGenp6dXqRVXYExERERGRKlPQqz3Vfa8V9kRE\nREREJCFs376dxx577Jgee+6557J9+/YqH3/PPffwyCOPHNNz1RUKeyIiIiIikhCOFPaKioqO+NgJ\nEybQunXreJRVZ8U17JnZKDP7zMyWmtldFdzfxsxeN7N5ZvaxmfWOZz3xMu/zeby5/s2wyxARERER\nqdfuuusuli1bRv/+/bnzzjvJzc1l+PDhXHjhhfTs2ROAiy++mEGDBtGrVy/Gjx9/4LFZWVls2bKF\nlStXcsopp3DDDTfQq1cvzj77bPbu3XvE583Ly2Po0KH07duX0aNH88UXXwDwyCOP0LNnT/r27cvl\nl18OwNSpU+nfvz/9+/dnwIAB7Nq1C4CxY8dy6qmn0rdvX37+858DsHv3bs477zz69etH7969eeml\nl2r0/Ypb2DOzZOBR4BygJ3CFmfUsd9jdQJ5zri9wNfBwvOqJp38u+icPLXmIL/Z+EXYpIiIiIiL1\n1n333ccJJ5xAXl4eY8eOBWDOnDk8/PDDLF68GIC//vWvzJ49m1mzZvHII4+wdevWQ86zZMkSbr75\nZhYsWEDr1q159dVXj/i8V199Nffffz/z5s2jT58+/OIXvzhQzyeffMK8efMYN24cAA888ACPPvoo\neXl5TJ8+nSZNmjBp0iSWLFnCxx9/TF5eHrNnz2batGm8++67HHfcccydO5f58+czatSomny74nrp\nhSHAUufccgAzexG4CPhfzDE9gfsAnHOLzCzLzDKcc5/Hsa4aF0QCfp77c6aumsrFJ18cdjkiIiIi\nInF367u3krcxr0bP2b9jfx4a9dBRPWbIkCEHXZrgkUce4fXXXwdgzZo1LFmyhPT09IMeE4lE6N+/\nPwCDBg1i5cqVhz3/jh072L59OyNHjgTgmmuu4dJLLwWgb9++XHnllVx88cVcfLHPAWeccQa33XYb\nV155JZdccgmZmZlMmjSJSZMmMWDAAADy8/NZsmQJw4cP5/bbb+dHP/oR559/PsOHDz+q116ZeIa9\nzsCamNtrgdPKHTMXuASYbmZDgG5AJnBQ2DOzMcAYgIyMDHJzc+NU8rEpLCkkLSmNZ957htYbG9Y4\n4ESWn59f536W5MjUZolF7ZV41GaJR22WWOpDe7Vq1erAsMSCggKKi4tr9PwFBQUHzl+R/Px8SkpK\nDhyzZ88e0tLSDtyePn06EydOZNKkSTRt2pRzzz2Xbdu2sWvXLpxz5Ofnk5+fT6NGjQ48pqioiN27\ndx/yvPv37yc5OfnAY0vvj63hxRdfZMaMGbzzzjvce++9fPjhh9x8881kZ2czadIkTj/9dF5//XX2\n79/PD3/4Q771rW8d8pqmTp3KpEmT+PGPf8zIkSO5666DZ7/t27fvmH9uwr6o+n3Aw2aWB3wKfAIc\n8hPjnBsPjAcYPHiwy87Ors0aq6Tv/L4sLlhMXaxNKpabm6v2SjBqs8Si9ko8arPEozZLLPWhvRYu\nXHjgIuOPXXhsq2JWR6dOndi9e/eBGpo2bUpKSsqB24WFhbRr146MjAwWLVrEzJkzadq0KS1atMDM\naN68OQBJSUkHHpOWlkZhYeEhF09PS0sjKSmJzMxM2rZtS15eHsOHD+f1118nJyeHZs2asXr1as47\n7zzOPvtsunXrhpmxadMmhg4dytChQ5k3bx5r1qzhggsu4Kc//Snf/va3ad68OevWraNRo0YUFRWR\nkZHBDTfcQKdOnXjyyScPqaNx48YHegSPVjzD3jqgS8ztzOi+A5xzO4HrAMxfRGIFsDyONcXNgNYD\nGL9iPJ/nf05G84ywyxERERERqXfS09M544wz6N27N+eccw7nnXfeQfePGjWKcePGccopp9CjRw+G\nDh1aI8/79NNPc+ONN7Jnzx6OP/54/va3v1FcXMxVV13Fjh07cM7xgx/8gNatW/PTn/6UKVOmkJSU\nRK9evTjnnHNIS0tj4cKFDBs2DIDmzZvz3HPPsXTpUu68806SkpJo1KgRjz/+eI3UWyqeYW8m0N3M\nIviQdznwjdgDzKw1sMc5VwBcD0yLBsCEM7DNQFgBU1ZO4fLel4ddjoiIiIhIvfT8888fdDu2tzQt\nLY133nmnwseVzstr164d8+fPP7D/jjvuqPD4e+6558DQzf79+/Phhx8ecsx77713yL4//vGPFZ7v\nlltu4ZZbbjlo3wknnMBXvvKVCo+vCXFbjdM5VwR8D5gILAReds4tMLMbzezG6GGnAPPN7DP8qp23\nVHy2uu/E5ifSKq0Vk1dMDrsUERERERGR+M7Zc85NACaU2zcu5vsPgJPiWUNtSbZkRmaNVNgTERER\nEZE6Ia4XVW9ogqyAZV8sY9X2VWGXIiIiIiIiDZzCXg0KIgHg5+2JiIiIiIiESWGvBvXq0Iv2Tdtr\nKKeIiIiIiIROYa8GJVkSOZEcJq+YjHMu7HJERERERKQBU9irYUFWwLpd61iybUnYpYiIiIiINHil\nF1Kv6v76RGGvhpXO29NQThERERERCZPCXg07se2JZLbMVNgTEREREalhd911F48++uiB2/fccw8P\nPPAA+fn5nHXWWQwcOJA+ffrwxhtvVPmczjnuvPNOevfuTZ8+fXjppZcA2LBhA6NGjaJ///707t2b\n6dOnU1xczLXXXnvg2AcffBCAZcuWMWrUKAYNGsTw4cNZtGgRAK+88gq9e/emX79+jBgxogbfiaqJ\n63X2GiIzI4gETFgygRJXQpIpT4uIiIiI1ITLLruMW2+9lZtvvhmAl19+mYkTJ9K4cWNef/11WrZs\nyZYtWxg6dCgXXnghZlbpOV977TXy8vKYO3cuW7Zs4dRTT2XEiBE8//zznHXWWfzyl7+kuLiYPXv2\nkJeXx7p165g/fz4A27dvB2DMmDGMGzeO7t2789FHH/Hd736XyZMn88tf/pKJEyfSuXPnA8fWJoW9\nOAiyAp6Z+wzzN82nb0bfsMsREREREal5t94KeXk1e87+/eGhhw5794ABA9i0aRPr169n8+bNtGnT\nhi5dulBYWMjdd9/NtGnTSEpKYt26dXz++ed07Nix0qd87733uOKKK0hOTiYjI4ORI0cyc+ZMTj31\nVK699lqSkpK4+OKL6d+/P8cffzzLly/n+9//Pueddx5nn302+fn5vP/++1x66aUHzrl//34Azjjj\nDK699lq+/vWvc8kll1T//TlK6naKg5xIDqB5eyIiIiIiNe3SSy/lH//4By+99BKXXXYZAH//+9/Z\nvHkzs2fPJi8vj4yMDPbt21et5xkxYgTvvvsunTt35tprr+WZZ56hTZs2zJ07l+zsbMaNG8f1119P\nSUkJrVu3Ji8v78C2cOFCAMaNG8evfvUr1qxZw6BBg9i6dWu1X//RUM9eHHRt1ZUT257I5BWTuXXo\nrWGXIyIiIiJS847QAxdPl112GTfccANbtmxh6tSpAOzYsYMOHTrQqFEjpkyZwqpVq6p8vuHDh/PE\nE09wzTXXsG3bNqZNm8bYsWNZtWoVHTp04IYbbmD//v3MmTOHc889l9TUVL761a/So0cPrrrqKlq2\nbEkkEuGVV17h0ksvxTnHvHnz6NevH8uWLeO0007jtNNO45133mHNmjWkp6fH6605hMJenARZAS8u\neJGikiJSkvQ2i4iIiIjUhF69erFr1y46d+5Mp06dALjyyiu54IIL6NOnD4MHD+bkk0+u8vlGjx7N\nBx98QL9+/TAzfve739GxY0eefvpp7r//ftLS0mjevDnPPPMM69at47rrrqOkpASA3/72t4DvWbzp\nppv41a9+RWFhIZdffjn9+vXjzjvvZMmSJTjnOOuss+jXr1/NvyFHoBQSJ0EkYPyc8czZMIchnYeE\nXY6IiIiISL3x6aefHnS7Xbt2fPDBBxUem5+ff8T9ZsbYsWMZO3bsQfdfc801XHLJJbRo0eKg/XPm\nzDnkXJFIhHffffeQ/a+99trhX0Qt0Jy9OMnOygY0b09ERERERMKhsBcnGc0z6N2ht8KeiIiIiIiE\nQmEvjoKsgPdWv8f+ov1hlyIiIiIiIg2Mwl4cBZGAvUV7+WjdR2GXIiIiIiJSI5xzYZfQYFT3vVbY\ni6ORWSNJsiQN5RQRERGReqFx48Zs3bpVga8WOOfYunUrjRs3PuZzaDXOOGrduDUDOw1k8orJ3JN9\nT9jliIiIiIhUS2ZmJmvXrmXz5s1hl1Ir9u3bV62wVV2NGzcmMzPzmB+vsBdnQVbAgx8+yO6C3TRL\nbRZ2OSIiIiIix6xRo0ZEIpGwy6g1ubm5DBgwIOwyjpmGccZZEAkoLClkxpoZYZciIiIiIiINiMJe\nnJ3Z9UxSklI0b09ERERERGqVwl6cNUttxtDMoQp7IiIiIiJSq+Ia9sxslJl9ZmZLzeyuCu5vZWZv\nmdlcM1tgZtfFs56wBFkBszfMZvu+7WGXIiIiIiIiDUTcwp6ZJQOPAucAPYErzKxnucNuBv7nnOsH\nZAO/N7PUeNUUliASUOJKmLZqWtiliIiIiIhIAxHPnr0hwFLn3HLnXAHwInBRuWMc0MLMDGgObAOK\n4lhTKIZmDqVxSmMN5RQRERERkVoTz7DXGVgTc3ttdF+sPwGnAOuBT4FbnHMlcawpFGkpaZzZ9UyF\nPRERERERqTXmnIvPic2+Boxyzl0fvf1N4DTn3PfKHXMGcBtwAvBvoJ9zbme5c40BxgBkZGQMevHF\nF+NSc3Xk5+fTvHnzw97/99V/58kVT/LasNdok9qmFiuTw6mszaTuUZslFrVX4lGbJR61WWJReyWe\nutpmOTk5s51zgys7Lp4XVV8HdIm5nRndF+s64D7nE+dSM1sBnAx8HHuQc248MB5g8ODBLjs7O141\nH7Pc3FyOVFeTtU148i9PUti5kOxehz9Oak9lbSZ1j9ossai9Eo/aLPGozRKL2ivxJHqbxXMY50yg\nu5lFoouuXA68We6Y1cBZAGaWAfQAlsexptAMOm4QLVJbaCiniIiIiIjUirj17Dnniszse8BEIBn4\nq3NugZndGL1/HHAv8JSZfQoY8CPn3JZ41RSmlKQURmaNVNgTEREREZFaEc9hnDjnJgATyu0bF/P9\neuDseNZQlwRZAW8vfps1O9bQpVWXyh8gIiIiIiJyjOJ6UXU5WBAJAJiyckrIlYiIiIiISH2nsFeL\n+mT0Ib1JuoZyioiIiIhI3Cns1aIkSyInksPkFZOJ1yUvREREREREQGGv1gVZAWt2rmHZF8vCLkVE\nREREROoxhb1aVjpvT0M5RUREREQknhT2atlJ6SdxXIvjFPZERERERCSuFPZqmZkRRALN2xMRERER\nkbhS2AtBkBWwec9mFmxeEHYpIiIiIiJSTynshUDz9kREREREJN4U9kLQrXU3jm9zvMKeiIiIiIjE\njcJeSIKsgNyVuRSXFIddioiIiIiI1EMKeyEJIgE79u/gk42fhF2KiIiIiIjUQwp7IcmJ5AAwZcWU\nkCsREREREZH6SGEvJB2bd6Rn+55MXql5eyIiIiIiUvMU9kIUZAVMXzWdguKCsEsREREREZF6RmEv\nREEkYHfhbmaumxl2KSIiIiIiUs8o7IVoZNZIDNMlGEREREREpMYp7IWobZO2DOg0QPP2RERERESk\nxinshSzICnh/zfvsLdwbdikiIiIiIlKPKOyFLIgEFBQX8P6a98MuRURERERE6hGFvZCd2fVMUpJS\nNG9PRERERERqlMJeyFqktWBI5yGatyciIiIiIjVKYa8OCLICZq6byc79O8MuRURERERE6gmFvTog\nJ5JDsStm+qrpYZciIiIiIiL1RFzDnpmNMrPPzGypmd1Vwf13mlledJtvZsVm1jaeNdVFwzKHkZac\npnl7IiIiIiJSY+IW9swsGXgUOAfoCVxhZj1jj3HOjXXO9XfO9Qd+DEx1zm2LV011VZNGTTi9y+ma\ntyciIiIiIjUmnj17Q4ClzrnlzrkC4EXgoiMcfwXwQhzrqdOCSEDexjy27tkadikiIiIiIlIPxDPs\ndQbWxNxeG913CDNrCowCXo1jPXVaEAkAyF2ZG24hIiIiIiJSL6SEXUDUBcCMww3hNLMxwBiAjIwM\ncnNza7G0qsnPz69WXUUlRTROasyzM54lfVN6zRUmh1XdNpPapzZLLGqvxKM2Szxqs8Si9ko8id5m\n8Qx764AuMbczo/sqcjlHGMLpnBsPjAcYPHiwy87OrqESa05ubi7VrStnQw6fbf+s2ueRqqmJNpPa\npTZLLGqvxKM2Szxqs8Si9ko8id5m8RzGORPobmYRM0vFB7o3yx9kZq2AkcAbcawlIQSRgEVbFrF+\n1/qwSxERERERkQQXt7DnnCsCvgdMBBYCLzvnFpjZjWZ2Y8yho4FJzrnd8aolUZTO25uyYkrIlYiI\niIiISKKL65w959wEYEK5fePK3X4KeCqedSSKfhn9aNO4DZNXTObKvleGXY6IiIiIiCSwuF5UXY5O\nclIy2VnZut6eiIiIiIhUm8JeHRNEAlZuX8mKL1aEXYqIiIiIiCQwhb06pnTe3uQV6t0TEREREZFj\np7BXx5zS7hQymmVoKKeIiIiIiFSLwl4dY2YEkYDJKybjnAu7HBERERERSVAKe3VQEAnYmL+RRVsW\nhV2KiIiIiIgkKIW9Okjz9kREREREpLoU9uqgSOsI3Vp107w9ERERERE5Zgp7dVDpvL0pK6ZQ4krC\nLkdERERERBKQwl4dFUQCvtj3BXM3zg27FBERERERSUAKe3VUTlYOoHl7IiIiIiJybBT26qjOLTvT\nI72H5u2JiIiIiMgxUdirw4JIwLRV0ygsLgy7FBERERERSTAKe3VYEAnIL8hn1vpZYZciIiIiIiIJ\nRmGvDsvOygY0b09ERERERI6ewl4d1q5pO/pl9NO8PREREREROWoKe3VcEAmYsXoG+4r2hV2KiIiI\niIgkEIW9Oi6IBOwv3s8Haz4IuxQREREREUkgCnt13IhuI0i2ZM3bExERERGRo6KwV8e1TGvJ4OMG\na96eiIiIiIgcFYW9BBBEAj5e9zG79u8KuxQREREREUkQCnsJIIgEFJUU8d7q98IuRUREREREEoTC\nXgI4vcsubZsNAAAgAElEQVTppCanat6eiIiIiIhUmcJeAmjaqCnDModp3p6IiIiIiFRZXMOemY0y\ns8/MbKmZ3XWYY7LNLM/MFpjZ1HjWk8iCSMAnGz5h295tYZciIiIiIiIJIG5hz8ySgUeBc4CewBVm\n1rPcMa2Bx4ALnXO9gEvjVU+iCyIBDsfUlcrDIiIiIiJSuXj27A0BljrnljvnCoAXgYvKHfMN4DXn\n3GoA59ymONaT0IZ0HkLTRk01b09ERERERKoknmGvM7Am5vba6L5YJwFtzCzXzGab2dVxrCehpSan\nMrzrcM3bExERERGRKkmpA88/CDgLaAJ8YGYfOucWxx5kZmOAMQAZGRnk5ubWdp2Vys/Pj3td3Uq6\nMXHzRF6b9BptU9vG9bkagtpoM6lZarPEovZKPGqzxKM2Syxqr8ST6G0Wz7C3DugSczszui/WWmCr\nc243sNvMpgH9gIPCnnNuPDAeYPDgwS47OzteNR+z3Nxc4l1X8/XNGf/n8ezvtJ/sPvF9roagNtpM\napbaLLGovRKP2izxqM0Si9or8SR6m8VzGOdMoLuZRcwsFbgceLPcMW8AZ5pZipk1BU4DFsaxpoQ2\noOMAWqW10rw9ERERERGpVNx69pxzRWb2PWAikAz81Tm3wMxujN4/zjm30MzeBeYBJcCTzrn58aop\n0SUnJZOdla15eyIiIiIiUqm4ztlzzk0AJpTbN67c7bHA2HjWUZ8EkYA3PnuDldtXktU6K+xyRERE\nRESkjorrRdWl5gWRAIApK6aEXImIiIiIiNRlCnsJplf7XrRv2l5DOUVERERE5IgU9hKMmRFEAiav\nmIxzLuxyRERERESkjlLYS0BBJGD9rvUs3rq48oNFRERERKRBUthLQKXz9nQJBhERERERORyFvQR0\nQpsT6NKyi+btiYiIiIjIYSnsJaDSeXu5K3MpcSVhlyMiIiIiInWQwl6CCiIBW/ZsYf4mXYNeRERE\nREQOpbCXoHKycgDN2xMRERERkYpVKeyZ2Qlmlhb9PtvMfmBmreNbmhxJl1Zd6N62u8KeiIiIiIhU\nqKo9e68CxWZ2IjAe6AI8H7eqpEqCSMDUVVMpKikKuxQREREREaljqhr2SpxzRcBo4I/OuTuBTvEr\nS6oiiATs3L+TORvmhF2KiIiIiIjUMVUNe4VmdgVwDfB2dF+j+JQkVZWdlQ1o3p6IiIiIiByqqmHv\nOmAY8Gvn3AoziwDPxq8sqYoOzTrQp0MfhT0RERERETlElcKec+5/zrkfOOdeMLM2QAvn3P1xrk2q\nICcrh/dWv8f+ov1hlyIiIiIiInVIVVfjzDWzlmbWFpgD/NnM/hDf0qQqgkjA3qK9fLTuo7BLERER\nERGROqSqwzhbOed2ApcAzzjnTgO+FL+ypKpGZo0kyZI0lFNERERERA5S1bCXYmadgK9TtkCL1AGt\nG7dmYKeBCnsiIiIiInKQqoa9XwITgWXOuZlmdjywJH5lydEIsgI+XPshuwt2h12KiIiIiIjUEVVd\noOUV51xf59xN0dvLnXNfjW9pUlVBJKCwpJAZa2aEXYqIiIiIiNQRVV2gJdPMXjezTdHtVTPLjHdx\nUjVndj2TlKQUDeUUEREREZEDqjqM82/Am8Bx0e2t6D6pA5qlNmNo5lCFPREREREROaCqYa+9c+5v\nzrmi6PYU0D6OdclRCrICZm+YzfZ928MuRURERERE6oCqhr2tZnaVmSVHt6uArfEsTI5OEAkocSVM\nWzUt7FJERERERKQOqGrY+xb+sgsbgQ3A14BrK3uQmY0ys8/MbKmZ3VXB/dlmtsPM8qLbz46idokx\nNHMojVMaayiniIiIiIgAkFKVg5xzq4ALY/eZ2a3AQ4d7jJklA48CXwbWAjPN7E3n3P/KHTrdOXf+\nUVUth0hLSePMrmcq7ImIiIiICFD1nr2K3FbJ/UOApdHLNBQALwIXVeP5pBJBVsCnmz5l0+5NYZci\nIiIiIiIhq07Ys0ru7wysibm9NrqvvNPNbJ6ZvWNmvapRT4MXRAIAclfmhluIiIiIiIiErkrDOA/D\n1cDzzwG6Oufyzexc4J9A9/IHmdkYYAxARkYGubm5NfDUNSs/Pz/0uopdMU2Tm/LcjOfosLlDqLUk\ngrrQZnJ01GaJRe2VeNRmiUdtlljUXokn0dvsiGHPzHZRcagzoEkl514HdIm5nRndd4BzbmfM9xPM\n7DEza+ec21LuuPHAeIDBgwe77OzsSp669uXm5lIX6go2BizasqhO1FLX1ZU2k6pTmyUWtVfiUZsl\nHrVZYlF7JZ5Eb7MjDuN0zrVwzrWsYGvhnKusV3Am0N3MImaWClyOvzD7AWbW0cws+v2QaD26pEM1\nBFkBS7YtYc2ONZUfLCIiIiIi9VZ15uwdkXOuCPgeMBFYCLzsnFtgZjea2Y3Rw74GzDezucAjwOXO\nuZoYHtpglc7bm7JySsiViIiIiIhImKozZ69SzrkJwIRy+8bFfP8n4E/xrKGh6ZPRh/Qm6UxeMZmr\n+10ddjkiIiIiIhKSuPXsSTiSLImcSA6TV0xGnaQiIiIiIg2Xwl49FGQFrNm5hmVfLAu7FBERERER\nCYnCXj1UOm9v8orJIVciIiIiIiJhUdirh05KP4njWhynsCciIiIi0oAp7NVDZkYQCTRvT0RERESk\nAVPYq6eCrIDNezazYPOCsEsREREREZEQKOzVU5q3JyIiIiLSsCns1VPdWnfj+DbHK+yJiIiIiDRQ\nCnv1WJAVkLsyl+KS4rBLERERERGRWqawV48FkYAd+3fwycZPwi5FRERERERqmcJePZYTyQE0b09E\nREREpCFS2KvHOjbvSM/2PRX2REREREQaIIW9ei7ICpi+ejoFxQVhlyIiIiIiIrVIYa+eCyIBewr3\n8PG6j8MuRUREREREapHCXj03Mmskhmkop4iIiIhIA6OwV8+1bdKWAZ0GKOyJiIiIiDQwCnsNQJAV\n8MHaD9hTuCfsUkREREREpJYo7DUAQSSgoLiA99e8H3YpIiIiIiJSSxT2GoAzu55JSlKKhnKKiIiI\niDQgCnsNQIu0FgzpPERhT0RERESkAVHYayCCrICZ62eyY9+OsEsREREREZFaoLDXQASRgBJXwvTV\n08MuRUREREREaoHCXgMxrMsw0pLTNJRTRERERKSBUNhrIBqnNOaMrmco7ImIiIiINBBxDXtmNsrM\nPjOzpWZ21xGOO9XMiszsa/Gsp6ELsgLmfj6XLXu2hF2KiIiIiIjEWdzCnpklA48C5wA9gSvMrOdh\njrsfmBSvWsQLIgEAuStzwy1ERERERETiLp49e0OApc655c65AuBF4KIKjvs+8CqwKY61CDD4uME0\nT22uoZwiIiIiIg1APMNeZ2BNzO210X0HmFlnYDTweBzrkKhGyY0Y0W2Ewp6IiIiISAOQEvLzPwT8\nyDlXYmaHPcjMxgBjADIyMsjNza2d6o5Cfn5+nayrvG7F3ZiwdQKvTHyF9mntwy4nVInSZlJGbZZY\n1F6JR22WeNRmiUXtlXgSvc3iGfbWAV1ibmdG98UaDLwYDXrtgHPNrMg598/Yg5xz44HxAIMHD3bZ\n2dnxqvmY5ebmUhfrKq/VhlY8Pv5x9nfaT3bf7LDLCVWitJmUUZslFrVX4lGbJR61WWJReyWeRG+z\neA7jnAl0N7OImaUClwNvxh7gnIs457Kcc1nAP4Dvlg96UrP6dexHm8ZtNJRTRERERKSei1vPnnOu\nyMy+B0wEkoG/OucWmNmN0fvHxeu55fCSLImcSA7/XfFfnHMcafisiIiIiIgkrrjO2XPOTQAmlNtX\nYchzzl0bz1qkTJAV8NrC11ixfQXHtzk+7HJERERERCQO4npRdambSq+3p6GcIiIiIiL1l8JeA3Ry\nu5Pp2LwjU1ZOCbsUERERERGJE4W9BsjMCCIBk1dMxjkXdjkiIiIiIhIHCnsNVJAVsDF/I4u2LAq7\nFBERERERiQOFvQZK8/ZEREREROo3hb0GKtImQrdW3Zi8UmFPRERERKQ+UthrwIJIwJQVUyhxJWGX\nIiIiIiIiNUxhrwELIgFf7PuCuRvnhl2KiIiIiIjUMIW9BiwnKwfQvD0RERERkfpIYa8B69yyMz3S\ne2jenoiIiIhIPaSw18AFkYBpq6ZRWFwYdikiIiIiIlKDFPYauCASkF+Qz6z1s8IuRURERKR+2rED\nHniAfrfdBj/6EUydCoX6Q7vEn8JeTVi+nBaLFkFJ4q1qmZ2VDWjenoiIiEiNW70abr8dunSBO+8k\ndds2+MMfIDsb2rWDSy+Fv/0NNm4Mu1KppxT2asLjjzPoppv8L/JNN8G778L+/WFXVSXtmrajX0Y/\nzdsTERERqSmzZ8M3vgHHHw8PPwznnw+zZjHzqadg61Z47TX4+tfh/ffhW9+CTp1g8GD42c/gww+h\nuDjsVyD1hMJeTbjrLhb++McwdCg8+yyccw60bw+XXQbPPw/bt4dd4REFkYAZq2ewr2hf2KWIiIiI\nJKaSEvjXvyAnxwe3t9+GW26B5cv958FBg/xxLVvC6NHw5z/D2rXwySfw619D48b+67BhkJEBV10F\nL7zgw6HIMVLYqwnp6Xx+9tnw6quwZYv/5b78cj8e+8orffD70pfgT3/y3fl1TBAJ2F+8nw/WfBB2\nKSIiIiKJZd8+ePJJ6NXL9+AtWQJjx8KaNfD730PXrod/rBn07w933w3vvQebN/uAd+65MHGi7x3s\n0AHOOMMHwU8+Aedq77VJwlPYq2mNG8N558H48bB+ve+ev/12WLcOvv996NYNBg6EX/wC5s6tE7+w\nI7qNINmSNW9PREREpKq2bIF77/Wf7W64wX8GfO45WLEC7rgDWrU6+nO2bes7DJ55xs/j+/BD+H//\nDwoK/NeBA6FzZ/j2t30nw86dNf+6pF5R2IunpCTfFX/ffbBwISxaBPffD02a+LDXvz9EIr6Lf/Lk\n0FZlapnWksHHDda8PREREZHKLFkC3/2u77H72c/88Mz//AfmzPEjuho1qpnnSU6G007znxlnzvTh\n76mnYPhwH/S+9jVIT4cggAcegP/9r050IkjdorBXm3r0gP/7P5gxAzZs8GO1+/SBJ56As87y47O/\n+U34xz9g165aLS2IBHy87mN27a/d5xURERGp85zzn99Gj/af5/7yF7jiCpg/HyZM8J/jzOJbQ0YG\nXHMNvPSSH+45daofPbZlC9x5px9GGon4IPqvf8GePfGtRxKCwl5YMjLg+uvhrbfKVmW68EL/D8al\nl/rleEuHg27YEPdygkhAUUkR761+L+7PJSIiIpIQiov9H+GHDYMzz/QB6+67YdUqH/h69QqnrkaN\nYMQIP3ps3jy/JsQTT/hRY8884+cOtm3rFw384x9h2bJw6pTQKezVBc2a+b8UPfUUfP455ObCzTf7\nYZ/f+Q4cd5xf6fO3v/XDQePQRX96l9NJTU7VvD0RERGR/Hwfkrp393+E37zZL7S3Zg386lfQsWPY\nFR6sSxcYMwb++U/fiTBpkr8c2PLl8IMfwIkn+h7J227zQ04T5BJhUn0Ke3VNSgqMHOkvuLl0KXz6\nqZ/8W1zs/5LUs6f/Zb3zTr9qUw1dh6Vpo6YMyxymeXsiIiLScG3Y4D9vde3qQ1LHjn5+3OLF/g/x\nzZqFXWHl0tLgy1+GBx+Ezz7zcwwfecRf8++xx/x96elw8cV+BNmaNWFXLHGksFeXmUHv3n71pZkz\n/S/jY4/58dgPP+wn6Hbq5FdkevNN2Lu3Wk8XRAI+2fAJ2/Zuq6EXICIiIpIA5s+H667zK2ved5+/\nVt6MGX5V9Usu8YulJKoTT/Qrwr/zju/1e+stuPpqfxmH73zHB9u+feGuu2D6dCgqCrtiqUEKe4kk\nM9N3yU+c6IcTvPiiv37fP/4BF13k/0pTOhx0y5ajPn0QCXA4pq6cWvO1i4iIiNQlzvkhjaNG+QXz\nXnrJD4VcvNj35p1+etgV1rxmzfx8vsceg5UrYcECf03Adu38NQFHjPDfX3YZPP20n14kCS2uYc/M\nRpnZZ2a21MzuquD+i8xsnpnlmdksMzsznvXUK61a+V/E55/3wW/SJPjWt2DWLP+XqYwM/wv7hz9U\neVLukM5DaNqoqebtiYiI1GXO+Wv5/vvfNFu2TD0xR6ugAJ59FgYM8EMa8/L8PLw1a/y8vBNPDLvC\n2mHmpwfdcYe/BNiWLb4D4atfhWnT4Npr/TDWU0+Fn/8cPv4YSkrCrlqOUkq8TmxmycCjwJeBtcBM\nM3vTOfe/mMP+C7zpnHNm1hd4GTg5XjXVW6mp/h+rL3/ZTyaeMwfeeMNvt9/ut169fO/fRRfB4MH+\nGoDlT5OcyvCuwzVvT0REpK4oLPTzrubO9aEkL89/v3kzAKcCfO97PrgMHly29eiR2EMP42H7dj9H\n7ZFHYN06OOUUePJJf228xo3Dri58rVr5oPfVr/pQN3euXyV+wgQfhn/5S2jf3veEnnsunH22X/Gz\nviouhqIikvbtC7uSaolb2AOGAEudc8sBzOxF4CLgQNhzzuXHHN8M0JUgq8vMX9xz0CD/S7liRVnw\nu/9++M1v/OqeF17og19Ojp/IGxVEAn70nx+xMX8jHZvXsZWmRERE6rMdO/wH7Nhgt2BB2cqJaWl+\nLv8FF/gl9nv35n+TJ9Nzzx4/sudvf/N/9AU/XG/gwIMD4IknVvjH3npv1Sq/1sGf/+xX2QwCH/pG\njWqY70dVJCX5PyAMGAA/+Ymf6zdxYln4e/ZZf8zpp/vgd8450Lmz72UuLDz0a1X31aXjo6vfn5ae\nfkzTo+qKeIa9zkDs8j5rgdPKH2Rmo4HfAh2A8+JYT8MUicCtt/pt61b/C/rGG/6XdNw4aNHC/2N3\n8cVw7rkEkQCAKSumcEWfK0IuXkREpB5yzgeQ0l660mC3cmXZMe3b+0D3gx9Av37++x49/KrdMTaZ\n0TM7298oLvbzzWbNKtvGjStbwK1lS//H4NgAGInE/2LgYZk1y89De+UVf/uyy/xop4EDw60rEaWn\nwze+4bfiYr9w4IQJ/uLtd9/tt3gz89cXTEk5+GtF+2Lva9QImjY9+sdFv1+1cSMnxf/VxY25OFyz\nDcDMvgaMcs5dH739TeA059z3DnP8COBnzrkvVXDfGGAMQEZGxqAXX3wxLjVXR35+Ps2bNw+7jCpL\nKiig9ezZtJsxg3bvv0/qF19QkpzM9n59+U2nBWw940yuGfbTsMuMq0RrM1GbJRq1V+JRm9W8pIIC\nmq5cSfOlS2m+bNmBrym7dwPgzNibmUn+iSeSf8IJ/uuJJ1LQtm2VQlhlbWbFxTRdtYoWn312YGu+\nbBlJhYUAFLZowa4ePdh10kn+a48e7O/QIXEDYEkJ6R9+SJeXX6b13LkUNW3KhvPPZ+0ll7A/IyPs\n6url71jq1q20mT2blN27KUlOxiUn41JS/Fb6fXKyv6/cvqO5L6xe2LraZjk5ObOdc4MrOy6eYW8Y\ncI9z7ivR2z8GcM799giPWQ4Mcc4dtq908ODBbtasWTVdbrXl5uaSXfqXtURTUgIffVQ23HPRIr9/\nwAA/1HPUKP+XwJR4dgTXvoRuswZKbZZY1F6JR21WTZs3l/XUlX5duLDsmrjNmvkl7vv3L+ut6927\nWtduO6Y2Kyjww0NnzizrAfz007KFXtq3P7j3b/BgPwWkLtu3z49a+v3v/RzHLl38qKbrr/c9mnWE\nfscST11tMzOrUtiL56f3mUB3M4sA64DLgW/EHmBmJwLLogu0DATSgK1xrEkqkpQEw4b57b77eObl\nnzD/z7/hlzuTaPyLX8A99/hJu0HgF4H50pf8uP9E/aufiIhIdRQXw9Klhwa79evLjsnM9IHuoovK\nwt0JJ9SNOWKpqWXzscaM8fv27YN58w4eAjpxYtnqi506HRoAO3QI7zWU2rLFX0bgT3/yYXvgQPj7\n3+HSS/1QPJEGLm5hzzlXZGbfAyYCycBfnXMLzOzG6P3jgK8CV5tZIbAXuMzFq6tRqmxg9hVcs/A3\nnHLhzVyXeb5fjvff//bb66/7g7p186Hvy1/2IbB9+3CLFhERiYf8fN/rFRvs5s2DPXv8/Skpfvn6\ns87yoa402KWnh1v30WrcGIYM8VupPXv8a44NgG+/fWDhCrp0OTj8DRpUe6978WJ48EF/beF9++C8\n8/x8vOxs/TFaJEZcx+U55yYAE8rtGxfz/f3A/fGsQY5er/a9aN+0PZNXTua6Adf5Cc2XXeb/cV+6\n1F+A9N//9tdi+ctf/IMGDCgLf2eeCU2ahPsiRETiwTnYudMPu6tnQ9sbvNJr18X21M2dC0uWlIWb\n1q19mLvhhrJgd8opB61qXa80bepXW4y9uPiuXfDJJwcHwNI/BINf8KV8AGzVqmbqcQ5mzIAHHoA3\n3/Q9d1dfDT/8oQ/cInII/U8lhzAzgkjA5BWTcc5hpX8hM4Pu3f12001+bP/s2WXh76GHYOxY/5/e\nmWeWDfkcMKBuDFsREalMYaG//taqVbB6ddkWe3v3bv9vWseOvmcjM7Pir506KRDWVYWFfn56+dUw\nt8bMJDn+eB/mrryyLNh16aJeoxYtYMQIv5Xavt1f4zc2AJaugAlw0kkHB8ABA+BoFrwoKvKB8oEH\n/IW927b1lwO4+Wb/eygih6X/haRCQSTgpQUvsXjrYnq061HxQSkpcNppfvvJT/wHoGnTysLfXXf5\n49LTD57vF4nU3gsRESnlnL+O2ZGC3Pr1Zb04pdq390PXTzkFvvIVfy2pXbtgzRpYu9YvdPHuu/7f\nwFhJST7wHS4MZmYqEMZTSYkPIVu2+AA/b15ZsFuwwC9SAn74Yu/eMHp02aIpffvWqUU96rzWrf3/\n80FQtm/rVv8H4dJFYKZNg+ef9/eZ+d+n2ADYr5/vSYyVnw9//asfrrlypV8v4NFH4ZprqrWojUhD\nov9hpEKl19ubvGLy4cNeec2a+YtqnnOOv71xI/z3v2Xhr/SvfCecUDbkMyfH/4VORKS6ior8h/rD\nBbnVq31Ii5Wa6oNXt27+36SuXf3WrZv/2qVL1YallwbJtWvLQmDs1/nz4Z13FAiPVUkJfPGFDxBb\ntvitsu+3bStbXKRUhw4+zN16a9ncupNO0vsbD+npcPbZfiu1caMPgLELwDzzjL8vORl69SoLf6tX\n+2sEbt/uh5H+4Q9w4YX+OBGpMv3rJhU6oc0JdGnZhckrJ3PTqTcd20k6dvTDX6680n8Q+uyzsoVe\nnn8ennjC/3Vv8OCy8Hf66fV37oOIVM+OHQcHt/Jhbt26Qz/cp6f70Na9u19Ao3yY69ChZoaZm/ne\njdatfS9RRUoDYWkIPNpAeKQhox07Jk5gKS4u63GrSmjburXi4FYqNRXatfNbejr06VP2fen+Dh38\nfg35C1fHjn4hlfPO87dL50nGDv98803fm5eU5Htbb7/drxYuIsckQf5nkNpWOm/v7cVvU+JKSLJq\nfhgyg5NP9tv3v+/nS3z8cVmv3+9+B7/9rf8L+ogRZUM++/TRfD9JXIWFsGGDDyGH29avh/37fc94\nPLYmTRJjjlFRkX+vjtQrt2PHwY9JSSnrlcvJKQtypWGuS5e6NdQrNhD26VPxMeUDYfmvn34KEyaU\nrQRZKinJXwftSD2E8QiExcXH1uN2uIW3Y4Nbu3Z+OGVskKvo+2bNEuNnXA5l5odFd+7sL1EB/mdj\n1Sr/s5qZGW59IvWAwp4cVhAJeHru0zz4wYMM7zacnu170jz1KCZUH0mjRnDGGX77+c/96nZTp5aF\nvzvu8Md16OD/Gl8a/rp0qZnnF6mO0g/kRwpx69bBpk2HfqhNTfUfyjt39teDuuACH8h27z5027XL\nD3sqv/9orlBj5ufBHCkQVnb/kbaqDqnatevIQW7t2rILT5dq08YHt0gERo48tFcuI6P+DemqaiDc\nvr3i3sG1a/3ctH/969BAmJxc6ZDRRjt2+IVLjqbH7XA/j2lpB4ezfv2OHNrS0xXcxLd/VlbYVYjU\nGwp7clhnn3A2bZu05Y5/33FgX6R1hN4deh+09UjvQVpKNYdetmzpP/RecIG/vW6dD36l2wsv+P09\nepQN+czOrrnlnEVKFRb6gFVZkCv/QRr8h9XOnX2YGzCg7C/WsVu7dtX7MOucv6ZUReHwaLdt2w7d\nV1R0dPWkpR02CPb54gvYu9cHu+3bD35ccrIPGd26wfDhFffKtWhx7O9TfWbmg3CbNlULhBX1Eh4m\nEJ5xuOdMS/ML1ZSGswEDyr4/XHhr2lTBTUQkZAp7clgdm3dk0x2bWP7FcuZvms/8TfNZsHkB8zfN\n552l71BU4j8UJlsy3dO7+/DXviwEntD2BFKSjvFHrHNnv9rWNdf4Dy0LFvgev//8x19A9dFH/YfF\nIUPKwt/Qob7HUKQipddHqyzEff75kXvjBgyA888/NMQdd5xf1S/ezHxPYJMm/gN1TSsoqJkguWkT\nqTt2+KHbZ5xxaK9cp071r1euLjmaQBgTApcsWED3YcMODXAKbiIiCUlhT44oOckHue7p3Rl9yugD\n+wuKC1i8dfFBITBvYx6v/u9VHP6DclpyGie3O/mQnsCurboe3RxAM7/gQe/e/sKpBQXw4Ydl4e/X\nv4Z77/XX7Bk5smzIZ8+e+nDSUBQVVa03rvzCF+BXgy0NbP37x6c3LpGkpvqtTZtqn2p2bi7Z2dnV\nr0niIzYQ9u0LwLrcXLqrzURE6g2FPTkmqcmpB8JbrD2Fe1i4eeFBIXDaqmn8/dO/HzimeWpzerXv\nRe8OvQ987d2hNx2bdyy7gPsRnzy17IKu997r/zKdm1sW/v71L39cp05lvX5nneV7XiQxFBX5OV47\nd/qv0e87TpkCM2YcfW9c//5+9beweuNEREREQqCwJzWqaaOmDDpuEIOOG3TQ/h37dhwYAlq6vbX4\nLf7yyV8OHNO2SduDhoL26uCDYNsmlVyHr3VruPhiv4Ff7KF0oZd33oFnn/X7e/UqC38jRtTkyxbn\nyrEb6lAAABOQSURBVBYUqcoWG+Iq2vbtq/BpTi79Rr1xIiIiIpVS2JNa0apxK07vcjqndzn9oP2b\ndm9iwaaYELh5Ps99+hw79+88cEyn5p0OGQp6xJVBu3aFb33LbyUlfiGC0vD3xBPw8MOQksJp7dv7\nBV5SU/3iA2F+DSOUFBRUP5SVbvn5h78GViwzP9y2RYuDt27d/CI95feX2z5cuZKhl1xStYtci4iI\niDRwCnsSqg7NOtAh0oGcSM6Bfc451u1ad1Av4PxN8xk3axx7i/YeOC6rddYhi8L0aNeDxikxw/KS\nknzPT//+/nIO+/bB++/Df/7DjlmzaNKmjb/GWUFB2ddduw6+/f/bu/cYuc7yjuO/Z647uzO7viSx\nY6+JIYkAk3Jx3YQAap1QKlMQrtSKSylKWxAClUsrVAitxF9VhdqqBUpalKZcJAJRRQNF5RJCiNtI\n3AKEOIlDiEkMu767Xu99Z+fy9I85Ozuzc9/d8dmZ/X6k0TnnPe+cfcavvd7fvudSb9nJre/bEY/X\nhsDVBsdIpL0Qt7jYXm3JZG3w2r69dGvsTKatkFbuMzi4pucmLuRyBD0AAIA2Efaw4ZiZRodHNTo8\nqkPXHSq3F4oFnbh0omoW8PFzj+u+4/cpV8xJWr4zaOW1gDdcdYOu23Zd6c6gAwPSrbdKt96qnx05\nop2ruRGBe+l5YM3CYLeWU1PN92ezpRm2eoHriitqw1ergJZOlwIkAAAAeg5hDz0jGonq2m3X6tpt\n1+rwCw6X23OFnJ6++HTVLODRs0d175P3lu8Mmogm9MIrXlh1U5iJuQnNLM50/qB4MykWK72Ghtbz\nI64Pd65VAwAAAGEPvS8ejWvflfu078p9euOL3lhun8/N68kLTy5fE3j+cT30q4eq7gx628O3acvA\nlvJM4mhmVHtG9pS39wyX1jPJHnq4M0EPAAAAIuyhj6XiKe2/er/2X72/qn1yYVLHzh/T1777NWV2\nZzQ+Na7x6XGNT43rkdOP6Ozs2ZpjDSeHq8LfyjA4OjyqkYGRy/XRAAAAgJYIe9h0RgZGdPOem5Xd\nkdXBVx2s2b9YWNSp6VManxrX2ORYKQxOjWtsqrR+9OxRnZk5Uz5FdEkmkWkYBEeHSzOGI8mR9p4l\nCAAAAKwRYQ9YIRFNaO+Wvdq7ZW/DPrlCrhwIV4bB8alx3feL+3R6+nRNIByKD1WFv9FMdRgcHR7V\n1oGtBEIAAACsGWEPWIV4NK5rtlyja7Zc07BPrpDTmZkzNUFwafvbz3xbp6ZPqejVz6cbjA9WzQiu\nvI5wdHhU21PbCYQAAABoirAHdEk8GteekT3aM7JHN+vmun3yxXw5EJaD4ORY+RrCIyeO6OTUSRW8\nUPW+gdhAw1NGr05frUwyo0wio3QirXQirWgkejk+MgAAADYQwh4QolgkVg5pjRSKBZ2dPVtzDeH4\ndGn7oV8+pJPTJ5Uv5hseIxVLKZ1IK5MsBcDKIJhJZpSOL++ru39F22B8kJlFAACADY6wB2xw0UhU\nuzK7tCuzSzfuvrFun6IXdW72nMYmx3Rm5oxmFmc0vTitmcWZ0np2uqZtYmFCY1NjVfuaBcZKJqsK\ngjUBMVivt79RWyLKw9sBAADWE2EP6AMRi2hneqd2pneu6TjZfHY5IAbBcCkMthMgT8+crtm/8iY1\njcQj8eYBMZ7W1Lkp/SD2A20Z2KKtqa2l5UBpufSKR+Nr+jMAAADoF10Ne2Z2SNLHJUUl3eXuH12x\n/62SPiTJJE1Lere7P9rNmgA0lowllYwltX1w+7ocz901l5trOyxOZ6c1k6vef37ufHl7Yn5CXxj7\nQtOvORQfqhsEl9aX9tVryyQynJ4KAAD6RtfCnplFJd0h6TWSxiU9bGZfdfdjFd2elfRb7j5hZq+V\ndKekm7pVE4DLy8w0lBjSUGJIO7Rjzcd78MEHddOrbtKlhUuamJ/QpYVLpfWFiaq2pe1LC5c0NjWm\nx849pon5CU1mJ5seP2KR+kEwuaXhbGJlWzKWXPNnBAAAWC/dnNm7UdJxd39GkszsHkmHJZXDnrt/\nt6L/9yU1vksFgE3PzDQYH9RgfFC7Mrs6fn+hWND04nTdULiybWl5+vzp8r75/HzT4w/EBhoGwWYz\njCPJEQ0lhpSMJplZBAAA66abYW+3pLGK7XE1n7V7u6RvdLEeAJtcNBItB6zVyOazDWcT6wXFszNn\n9dSFp8r7Vz5Co6Y+i5ZmQuNDSifSGkoEy8rteIP2JtupeEoRi6zqMwMAgN5l7u3dPKHjA5v9gaRD\n7v6OYPttkm5y9/fU6XuLpH+R9Cp3/786+98p6Z2StGPHjl+/5557ulLzWszMzCidToddBjrAmPWe\nXh4zd9d8YV7T+WnN5Gc0k58pr88V5rRQWNB8Yb68nC9WrFe0LxRLy8XiYkdffyAyoFQ0pVQ0pYFo\naX2pbSA6ULdtqX+99qVl1Bo/w7GXx2uzYsx6D2PWWxiv3rNRx+yWW275sbsfaNWvmzN7JyXtqdge\nDdqqmNmLJd0l6bX1gp4kufudKl3PpwMHDvjBgwfXvdi1OnLkiDZiXWiMMes9jNmyQrGg2dysZhdn\nNbM4o9lcsGy1vaJ9KjelU4unNDu7vL8TyWiy4ezi7MSsnjf6PA3Fh2pmLJfalt6zcn8qluKU1hDw\nb6z3MGa9hfHqPb0+Zt0Mew9Lut7MnqtSyHuzpD+s7GBmz5F0r6S3ufvPu1gLAPSVaCSq4eSwhpPD\n63rcohc1n5tvPzw2CJMnp07q/Mx5PXX8qXKfVqexVjKVrs9sKxx2GCYT0QRBEgCwKXQt7Ll73sze\nI+k+lR698Gl3f8LM3hXs/5Skj0jaLulfgv948+1MRwIAuiNikfIdVK8aumpNx6r8bai7a7GwWDMb\nObs4WxUcG+2v7HNu9lxNn3af5yjVvzaykwA5FB9SPBqXyRSxiMyC5Yrtem1L283e106f1bwPALD5\ndPU5e+7+dUlfX9H2qYr1d0h6RzdrAACEz8zKz3Hcltq2rsd2dy3kFzoOkCu3JxcmdXLqZFXbXG5u\nXWsNk8lahkUrmoYfGdZgfFCpWKp899tUvGI91mC9UZ8V7fFoPOw/CgDYNLoa9gAA6DYzUyqeUiqe\n0pW6cl2PXfSi5nJzNQGy4AUVvSh3Ly3lVdv12jrp0+x97fRZ7dd/duxZbbtqm+bz85rLzWkuN6f5\n3Lwuzl8sra9o72RGdUksEmsrKK4mSFa2D8QGuAstgE2PsAcAQAMRiyidSCud2Hh3YuuGTm5E4O7K\nFrLl4FcOgUEgrNfWtD1fCpX19i8WOrv77JJULFUOhEvXuJZfieHatopXJplZXk9kmJEE0JMIewAA\noGNmpoHYgAZiA1Kqu1+rUCy0DIzNwuVsblbTi9Oayk5pKjul8anx8vp0drqtGcpULNU0HFYFxUSm\n4b6B2ADXUAK4bAh7AABgQ4tGol2bYS16UbOLs+Xw1+hVGRaXXicunSivT2YnlS/mW369WCS2qpC4\n8pVOpDlNFUBLhD0AALBpRSyiTDKjTDKj3dq96uMsndbaMjRmg9C4uNx2bvacjl88Xt5u98ZA6URa\nkWJEqR+lFIvE1uUVtei6Havm2JH2j52IJsqn4RJqgdUj7AEAAKxR5Wmta31sSb6YXw6FTWYZJxcm\n9ezYs9qxc4cKXlC+mG/5WsgvtNWv8lV57DAko8nytZflG/jEU1XrS/vqtrX73ngpNAP9hL/RAAAA\nG0gsEtPW1FZtTW1t2beTm+qs1dJdWzsNi+2EyJWvbD5bdT1meX1F2/m58+XrNefz8+X1XDG3qs8Y\nj8TbD4pthMeV/S5kL+jszNnyI1AqX5WPRqlqDx6bAqwGYQ8AAAAtmZmiFlU0ElVSybDLaSpfzJcD\nYWUYrAmPdYJiZbCsbJtYmNCp6VM1780Wsp0V9/3VfaZ6IbCT0Lia93TSPx6JKxFNKBlLKhEpLZPR\n5HJbNKFkNFm1vrSvXr9m+2KRGAG4TYQ9AAAA9JVYJFa+FrPbil7UQn6hdXjMzevRY4/quuuvq3re\nZeWr8jmaVe0d9q95j1Z/rJX988V8zXsKxdIsbbaQVTaf1WJhUdlCsMxnOw/ELZisrRDZcF8H/X9x\n8Rc6qIPrWv/lRNgDAAAAViliEQ3GBzUYH2zZ98ilIzr4Gwe7X9QG4+7lMFgZAJfW64XDlfta9ivW\n9s/mSzdNavU1m9mW2KYP6oOX6U9q/RH2AAAAAHSNmSkejSsejYddSg13V66Yaxgmf/jwD8MucU0I\newAAAAA2JbPSKaGJaKLu/ovpi5e5ovXFg0sAAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R\n9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA+Zu4ddQ0fM7Lyk\nX4ZdRx1XSLoQdhHoCGPWexiz3sJ49R7GrPcwZr2F8eo9G3XMrnH3K1t16rmwt1GZ2Y/c/UDYdaB9\njFnvYcx6C+PVexiz3sOY9RbGq/f0+phxGicAAAAA9CHCHgAAAAD0IcLe+rkz7ALQMcas9zBmvYXx\n6j2MWe9hzHoL49V7enrMuGYPAAAAAPoQM3sAAAAA0IcIe+vAzA6Z2VNmdtzMbg+7HjRnZnvM7EEz\nO2ZmT5jZ+8OuCa2ZWdTMHjGz/w67FrRmZlvM7Etm9jMze9LMbg67JjRmZn8RfD983My+aGYDYdeE\nWmb2aTM7Z2aPV7RtM7P7zezpYLk1zBqxrMF4/X3wffGomX3ZzLaEWSOq1Ruzin0fMDM3syvCqG21\nCHtrZGZRSXdIeq2kfZLeYmb7wq0KLeQlfcDd90l6uaQ/Y8x6wvslPRl2EWjbxyV9091fIOklYuw2\nLDPbLel9kg64+w2SopLeHG5VaOCzkg6taLtd0gPufr2kB4JtbAyfVe143S/pBnd/saSfS/rw5S4K\nTX1WtWMmM9sj6Xck/epyF7RWhL21u1HScXd/xt0XJd0j6XDINaEJdz/t7j8J1qdV+iF0d7hVoRkz\nG5X0Okl3hV0LWjOzEUm/KenfJcndF939UrhVoYWYpJSZxSQNSjoVcj2ow93/V9LFFc2HJX0uWP+c\npN+7rEWhoXrj5e7fcvd8sPl9SaOXvTA01ODfmCT9k6QPSuq5m50Q9tZut6Sxiu1xERx6hpntlfQy\nST8ItxK08DGVvskWwy4EbXmupPOSPhOcenuXmQ2FXRTqc/eTkv5Bpd9Yn5Y06e7fCrcqdGCHu58O\n1s9I2hFmMejIn0r6RthFoDkzOyzppLs/GnYtq0HYw6ZlZmlJ/ynpz919Kux6UJ+ZvV7SOXf/cdi1\noG0xSfsl/au7v0zSrDi1bMMKrvE6rFJI3yVpyMz+KNyqsBpeusV6z808bEZm9tcqXVZyd9i1oDEz\nG5T0V5I+EnYtq0XYW7uTkvZUbI8GbdjAzCyuUtC7293vDbseNPVKSW8wsxMqnSZ9q5l9PtyS0MK4\npHF3X5ox/5JK4Q8b029Letbdz7t7TtK9kl4Rck1o31kzu1qSguW5kOtBC2b2x5JeL+mtzjPQNrpr\nVfpF2KPBzyGjkn5iZjtDraoDhL21e1jS9Wb2XDNLqHRR+1dDrglNmJmpdC3Rk+7+j2HXg+bc/cPu\nPurue1X69/Udd2fWYQNz9zOSxszs+UHTqyUdC7EkNPcrSS83s8Hg++OrxQ11eslXJd0WrN8m6b9C\nrAUtmNkhlS5LeIO7z4VdD5pz98fc/Sp33xv8HDIuaX/w/1xPIOytUXCR7Xsk3afSf47/4e5PhFsV\nWnilpLepNEP00+D1u2EXBfSZ90q628yOSnqppL8NuR40EMzAfknSTyQ9ptLPBneGWhTqMrMvSvqe\npOeb2biZvV3SRyW9xsyeVmmW9qNh1ohlDcbrk5Iyku4Pfv74VKhFokqDMetpxuwxAAAAAPQfZvYA\nAAAAoA8R9gAAAACgDxH2AAAAAKAPEfYAAAAAoA8R9gAAAACgDxH2AACbipkVKh678lMzu30dj73X\nzB5fr+MBALAWsbALAADgMpt395eGXQQAAN3GzB4AAJLM7ISZ/Z2ZPWZmPzSz64L2vWb2HTM7amYP\nmNlzgvYdZvZlM3s0eL0iOFTUzP7NzJ4ws2+ZWSro/z4zOxYc556QPiYAYBMh7AEANpvUitM431Sx\nb9Ldf03SJyV9LGj7Z0mfc/cXS7pb0ieC9k9I+h93f4mk/ZKeCNqvl3SHu79I0iVJvx+03y7pZcFx\n3tWtDwcAwBJz97BrAADgsjGzGXdP12k/IelWd3/GzOKSzrj7djO7IOlqd88F7afd/QozOy9p1N2z\nFcfYK+l+d78+2P6QpLi7/42ZfVPSjKSvSPqKu890+aMCADY5ZvYAAFjmDdY7ka1YL2j5+vjXSbpD\npVnAh82M6+YBAF1F2AMAYNmbKpbfC9a/K+nNwfpbJT0UrD8g6d2SZGZRMxtpdFAzi0ja4+4PSvqQ\npBFJNbOLAACsJ36rCADYbFJm9tOK7W+6+9LjF7aa2VGVZufeErS9V9JnzOwvJZ2X9CdB+/sl3Wlm\nb1dpBu/dkk43+JpRSZ8PAqFJ+oS7X1q3TwQAQB1cswcAgMrX7B1w9wth1wIAwHrgNE4AAAAA6EPM\n7AEAAABAH2JmDwAAAAD6EGEPAAAAAPoQYQ8AAAAA+hBhDwAAAAD6EGEPAAAAAPoQYQ8AAAAA+tD/\nA+eq6Y9ASgToAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x17b267b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(15, 5))\n",
    "plt.plot(fitted_model.history['loss'], 'g', label=\"train losses\")\n",
    "plt.plot(fitted_model.history['val_loss'], 'r', label=\"val losses\")\n",
    "plt.grid(True)\n",
    "plt.title('Training loss vs. Validation loss - VGG16')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x175aa3c8>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4IAAAFNCAYAAABVKNEpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8lNW9x/HPyb6SkIVAEkJYw05CQoKyCFoQFbCyuKAI\nKHoRsGirVrm16tW2aC0uiFJqC65VL8rVWmsrYAQUwYCRNSxCgBCEsGUl65z7x5lMZrIDCZNJfu/X\n63kx88yznGfOAPOdc55zlNYaIYQQQgghhBBth5uzCyCEEEIIIYQQ4vKSICiEEEIIIYQQbYwEQSGE\nEEIIIYRoYyQICiGEEEIIIUQbI0FQCCGEEEIIIdoYCYJCCCGEEEII0cZIEBRCiIuklHJXShUopWKa\nclvhWpRSG5VSM62PZyil/tWYbS/iPN2UUgUXV0ohhBDCkQRBIUSbYQ1ilYtFKXXe7vntF3o8rXWF\n1jpAa32kKbe9WEqp2UoprZSa3FznaG2UUncopX6sZb2XUuqUUmrchRxPa/2G1vq6JipbllJqlN2x\nD2qtA5ri2HWcz00pdVgptb25ztHSKaXWKKV+W8v6yUqpY0opd+vzZKXUZ0qpc9Zll1LqaaVUsN0+\nkUqpvyqlsq3/xhxUSv1NKRVnt81flVL7rP8e3VHLeXtYz5Nv/Tz+obmuXQjR9kgQFEK0GdYgFmD9\nMn0EmGC37p3q2yulPC5/KS/JDOAMcOflPnHlF2QX9BEQrpQaXm399UAp8MXlL5LTjAZCgN5KqYTL\neeIW9HftDWB6LeunA29rrSuUUiOAdUAq0EtrHYz5vAD0B1BKhQPfAl7AcCAQSAS+Bn5md9zvgTnA\nD9VPqJTyxnz+/g1EAJ2Bdy/t8oQQoooEQSGEsFJKPaOUel8p9XelVD5wh1LqCqXUt9Zf/Y8rpV5W\nSnlat/ewtsDFWp+/bX39X9Zf8Dcppbpe6LbW16+zthTkKqWWKKW+rq9LoVKqOzAMuBe4zvpF1P71\nSUqpdKVUnlLqgFJqrHV9qFJqpfXaziqlPrSun62USrXbv7byL1VKfa6UKgRGKKUm2p3jiFLq8Wpl\nGGl9L3OVUkeVUtOt72+2UsrNbrublVJba7nGYdZWGfttpyqltlkfD1VKbbOe/4RS6o91vV+VtNZF\nwCpqhuc7gXesX/xDra0yOdb36B9KqajajlfL+zZOKbXXes0vAcrutZ5KqS+VUmesrT1vKaWCrK/9\nHYgE/mVtTfqltXVI2+0frZT61Lr/fqXUXXavPWP9HL9t/XztVEoNbuDtmIEJxp9bH9tfV62fE+tr\ndX22HFo0rWVaaX3cw/p5mqWUOgL8R5kWyVVKqZ+U+fuWqpTqY7e/n1LqBetnK1cptV4p5a2U+rdS\n6r5q5d2tlJrQwPXW5iOgo1LqSvtrxwS9N62r/gj8RWv9nNb6JIDW+rDW+nGt9UbrNr8CcoA7rS25\nWmt9Vmv9V6310spja61f0VqvA0pqKcvdQKbW+iWtdZHW+rzWesdFXJMQQtRKgqAQQji6CfOrexDw\nPlAOLADCMEFrHPBf9ew/DXgc07JyBHj6QrdVSnUAPgAetp73EJDcQLnvBL7VWn8I/Gg9NtbjXQn8\nDfPlNBjT8nPY+vK7mFaLvkAH4KUGzlO9/E9hWjs2AQXA7dZzTAAWKKXGW8vQFfgMWAyEAgnADq31\nJiAfuMbuuNOp+tJt7xugDLiqWhkqW0mWAH/UWrcDemACXmO8AUxVSvlYyxoC3GBdD+b/yr8AMUAX\naxkafJ+s9bgKeBRTj1lAiv0mwDNAR8z73w3zeUBrfRuQDVxnbbFeXMsp3sd8NiKBW4DnlFL2783P\ngbcw9fEv4OV6yhoATALesS7TlGMrXa2fkwY+W40xEuiNeb8BPgV6Yt6TndbyV3oBGIh5D0OAhYAF\nU0+2bpVKqUTM+13nvZp10VoXUvOHgVuB7VrrXUqpdpi/ix/Wtr+dnwGrtda6ge3qMxQ4Yg26p5RS\n65RS/S7heEII4UCCoBBCONqotf6H1tpi/QX+O631Zq11udb6ILAcxyBS3SqtdZrWugzzhTr+IrYd\nD6RrrT+2vvYCcKqugyilFOaLa2UgehfHL7J3Y1ow1lqv66jWeq9SqjMmgN1nba0o01qvr6e81a3W\nWm+yHrNEa71Oa73L+vwH4D2q3qs7gH9prT+wvpentNbp1tfetL6OUirMWqa/Vz+Z9Uv1e8Bt1m2D\ngWut68AEtJ5KqVCtdb7WenMjr2M9cA6YaH1+C7BTa73Tet4crfVq6+chD/g99X8GKlXW42prPf4J\n00pUeT37rHVSam1ZeqGRx60M1snAo1rrYq31NmAFjt0av9Ja/1trXYEJVPV9Fqdggvxa4BPAF7jO\neq76Pie1frYacw1WT9i1dlm01iutdVcMPAkkKqX8lel6PBP4hdb6uPWe243W93U10E9VtahPB97T\nWpdfQDnsvQHcrEzXTDB/lyp/FAjBBPifKjdWSi22tmAWKqUeta4Oq7bNJOs2+UqpzxpZjmjMZ/1P\nmLD/BfCxsvZIEEKISyVBUAghHB21f6KU6q2U+qe1u1oe8D+YL3l1+cnucRFQ3+AedW0baV8OawDK\nquc4IzFfGt+3Pn8XGKyU6m993hnTSlhdZ+CU1jq3nmPXp/p7dYW1O1+OUioXmE3Ve1VXGcCElBuV\nUr6Y1pcvK7vc1eJdYLL1y/BkYLPWuvK9mYVpsdqrlNqilLq+jmM4sL6/b1IVnh1aJJVSAUqp161d\nEvMw94fV9xmoVL0eLdjVo1Kqo1LqA2W6u+YBKxt53Mpjn7K2YFU6DNh3Wa3++fKv53gzgPetAes8\nJlxVdg+t73NSX702hu39UWZk3eeUGVQlDzhgfSkMc4+cV23nspZ3FaYrtzvmM/RW9e2s53hdVQ0Q\n9UgdZfoKyAMmKKV6YVqvK3+YOANooJPd+X9pvU/wH0BlK+rpatt8ZN3mYet1NMZ5TJj/j9a6FHjW\nesxejdxfCCHqJUFQCCEcVe/K9WdMF7Ue1i6Hv8XuPq9mchwT7ABbi1+t96RZzcD8e75DKfUTZkAK\nTdUX+aNA91r2OwqEWbu7VVcI+Nk971jLNtXfq/cwXeY6a62DgNepeq/qKgPajKS6FdOVcTp1fIm3\nbrsdE3CuxbFbKFrrvVrrWzFdF/8EfFjZ3bMR3gTGWrs6JuE4KMfDQFcg2foZuLqRxzyOCUqAGZUT\nu3rFfLEvAQZYjzsTx89Wfd0KszF1Zx/uYoBjjSybjVKqC6Ylcqb1B4+fMHUxQSnVnvo/J3XWK434\nDFXrOnkn5l68qzFds3tUFhE4gRm8p65zvYHpljwWOKu1/q62jbTWs3XVAFHP1bGN/Q8D04HPtNan\nrK/lAWmYbrT1WQvcZP27e7G24/gZ0NT/mRBCiAsiQVAIIeoXCOQChdaBK+q7P7CpfIpp0ZtgvU9r\nARBe24ZKKT9Mt767MV3/KpcHgdutLSR/BWYrpUZbB+SIVkrFaa2PAmuApUqpYKWUp1JqpPXQPwAD\nlVIDrC11TzSi3IHAGa11sVJqKKZlptLbwDhlhuH3UEqFKaUG2b3+JvAY5n6xjxs4z7vW67sCu/sA\nlRl8Jsza8paL+dJsaUS50Vr/CGy2HvtfWuscu5cDMS1qZ60Dh9SYXqAOnwLxSqkbrS2YD+JYj4GY\nsJRr7X75ULX9T2DuG6ytvIcwgeT31gFT4jEtom83smz27gR2A3FUfX7iMIH71gY+J7V+tqyvpQO3\nWus7mYbDUyAmGJ/GBMjf2V1vBabF9EVrS6q7MoMHVXaT3IhpaXuWen5IuABvYu4HvouqbqGVHgbu\nVUo9oqyDMlnrr4vdNs9jfpB4U5n5H5U1SNt/5iunKfHBhF1PpZSPXXh8CxiulLra+vf4IcwPABfS\n9VYIIeokQVAIIer3K0zLWj6mdfD9+je/dFrrE5j71BZjvhR3xwwzX9vIgpOsZXtba/1T5YIZ3MQX\nGKO1/ga4BzNYSC7wJVUtVZWDbOzDBI/7rWXYjbkXLhXzxbMx9w7eB/xBmRFXF2IGvKm8pkOYAWR+\njeletw0YYLfvh5jQs8ra1a8+72Jajb7QWp+1W389sMd6/ueBW7TWpdbQUKCUuqKB476B+TJffaCa\nxZgWqtOYAWsaNQiJXT3+EXOPZwwmbFZ6AnOfXy7mvrzqA5D8HnjKem/ZA7Wc4hbMwCo/YQLxQq11\namPKVs2dwFL7z4/W+jjm817ZqlzX56S+z9Z/Y4L9OcwgOA1NfbACE3SygV2Y99reg8AeTOvxGcz7\no6zlqGzF64+53/aSaK0PAFsAb+Cf1V77CjMYzGjggFLqHOYzsQZ41brNScxgL+XW68jHfOZ9gHl2\nh1uH6QKajBl05zxmUKrKv4MzMC3rZzGf759fwr2PQgjhQOlLGtBKCCFEc7O2BmQDU7TWG5xdnuZg\nbQU5BMy8yDAj2jhlps+4U2s9ytllEUIIVyAtgkII0QIpM/9csDIjFz6OGRFzi5OL1ZxuxrR4fuXs\nggjXY71Xci5mVF8hhBCNIEFQCCFapuHAQcx0A9cCN2mta+sa6vKUUhsxXQvnaemmIi6QUuoG4CRm\nLs5m77othBCthXQNFUIIIYQQQog2RloEhRBCCCGEEKKNkSAohBBCCCGEEG2Mh7ML0JTCwsJ0bGys\ns4tRQ2FhIf7+/g1vKFoEqS/XI3XmeqTOXIvUl+uROnM9Umeup6XW2datW09prWudf9heqwqCsbGx\npKWlObsYNaSmpjJq1ChnF0M0ktSX65E6cz1SZ65F6sv1SJ25Hqkz19NS60wpdbgx20nXUCGEEEII\nIYRoYyQICiGEEEIIIUQbI0FQCCGEEEIIIdqYVnWPYG3KysrIysqiuLjYaWUICgpiz549Tju/uDDO\nrC8fHx+io6Px9PR0yvmFEEIIIUTb0OqDYFZWFoGBgcTGxqKUckoZ8vPzCQwMdMq5xYVzVn1prTl9\n+jRZWVl07dr1sp9fCCGEEEK0Ha2+a2hxcTGhoaFOC4FCNJZSitDQUKe2XgshhBBCiLah1QdBQEKg\ncBnyWRVCCCGEEJdDmwiCwnnS09P57LPPLni/7OxspkyZ0gwlMlJTU/nmm2+a7fhCCCGEEEK0ZBIE\nW5Hy8nJnF6GG+oJgfeWNjIxk1apVzVUsCYJCCCGEEKJNkyB4Gdx2220kJibSr18/li9fblv/+eef\nM3jwYAYNGsQ111wDQEFBAbNmzWLAgAEMHDiQDz/8EICAgADbfqtWrWLmzJkAzJw5kzlz5pCSksIj\njzzCli1buOKKK0hISODKK69k7969AFRUVPDQQw/Rv39/Bg4cyJIlS1i3bh0///nPbcf94osvuOmm\nm2qUf+3atSQkJDBgwADuuusuSkpKAIiNjeWJJ55g8ODBDBgwgIyMDIf9SktL+e1vf8v7779PfHw8\n77//Pk8++STTp09n2LBhTJ8+nczMTEaMGMHgwYMZPHiwLZxlZmbSv39/AFauXMmkSZMYN24cPXv2\n5JFHHqn1fX700Ufp27cvAwcO5KGHHgIgJyeHyZMnM2TIEIYMGcLXX39NZmYmy5Yt44UXXiA+Pp4N\nGzY0siaFEEIIIURrVmGpoKC0gJOFJ8k8l8nunN2kZaex/vB6Pj/wOR/t+Yi3t7/N8q3LySnJcXZx\nL0mrHzW0JVi6dCldunTh/PnzDBkyhMmTJ2OxWLjnnntYv349Xbt25cyZMwA8/fTTBAUFsWPHDgDO\nnj3b4PGzsrL45ptvcHd3Jy8vjw0bNuDh4cGaNWtYuHAhH374IcuXLyczM5P09HQ8PDw4c+YM7du3\nZ+7cueTk5BAeHs6KFSu46667HI5dXFzMzJkzWbt2Lb169eLOO+/ktdde44EHHgAgLCyMbdu28eqr\nr/L888/z+uuv2/b18vLif/7nf0hLS+OVV14B4Mknn2T37t1s3LgRX19fioqK+OKLL/Dx8WH//v3c\ndtttpKWl1bjG9PR0vv/+e7y9vYmLi+P++++nc+fOttdPnz7N6tWrycjIQCnFuXPnAFiwYAEPPvgg\nw4cP58iRI1x77bXs2bOHOXPmEBAQYAuMQgghhBCi5dJaU1pRSlFZ0cUv5TXXFZYWOjwvqShpdJkW\n9V/UjFfc/NpUEHzg8wdI/ym9SY8Z3zGeF8e9WO82y5Yts3WPPHr0KPv37ycnJ4eRI0fapgkICQkB\nYM2aNbz33nu2fdu3b99gGaZOnYq7uzsAubm5zJgxg/3796OUoqyszHbcOXPm4OHh4XC+6dOn8/bb\nbzNr1iw2bdrEm2++6XDsvXv30rVrV3r16gXAjBkzWLp0qS0ITpo0CYDExEQ++uijBssKMHHiRHx9\nfQEzz+P8+fNJT0/H3d2dffv21brPNddcQ1BQEAB9+/bl8OHDDkEwKCgIHx8f7r77bsaPH8/48eNt\n1717927bdnl5eRQUFDSqnEIIIYQQovG01pwvP8+54nO2Ja8kr9FBrbCssN7XLdpywWXy8fDBz9Ov\n1iXML8w89qj99YaWXWm7muFdvHzaVBB0htTUVFJTU9m0aRN+fn6MGjXqoqYHsB9Nsvr+/v7+tseP\nP/44o0ePZvXq1WRmZjJq1Kh6jztr1iwmTJiAj48PU6dOtQXFxvL29gbA3d290fco2pf3hRdeICIi\ngh9++AGLxYKPj0+956nrXB4eHmzZsoW1a9eyatUqXnnlFdatW4fFYuHbb7+t87hCCCGEEMLQWlNU\nVuQQ5OpcSmpfX25p3PdBN+WGv6d/rQEr3C/8ooJZ9cXXwxd3N/dme7/2u+1vtmNfDm0qCDbUctcc\ncnNzCQ4Oxs/Pj4yMDL799lsAhg4dyty5czl06JCta2hISAhjxoxh6dKlvPiiKevZs2dp3749ERER\n7Nmzh7i4OFavXl3nhOe5ublERUUB5t66SmPGjOHPf/4zo0ePtnUNDQkJITIyksjISJ555hnWrFlT\n43hxcXFkZmZy4MABevTowVtvvcVVV13V6OsPDAwkPz+/3vcnOjoaNzc33njjDSoqKhp9bHsFBQUU\nFRVx/fXXM2zYMLp16wbA2LFjWbJkCQ8//DBgupjGx8cTGBhIXl7eRZ1LCCGEEKIl0lpTWFbYuCBn\nt+SW5DY6yPl6+BLsE0ywTzBBPkGE+YXRI6QHwd7BtvX2S6B3YK2Bz8vdS6bNcrI2FQSdYdy4cbzy\nyiv06dOHuLg4hg4dCkB4eDjLly9n0qRJWCwWOnTowBdffMFvfvMb5s2bR//+/XF3d+eJJ55g0qRJ\nLFq0iPHjxxMeHk5SUlKd3RsfeeQRZsyYwTPPPMMNN9xgWz979mz27dvHwIED8fT05J577mH+/PkA\n3H777eTk5NCnT58ax/Px8WHFihVMnTqV8vJyhgwZwpw5cxp9/aNHj2bRokXEx8fz2GOP1Xh97ty5\nTJ48mTfffJNx48Y5tBZeiPz8fG688UaKi4vRWrN48WIAXn75ZebNm8fAgQMpLy9n5MiRLFu2jAkT\nJjBlyhQ+/vhjlixZwogRIy7qvEIIIYQQTUVrTUFpwQUFt+pLha7/R3U/Tz+CvINsQa2Dfwd6hfaq\nNcRVX4K8g/D28K73+MJ1KK21s8vQZJKSknT1gUb27NlTa8C5nPLz8+tswWsJ5s+fT0JCAnfffbez\ni9IiOLu+WsJn1tWkpqY22A1atCxSZ65F6sv1SJ21XKeLTpNxKoO9p/ey99Re9p7ey77T+zh27hgF\nFQUN3gfn5+lXe1Cro0XOIcj5BOHl7nWZrrT1a6l/z5RSW7XWSQ1t16wtgkqpccBLgDvwutZ6UbXX\n2wN/A7oDxcBdWuuddq+7A2nAMa31+OYsa1uVmJiIv78/f/rTn5xdFCGEEEKIVqGsoowfz/7I3lN7\nq0KfNfidPn/atp2Xuxc9QnrQO6w3cV5x9Over97WOAlyoik1WxC0hrilwBggC/hOKfWJ1nq33WYL\ngXSt9U1Kqd7W7a+xe30BsAdo11zlbOu2bt3q7CIIIYQQQrgcrTU5RTm2Vr29p/aScTqDvaf2cvDs\nQYcumhH+EfQO683kPpOJC4sjLjSOuLA4YoNj8XAzX8dbauuSaL2as0UwGTigtT4IoJR6D7gRsA+C\nfYFFAFrrDKVUrFIqQmt9QikVDdwA/A74ZTOWUwghhBBCiFqVlJfw49kfTcteZeg7bVr6zhWfs23n\n7e5Nz9CeDIwYyM39braFvbjQOIJ8gpx4BULUrjmDYBRw1O55FpBSbZsfgEnABqVUMtAFiAZOAC8C\njwAt9+Y6IYQQQgjh8rTWnCg8YQt69vfwHTp3yOG+vcjASOJC47i13622oNc7rDcxQTHNOlWBEE2t\n2QaLUUpNAcZprWdbn08HUrTW8+22aYe5hzAB2AH0Bu7BhMHrtdZzlVKjgIfqukdQKXUvcC9ARERE\nov1k7GAmGu/Ro0cTX92FqaiosE34Llo+Z9fXgQMHyM3Nddr5XVFBQQEBAQHOLoa4AFJnrkXqy/VI\nndWu1FJKVlEWR88f5WjRUY6cP8LRIvO4sKLQtp2XmxfRvtHE+MXQ2bcznf06E+MXQ7RvNP4eFzfC\neUOkzlxPS62z0aNHO32wmGNAZ7vn0dZ1NlrrPGAWgDITiRwCDgK3ABOVUtcDPkA7pdTbWus7qp9E\na70cWA5m1NDqfav37Nnj9BE7nT0Kpbgwzq4vHx8fEhISnHZ+VyT3VbgeqTPXIvXletpynWmtOV5w\nvNbWvcxzmWiqGkGi20UTFxbH1aFXmwFbrC18nYM646bcLmu523KduSpXr7PmDILfAT2VUl0xAfBW\nYJr9BkqpYKBIa10KzAbWW8PhY9YFuxbBGiFQtD6pqak8//zzfPrpp3zyySfs3r2bRx99tMZ2AQEB\ndc6lCHDu3Dneffdd5s6dC0B2dja/+MUvWLVqVbOVvTapqal4eXlx5ZVXXtbzCiGEaFvKKso4WXiS\nk4UnOVF4gs05mzmXcQ4PNw883DxwV+5Vj93c61zXmPVuyq1FTAReVFbE/tP7HaZhyDiVwb7T+8gv\nzbdt5+fpR1xoHCnRKdw56E7bvXu9QnsR4NXyWnOEuFyaLQhqrcuVUvOBf2Omj/ib1nqXUmqO9fVl\nQB/gDaWUBnYBMpHdJSgvL8fDo1lnBLmsJk6cyMSJEy9q33PnzvHqq6/agmBkZORlD4FggmBAQIAE\nQSGEEBfsfNl5ThSe4ETBCVvAO1FwwvxZaF1nfX7m/JmaB9hdc1VTqQyFFxIg61pf77bKcb9yS7lt\nWoYjuUccWvdigmKIC41jZvxMh4FaotpFXfbWPSFcQbOmBq31Z8Bn1dYts3u8CejVwDFSgdRmKN5l\nc9ttt3H8+HGKi4tZsGAB9957LwCff/45CxcupKKigrCwMNauXUtBQQH3338/aWlpKKV44oknmDx5\nskML2KpVq/j0009ZuXIlM2fOxMfHh++//55hw4Zx6623smDBAoqLi/H19WXFihXExcVRUVHBr3/9\naz7//HPc3Ny455576NevHy+//DL/93//B8AXX3zBq6++yurVqx3Kv3btWh566CHKy8sZMmQIr732\nGt7e3sTGxjJjxgz+8Y9/UFZWxv/+7//Su3dvh32HDh3KX//6V/r16wfAqFGjeP7557FYLLWW097K\nlStJS0vjlVde4dChQ0ybNo2CggJuvPFG2zaVz8+ePUtZWRnPPPMMN954I48++ig//vgj8fHxjBkz\nhnnz5jF+/Hh27txJcXEx9913H2lpaXh4eLB48WJGjx7NypUr+eSTT8jLy+Pw4cPcdNNNPPfcczXq\n89FHH+WTTz7Bw8ODsWPH8vzzz5OTk8OcOXM4cuQIAC+++CJRUVEsW7YMd3d33n77bZYsWULv3r1r\nbDds2LCL/mwJIYRwHVpr8kvzq8JcAwHPvlXLXjvvdkT4R9DBvwN9wvswKnYUEf4RRASYdRH+EWRs\nz2Bw4mAqdAXllnLKLeVUWMzj2tY1Zv2FbFvXNvbryi3lFJcXX1CZALq178awmGHcFXqX6c4ZGkfP\n0J74efpdzuoUwuW1nuajFmzp0qV06dKF8+fPM2TIECZPnozFYuGee+5h/fr1dO3alTNnzC95Tz/9\nNEFBQezYsQOAs2fPNnj8rKwsvvnmG9zd3cnLy2PDhg14eHiwZs0aFi5cyIcffsjy5cvJzMwkPT0d\nDw8Pzpw5Q/v27Zk7dy45OTmEh4ezYsUK7rrrLodjFxcXM3PmTNauXUuvXr248847ee2113jggQcA\nCAsLY9u2bbz66qs8//zzvP766w7733LLLXzwwQc89dRTHD9+nOPHj5OUlFRnOeuyYMEC7rvvPu68\n806WLl1qW+/j48Pq1atp164dp06dYujQoUycOJFFixaxc+dO0tPTAcjMzHSoD6UUO3bsICMjg7Fj\nx7Jv3z4A0tPTWb9+PWFhYcTFxXH//ffTuXPVra6nT59m9erVZGRkoJTi3LlztvI9+OCDDB8+nCNH\njnDttdeyZ88e5syZQ0BAAA899BAA06ZNq3U7IYQQrsmiLZw5f8ahdc4h4NmFvJOFJykuL671OKG+\noUQERBDhH0FSZBId/DrYntsHvA7+HfD19G2wXGUHy0joJPebCyHq1raC4AMPgDUYNJn4eHjxxXo3\nWbZsGZ99ZhpGjx49yv79+8nJyWHkyJF07doVgJCQEADWrFmD/cin7du3b7AIU6dOtY1ymZuby4wZ\nM9i/fz9KKcrKymzHnTNnjq3raOX5pk+fzttvv82sWbPYtGkTb775psOx9+7dS9euXenVyzTczpgx\ng6VLl9qC4KRJkwBITEzko48+qlG2m2++mbFjx/LUU0/xwQcfMGXKlHrLWZevv/7aFhSnT5/Or3/9\na8D8urpw4ULWr1+Pm5sbx44d48SJE/Uea+PGjdx///0A9O7dmy5dutiC4DXXXENQUBA+Pj707duX\nw4cPOwTBytfuvvtuxo8fz/jx423v7+7dVX1w8vLyar2Hsa7tWuKIU0II0VaVW8rJKcyp2VpnF+gq\nn+cU5VBuKa9xDHflTrh/uC3I9Q7rbQty9gEvwj+CML8wPN09nXClQoi2rG0FQSdITU0lNTWVTZs2\n4efnx6ht+ml8AAAgAElEQVRRoygurv3XwPrY35RdfX9//6phjB9//HFGjx7N6tWryczMbHAko1mz\nZjFhwgR8fHyYOnXqBd9j6O3tDYC7uzvl5TX/I4yKiiI0NJTt27fz/vvvs2zZsosqJ1DrjenvvPMO\nOTk5bN26FU9PT2JjYy/q/a1+PXVdk4eHB1u2bGHt2rWsWrWKV155hXXr1mGxWPj222/x8fGp9/iN\n3U4IIUTzKSorYmv2Vr7L/o4juUdq3G93uui0w71nlbzdvW2tc1GBUQzuONgW5qoHvBDfELkvTQjR\norWtINhAy11zyM3NJTg4GD8/PzIyMvj2228Bc+/c3LlzOXTokK1raEhICGPGjGHp0qW8aC3r2bNn\nad++PREREezZs4e4uDhWr15d5/QGubm5REVFAeYeu0pjxozhz3/+M6NHj7Z1DQ0JCSEyMpLIyEie\neeYZ1qxZU+N4cXFxZGZmcuDAAXr06MFbb73FVVdddUHvwS233MJzzz1Hbm4uAwcOrLecdRk2bBjv\nvfced9xxB++8847D9Xbo0AFPT0++/PJLDh8+DEBgYCD5+bXfWzFixAjeeecdrr76avbt28eRI0eI\ni4tj27ZtDZajoKCAoqIirr/+eoYNG0a3bt0AGDt2LEuWLOHhhx8GTBfT+Ph4AgMDycvLs+1f13ZC\nCCGaR4WlgoxTGWw+tpnNWZvZkr2FHSd2UKHN/WYBXgG28NYztCfDY4Y7tNbZB7x23u1axGiZQgjR\nFNpWEHSCcePG8corr9CnTx/i4uIYOnQoAOHh4SxfvpxJkyZhsVjo0KEDX3zxBb/5zW+YN28e/fv3\nx93dnSeeeIJJkyaxaNEixo8fT3h4OElJSXVOnfDII48wY8YMnnnmGW644Qbb+tmzZ7Nv3z4GDhyI\np6cn99xzD/Pnzwfg9ttvJycnhz59+tQ4no+PDytWrGDq1Km2wWLmzJlzQe/BlClTWLBgAY8//niD\n5azLSy+9xLRp03j22WcdBou5/fbbmTBhAgMGDCApKck2WE1oaCjDhg2jf//+XHfddcybN8+2z9y5\nc7nvvvsYMGAAHh4erFy50qElsD75+fnceOONFBcXo7Vm8eLFALz88svMmzePgQMHUl5ezsiRI1m2\nbBkTJkxgypQpfPzxxyxZsqTO7YQQQjSN7PxsthzbwuaszWw+tpm07DTboCtB3kEkRyXz2PDHSI5K\nJjkqmYiACCeXWAghnENpXbPrg6tKSkrSaWlpDuv27NlTa8C5nJw9QXlD5s+fT0JCAnffLbN3gPPr\nqyV8Zl2Nq0/o2hZJnbmWllpfBaUFbM3eyuZjm034O7aZrLwsADzcPIjvGE9yZDIp0SmkRKXQM7Rn\nm+mu2VLrTNRN6sz1tNQ6U0pt1VonNbSdtAi2cYmJifj7+/OnP/3J2UURQggh6lRhqWB3zm6HLp47\nT+7Eoi2AmVJgRMwIUqJSSI5KJqFTAj4ecj+2EELURYJgG7d161ZnF0EIIYSoISsvywQ+a0tfWnYa\nhWWFALT3aU9yVDI/j/s5KdEpDIkcQrh/uJNLLIQQrkWCoBBCCCGcKr8kn7TsNNPaZ+3mmZ2fDYCX\nuxfxHeO5K+EukqOSSYlKoUdIDxm0RQghLlGbCIJaa/kPQ7iE1nTPrhBC1KbcUs7OkzsdBnTZnbPb\nNl1Dz5CejI4dTUpUCinRKQyKGIS3R+MG9BJCCNF4rT4I+vj4cPr0aUJDQyUMihZNa83p06dljkEh\nRKuhteZo3lFb4NtybAtbj2+lqKwIgFDfUFKiU5jadyop0ebevhDfECeXWggh2oZWHwSjo6PJysoi\nJyfHaWUoLi6WL/cuxJn15ePjQ3R0tFPOLYS4dG29B0pucW6NLp4/FfwEmMnYEzolcM/ge2wDunRr\n361Nv19CCOFMrT4Ienp60rVrV6eWITU1lYSEBKeWQTSe1JcQoiGlFaVknMrgh59+4IcT1uWnHzhV\ndAo/Tz/8vfzx8/Sre/Go57VGLO5u7s5+CyirKGPHyR22wVw2Z20m41SGrYtnXGgcY7qNsXXxHBgx\nEC93LyeXWgghRKVWHwSFEEKIS5FTmGMLepWhb0/OHsosZQD4ePjQv0N/JsZNpFNAJ86Xn6eorMhh\nKSwr5Mz5M2TlZTmuLy2kQldccJm83L0aFxrtAmeD4bTa4u3ubWut01qTeS6zRhfP4vJiAML9wkmJ\nTmHagGkkRyUzJHII7X3bN10lCCGEaHISBIUQQgjMICb7Tu+r0cp3vOC4bZvIwEgGRQzi+h7XM6jj\nIAZFDKJnaE883C7+v9OyirIawfFClsKyQofnZ8+frbHN+fLzF1wuhbKFwpLSEvLW5wEm+CZ2SuS+\npPtsrX1dgrpIF08hhHAxEgSFEEK0OWfPn2X7ie0OLX27cnbZWrg83TzpG96XMd3HMCjCBL5BHQcR\n5hfW5GXxdPckyD2IIJ+gJj92JYu2UFxefHFBs7SQY9nHuC7hOlKiUxjQYQCe7p7NVlYhhBCXhwRB\nIYQQrZZFW/jxzI81unYeyT1i2ybcL5xBHQcxb8g8W+DrHda7Vd3P5qbcbK17FyM1NZVRQ0Y1baGE\nEEI4lQRBIYQQrUJ+SX6NVr4dJ3fYpipwV+7EhcUxrPMw5ibNtXXt7BjQUbo1CiGEaHMkCAohhHAp\nlQOXVG/lO3j2oG2bYJ9gBkUMYnbCbFvg69ehHz4eMpWPEEIIARIEhRBCtGBFZUXsPLnTIfBtP7Gd\nvBIzcIlC0TO0J4mdErkr/i5b6ItuFy2tfEIIIUQ9JAgKIYRwOq01x/KP1Rixc/+Z/Vi0BYBAr0AG\nRgzkjgF32AJf/w798ffyd3LphRBCCNcjQVAIIcRlVVxezL78fRz6/pBDK9+Z82ds23QN7sqgjoO4\ntf+ttgFcYoNjcVNuTiy5EEII0XpIEBRCCNFkLNpCTmEOR3KPcCT3CEfzjtZ4fKLgBBoN28DXw5cB\nEQOY3GeyLfANjBhIO+92zr4UIYQQolWTICiEEKLR8kvyHcNd7lGO5FU9Ppp3lNKKUod9/Dz9iAmK\noXO7ztzQ8wY6t+uM5aSFaVdPo0dID9zd3J10NUIIIUTbJUFQCCEEAGUVZRzLP1YV8Gpp0TtXfM5h\nHzflRlRgFJ2DOjMkagiT+0ymc1BnYoJibOEvxDekxsAtqampxIXFXc7LE0IIIYQdCYJCCNEGaK05\nVXSqRjdN+6B3PP+46bJpJ8Q3hJigGGKDYxnZZaQt3FUGvU6BnfBwk/9KhBBCCFcj/3sLIUQrUFha\n2GCXzeLyYod9fDx8bKFubPexxLSztuJZW/Q6t+ssI3IKIYQQrZQEQSGEaOHKLeVk52fbumvW1mXT\nfsRNMPPrRQZG0jmoMwmdEpgYN9Ghu2ZMUAxhfmEy154QQgjRRkkQFEKIFsKiLew7vY8tx7awOWsz\nP5z4gcO5h8nOz7bNpVcp2CfYFuiu7HylQ3fNzkGdiQqMwtPd00lXIoQQQoiWToKgEEI4yfH842w5\ntsUEv2Ob+S77O/JK8gAI8AogoWMCV3e9mph2MTUGYAn0DnRy6YUQQgjhypo1CCqlxgEvAe7A61rr\nRdVebw/8DegOFAN3aa13KqU6A28CEYAGlmutX2rOsgohRHPKL8ln6/GtDsEvKy8LAHflzsCIgUzr\nP43kqGSSo5LpHdZbplUQQgghRLNptiColHIHlgJjgCzgO6XUJ1rr3XabLQTStdY3KaV6W7e/BigH\nfqW13qaUCgS2KqW+qLavEEK0SGUVZew8udMh9O3O2W0bkbNb+24MjxlOSlQKyVHJJHRMwNfT18ml\nFkIIIURb0pwtgsnAAa31QQCl1HvAjYB9mOsLLALQWmcopWKVUhFa6+PAcev6fKXUHiCq2r5CCOF0\nWmsOnTtkC31bjm1h2/FtnC8/D0CobyjJUclM6TuFlKgUhkQNIcwvzMmlFkIIIURb15xBMAo4avc8\nC0ipts0PwCRgg1IqGegCRAMnKjdQSsUCCcDmZiyrEEI0yumi01WhL9v8earoFGCmYxjcaTD/lfhf\nJEclkxKdQtfgrjIypxBCCCFaHKW1bnirizmwUlOAcVrr2dbn04EUrfV8u23aYe4hTAB2AL2Be7TW\n6dbXA4CvgN9prT+q4zz3AvcCREREJL733nvNcj2XoqCggICAAGcXQzSS1Jfraa46K6ko4UDBAfbk\n72FP/h4y8jLILs4GzPQMXfy60Ltdb/oE9qFPYB+6+neVydUbSf6euRapL9cjdeZ6pM5cT0uts9Gj\nR2/VWic1tF1zfmM5BnS2ex5tXWejtc4DZgEo85P5IaCyK6kn8CHwTl0h0HqM5cBygKSkJD1q1Kim\nu4ImkpqaSkssl6id1JfraYo6s2gLGacybFM3bMnewvYT2ym3lAMQ3S6aoV2HkhxpBnNJikySkTsv\ngfw9cy1SX65H6sz1SJ25Hlevs+YMgt8BPZVSXTEB8FZgmv0GSqlgoEhrXQrMBtZrrfOsofCvwB6t\n9eJmLKMQoo06lnfMYTCXtOw08kvzAWjn3Y4hkUN4+MqHbaN4RgZGOrnEQgghhBBNp9mCoNa6XCk1\nH/g3ZvqIv2mtdyml5lhfXwb0Ad5QSmlgF3C3dfdhwHRgh1Iq3bpuodb6s+YqrxCi9corySMtO80W\n+rYc20J2vuni6eHmwaCIQUwfON0W+uLC4nBTbk4utRBCCCFE82nWm1mswe2zauuW2T3eBPSqZb+N\ngIyuIIS4YKUVpew4scM2mMvmrM1knMqwTd3QI6QHo2JH2aZuiO8Yj4+Hj5NLLYQQQghxecmoBkII\nl2XRFn488yNrTqzh/z7/P9vUDSUVJQCE+YWREpXCrf1vJSUqhaTIJEL9Qp1caiGEEEII55MgKIRw\nCRWWCvaf2c/W7K1sO76Nrce38v1P35NXkgeAr4cviZGJzBsyzzZ1Q5egLjJ1gxBCCCFELSQICiFa\nnHJLOXtP7WXr8arQl/5TOgWlBQB4u3szqOMgbh9wO4M7DUYf08y8fiae7p5OLrkQQgghhGuQICiE\ncKpySzm7c3Y7tPSl/5TO+fLzgGnpi+8Yz8xBMxncaTCJkYn0CevjEPpS81IlBAohhBBCXAAJgkKI\ny6a0opRdJ3c5tPRtP7Gd4vJiAPw9/UnolMC9ifea0NcpkbiwOJmkXQghhBCiicm3KyFEsygpL2HH\nyR0OLX07Tu6gtKIUgECvQAZ3GszcpLm2lr6eIT1xd3N3csmFEEIIIVo/CYJCiEt2vuw8209sd2jp\n23lyJ+WWcgCCfYIZ3GkwC1IWkNgpkcGdBtM9pLvM1SeEEEII4SQSBIUQF6SwtJAfTvxgWvp+2sbW\n7K3sztlNha4AIMQ3hMROiTx0xUMkRprQ1zW4q4zeKYQQQgjRgkgQFELUKb8kn/Sf0h1a+jJOZWDR\nFgDC/cJJjExkYtxEW0tfTFCMhD4hhBBCiBZOgqAQAoDc4ly+/+l7W+Dbmr2Vfaf3odEAdAroRGJk\nIlP6TLG19EUFRknoE0IIIYRwQRIEhWiDzp4/y7bj26pC3/GtHDhzwPZ6dLtoEjslMm3ANFtLX6fA\nTk4ssRBCCCGEaEoSBIVo5UrKS0jLTmPjkY2kHU9ja/ZWDp07ZHu9S1AXEiMTmTlopq2lr4N/ByeW\nWAghhBBCNDcJgkK0MrnFuXxz9Bs2HNnAxiMb2XJsCyUVJQB0a9+NpMgk7k28l8ROiSR0SiDML8zJ\nJRZCCCGEEJebBEEhXFx2fjYbDpvQt+HIBraf2I5G4+HmweBOg5mfPJ8RMSMYFjNMQp8QQgghhAAk\nCArhUrTW7Du9jw1HNtha/A6ePQiAv6c/V3S+gieueoIRXUaQEpWCv5e/k0sshBBCiDalpAS2bIHU\nVLMcOQLh4dChQ83Ffn1oKHhINLmc5N0WogUrt5Tz/fHvbaFv45GN5BTlAGbqhuExw5k/ZD7DY4YT\n3zEeT3dPJ5dYCCGEEG1K9eD3zTdQXAxKwaBBkJgIp0/DoUOweTPk5EBFRc3jKGXCYG2BsXpo7NAB\ngoLMPuKiSRAUogUpLC1k87HNbDhsWvy+zfqWwrJCwNzfd33P6xkeM5wRMSPoFdpLpm4QrqG8HI4e\nhR9/NEtmJt0OH4avvgJf36rFz8/xefWl8nVvb/nPv7lobb7UlZSYL3LWx94nTzq7ZEKIlqKkxAS6\nyuC3aVNV8IuPh/vug1GjYMQIaN++5v4WC5w9CydP1lxycqoe//CD+fPs2drL4enZ+NDYoYP5/0M4\nkCAohBOdKjpl7u07vIGNRzey7fg2yi3lKBQDIwYyK34WI7qMYHjMcCIDI51dXCHqVlBgQt7Bg1WB\nr/Lx4cMmDFby8CBaKSgru7hzKQU+PhcXIi/mdR+f5g2eFguUljoEr+pBrFGPL2af6vuXltZaxCvA\nfPH7wx/Ay6v53gshRMtzqcGvOjc30/IXGgp9+jS8fWkpnDpVe3C0D49798KJE3D+fO3HCQhoXGDs\n0AHCwtpEN9XWf4VCtBBaazLPZdoGddlwZAMZpzIA8Hb3JjkqmUeufIThMcO5svOVBPkEObnEQtjR\nGn76qe6wV73FqH176N4dkpLg5pvN4+7doVs3iIpi/YYNjBoxwnyZOH/ecSkqqrmuMa9Vvp6TU/tr\nJSUXf/2NDZFubhce0C42EFfn5mZCq7e3WWp77O9vvnxVrq9ru2qPj/3v/xK1eDFs3Ajvvw+xsU1T\nZiFKSyE/v2opKHB8Xn1daSn07An9+kH//hAdLT0EmlpxsWNXz0sNfpfKywsiI83SGIWFdYfGyuB4\n+DB89515Xls3VajZTbWW0OhRUNB01+kEEgSFaCYWbWHnyZ22bp4bj2zkWP4xAIK8gxgWM4wZg2Yw\nImYESZFJeHt4O7nEos0rLYXMzNrD3sGDJmRVUgo6dzbhbuJEE/Dsw15jvhy4u5tg4n+ZBjWqqKg9\neF5qAD11qup1i8UxTHl7m1+hK8NXXaGrkYGs3tea8dfr/dHRRE2fDnffDQkJsHIl3Hhjs51PtGDV\ng1tjwltd6woK6myFrsHdHQIDzef81Kmq9e3amUBov/TrZ76oi8YpLq7Z4ldS4rzgd6n8/aFrV7M0\nxGKBc+caDo47dpjHZ8447N7u2Wdh/PhmupDmJ0FQiCZSUl7Cd9nf2Vr8vj7yNbkluQBEBUaZLp6d\nhzOiywj6d+iPm3JzcolFm3TuXM3WvMrHR4+a/xQr+fpWBbwxYxzDXpcuJny4kssdPFubKVNMCLz5\nZvj5z+HBB2HRIukq2tKVlNQdwi4mvDW2BdvDwwS3gADzZ+USEeH4vLZtaltnf2/wmTOwaxfs3Fm1\nrFoFy5dXnT88vGY47NcPgoOb/j12NfUFv4QEmDfPBL/hw10j+F0KNzcICTFL794Nb19W5tBNNc/+\nB1IXJEFQiItkP3H7hiMb+O7Yd7aJ2/uE9eHmfjczImYEI7qMoEtQFxnYRVweFgscO1Z32Kv2ayYd\nOpiAN3y4Y4te9+7QsaN0uRKOunc3IwI+9BC88AJ8/bV0FW1pzpyBX/6SYatXm1bqCw1u1UNYp06N\nD2v2z5tzUKeQENM6NWJE1Tqtzf1h9uFw505YscKE2UrR0TUDYt++pot3ayXBr+l4epq/E506AVCe\nmurc8lwiCYJCNFJjJm4fHjOcYZ2HEe4ffnkLV1JifqGqbzlzxvxHX9kiYr/4+dW+vrZFfv13vvPn\nzTDctYW9Q4ccu1p5eJjWu8r79ezDXrdu5gubEBfC2xuWLIGrrqrqKrpihWklFM71j3/AvffCqVOc\n+tnP6BQfX39Ys1/n6qPxKmV+vOrYEX72s6r1Fovp7VA9IH75ZdV9w0qZfw/tw2H//hAX55r/5xUX\nw7ffVgW/b7+tPfiNGCEtpG2cBEEhauHUidsrKkxoy8lpONxVLvn5dR8vONh0kQkJMSM3FhY6Lhc6\ngEZdYfJig6X94uvr2l9E6mOxmPe/osL8af+4tnWVdVUZ+OzDXna247EDA02469/f3K9nH/ZiYtrE\nyGfCCSq7it5yC9x0EzzwADz7rGt+cXZ1Z8+a9//NN2HAAPjsM/bm5tJp1Chnl8z53NzMj2FdusAN\nN1StLy83/65WD4ifflo1eIiHB/TqVTMgdu9uupq3FBL8xEWSbwdCWGmt+ef+f/LcrufI+C6jaSZu\n1xpycx1DW0MB7+xZs19t/P3NkMZhYSbcxcVVPa9tCQkx3RjqU1HhGAyLimqGxcYsubkmoFQ/Vl3X\nUhul6g6R9YTLqMxMSE+vP1Q1FLou9nFjt72Q96E2kZHmy8fYsY736nXvbgYiaa0BWrRs3bub7qEP\nPwwvvljVVbQxgzSIpvHZZ3DPPaZb5OOPw29+Y8K4i3dZa3aVIa9XL5g0qWp9SQns2+cYDtPS4IMP\nqrbx8THTHlQPiDExl+ff4rqCn5ubCX7z51d19ZTgJ+ohQVC0eRZtYfWe1Tyz4RnSf0onzCuM6/tc\nb+7vs5+4XWsTbLKya4a3usLd6dOO86fZ8/JyDG3x8VUBr7ZQFxraPJOhurubUdfatWv6Y2tdNeLi\nxYRL++XMmZrrrO9tz/rK4OlprtHDwyyVj2tbV9djH5+L2+9Ctq1tPx+fqpHPZCJc0VJ5e8PLL5sv\nnnfdVdVV9KabnF2y1i031wzYs2KFCSKffAKJic4ulevz9jatqgMGOK4vLIQ9exwD4rp18NZbVdsE\nBlaFwso/+/c3A+RcSkA8f75m8CstleAnLpkEQdFmVVgq+GDXB/xuw+/YlbOLXqG9eP/KFxn8Xjo9\njgdCzlo49b5jsCsurv1glZOjVoa2Xr3gyisdg1z1gBcQ0PpbcSpb+Pz8zDU3tbIyKCzk69RUho0a\nVTNUucnIrEJcNpMmmR+0brnFPF6wAJ57TrqKNod//xtmzza9MBYuhN/+1vVG8XU1/v7mPuukJMf1\nZ8+aEUztRzFdvRpef71qm9DQmq2H/fqZXju1aSj43X+/BD/RJCQIijanrKKMd3a8w+83/J79Z/bT\nL7wf7056l5sDh+J+zc/MPQPBwVWBLTq6qrWu+lIZ7oKDJXQ4g6cnBAdTFhws/xkK0RJ062YmnX/k\nEXjpJTPCqHQVbTp5eWbE1r/8xXRN3LQJkpOdXaq2rX17E8iGD69ap7WZXmDnTseA+Oabjvf0R0Y6\njFwau2EDPPGEBD9x2UgQFG1GaUUpK9NXsmjjIg6dO0R8x3hWTV3FTX1uwu3wERg9Gs6eZeurr5J4\n333OLq4QQrgmb28TAq+6SrqKNqU1a8worVlZ8Otfw5NPmu7jouVRynQHjYiAa66pWq911Qim9gHx\ntdeguJgubm4weDD84hdVwS8oyGmXIVo/CYKi1SsuL+b1ba/z7NfPkpWXRXJUMi9f9zI39LzB3Pt3\n8KAJgfn5sHYt+fWNwCmEEKJxJk2qmoB+0iTz5fa556QL44XKzzeD8fz5z2aAsK+/hqFDnV0qcTGU\nMgPKxMTA9ddXra+ogMxMNu7Zw4jx451XPtHmNGtfNqXUOKXUXqXUAaXUo7W83l4ptVoptV0ptUUp\n1b+x+wrRkMLSQhZvWkzXl7py/7/uJzY4ln/f8W++vftbxvcab0LggQPmV+uCAli7Vm60F0KIptS1\nq+kqumCBGVBm+HDz45tonHXrzKAly5ebLqHffy8hsDVyd4fu3akICHB2SUQb02xBUCnlDiwFrgP6\nArcppfpW22whkK61HgjcCbx0AfsKUau8kjwWbVxE7Eux/Oo/v6JveF++nPEl62euZ2z3sSYAghke\n+qqrzAAw69aZX66FEEI0LW9vM7XERx/B/v2m69tHHzm7VC1bQYEZCfKaa8xgOxs2wB//KKMHCyGa\nVHO2CCYDB7TWB7XWpcB7wI3VtukLrAPQWmcAsUqpiEbuK4SDs+fP8lTqU8S+GMtjax8jKTKJr+/6\nmrV3rmVU7KiqAAiQkWFCYFkZfPklDBrkvIILIURbcNNNpkWrVy+YPNl0FS0pcXapWp6vvoKBA+HV\nV830EOnpMGyYs0slhGiFmjMIRgFH7Z5nWdfZ+wGYBKCUSga6ANGN3FcIAE4VneK/1/43sS/F8uRX\nTzKyy0i2zN7Cv27/F1d2vrLmDrt2mZuwtTbDMvfvX3MbIYQQTa+yq+gDD8CSJSbgSFdRo7DQdKEd\nNcqMFvnVV7B4sZl+RwghmoHSWjfPgZWaAozTWs+2Pp8OpGit59tt0w7THTQB2AH0Bu4BejS0r90x\n7gXuBYiIiEh87733muV6LkVBQQEB0u+7yZ0pPcMHRz/g4+yPKbGUMDJ8JHfE3EGPgB517uN/8CCD\nfvUrtLs7PyxeTFFMTI1tpL5cj9SZ65E6cy3NUV9hGzcS9+yzKK3JePhhTl11VZMe35UE7dhB70WL\n8M3OJmvSJA7Ono3lEruByt8x1yN15npaap2NHj16q9Y6qaHtmjMIXgE8qbW+1vr8MQCt9R/q2F4B\nh4CBQL8L2bdSUlKSTktLa7JraCqpqamMGjXK2cVoNY7lHeO5r59j+bbllFaUclv/21g4YiF9wxu4\njfSHH8z9Fj4+pjtoz561bib15XqkzlyP1Jlrabb6ysw0E9Bv2WLuiXv++bY1qmhREfzmN+YeythY\n+NvfTItgE5C/Y65H6sz1tNQ6U0o1Kgg25/QR3wE9lVJdgWPArcA0+w2UUsFAkfU+wNnAeq11nlKq\nwX1F23P43GEWbVzE39L/hkVbmD5wOo8Nf4yeobUHOgfbtsGYMeDvb0Jg9+7NX2AhhBD1i401A6E8\n+ii88IKZIP3999vGv9HffAMzZ5oBdObNg0WLoAW2LAghWq9mC4Ja63Kl1Hzg34A78Det9S6l1Bzr\n68uAPsAbSikN7ALurm/f5iqraNkOnDnAHzb8gTe3v4mbcmNW/CweHf4oscGxjTtAWpoJgUFBJgR2\n7dqs5RVCCHEBvLzMvXBXXWWC0eDB8Ne/wpQpzi5Z8zh/Hn77W/jTn8x8cmvXwtVXO7tUQog2qFkn\nlFubFcEAACAASURBVNdafwZ8Vm3dMrvHm4Bejd1XtC17cvbwuw2/4+87/46Xuxf3Jd3HI8MeIbpd\ndOMPsnkzXHsthISYKSJiY5utvEIIIS7BjTeaUUVvuQWmTm2dXUU3bzZhNyMD/uu/zJQQgYHOLpUQ\noo1q1iAoxMXYfmI7z6x/hlW7V+Hr6csvh/6SX135KzoGdLywA33zDYwbBx06mBBYy8AwQgghWpDK\nrqKPPWZaCb/5Bj74wPW7ihYXw5NPmuAXFQX/+Y/pqSKEEE4kQVC0GFuzt/L0+qf5eO/HBHoF8tjw\nx3jwigcJ8wu78INt3AjXXQedOpkQGH0BrYhCCCGcx8vLdJu07yr6+uumldAVffeduY7du2H2bHNt\n7do5u1RCCNGs8wgK0Sibjm7i+neuJ+kvSXx1+CuevOpJDj9wmN9d87uLC4FffWVaAqOizDyBEgKF\nEML1TJxouor26QM332y6ihYXO7tUjVdSAv/933DFFZCbC59/Dn/5i4RAIUSLIS2Cwim01nx1+Cue\nXv806w6tI8wvjN9f/XvmJc+jnfcl/Ce5bh2MH2+6F61bBx0vsDupEEKIlqNLF1i/HhYuNC1plV1F\ne9Q9X2yLsG0bzJgBO3fCrFmmm2twsLNLJYQQDqRFUFxWWmv+8+N/GLlyJKPfGM2uk7t4fszzZC7I\n5LERj11aCPziC7jhBnMvSWqqhEAhhGgNvLzMoDEff2zmHRw82ITBlqi01IwImpwMZ87AP/9p5gaU\nECiEaIGkRVBcFlpr/rn/nzy9/mm2HNtCdLtolly3hLsT7sbX0/fST/D55/Dzn0NcHKxZA+Hhl35M\nIYQQLUdlV9FbbzUji371lWkl9PFxdsmM9HTTCrh9O9x5p5kkvn17Z5dKCCHqJC2CollZtIWP9nxE\n4vJEJvx9AicLT/Ln8X/mwP0HmJ88v2lC4GefmWHH+/Qx3UElBAohROtU2VX0oYfg1VfhyivhwAHn\nlqmsDJ56CoYMgZMn4ZNP4I03JAQKIVo8CYKiWVRYKvj7jr8z8LWBTP5gMgWlBay4cQX75u/j3sR7\n8fZoonmh/vEP0xI4YICZlDc0tGmOK4QQomXy9DTTMHzySVVX0fffd05Ztm+HlBQzNcQtt8CuXTBh\ngnPKIoQQF0iCoGhSZRVlvJH+Bn1f7cu0j6ah0bwz6R32zNvDzPiZeLp7Nt3JVq+GyZMhIcF0Bw0J\nabpjCyGEaNkmTDDdMfv1M91F5869fKOKlpXx/+3de5xVdb3/8deHARSBNMNGAswbomiAyC9NOoZ6\nTPFeXklUmDpqR8usR16yk5WVWaaWUYq5Aa94y8LL8ZJG2ckKbQYREOWAIYiKmuGICgPf3x9r1xmR\nywCzZ82a/Xo+Hjzce+11+czjgzO857vW98t3vgPDhsGiRdnPoxtv9OeQpELxGUG1iuUrlzOpYRKX\n/OES5r8+n8G1g7n9uNv59G6fplNU4PcNd9wBo0ZlP4Tvvx+23LL1ryFJat+22y67VfTCC7NRwsce\nyyaS6d+/ctd86qlsXcAnnsgC6FVXQa+NWOpIknLmiKA2ydtNbzPuL+PY+Sc7c9o9p9Fri15MOXEK\n9afXc+zAYysTAm+9Nfvhu/fe8MADhkBJqmZdusAPfpA9KrBgAey1V2VuFW1qgksuyc6/YEH2C8lb\nbjEESiosRwS1UZatWMY1j1/DD//4QxY3LmZ4v+Fce8S1fHKnTxIRlbvwzTfDySfDxz+eTcvdo0fl\nriVJKo7DD/+/WUVPPDFbRuiKK1pnVtFZs7JRwGnT4LjjYNw4JyaTVHiOCGqjHHHLEXz5wS+za69d\neeSUR3h07KMcvPPBlQ2B11+fhcBPfCKbKdQQKElqbrvtsmUlzj0Xrr4a9tkHnn1248+3cmV2y+nQ\noTBvXjbSeNtthkBJHYJBUBts1pJZPDL/ES7e/2IeOfUR9t9h/8oGQIAJE7Lfxh5wANxzD3TvXtnr\nSZKKqUsXuPTS7GfF889nIW7y5A0/z5w52d0n554Lhx6azQh6/PGtX68k5cQgqA02oX4CnTt15j+G\n/kfbXPDaa6GuDg46KJsufIst2ua6kqTiOuywbFbRQYOyycXOOAPeemv9x61cmS1UP2QIPPNM9kjC\nnXdCbW3la5akNmQQ1AZZsXIF1z95PYfvcji1Pdrgh+LVV8Npp8HIkfDrX0O3VliAXpJUHfr1y54V\nPPdcuOYa+NjHsnC3Ns8+C/vtly1Yf/DB2SjgqFFQ6bteJCkHBkFtkPuevY+X33yZuiF1lb/YT38K\nn/98NgHAXXe1zgP/kqTq8s9bRe+9FxYuzGb9vOWWd++zahX8+McweDDMng033JD93Nl223xqlqQ2\nYBDUBik1lNi2x7aM7D+yshe68kr4whfgqKOyW3I226yy15MkdWyHHprNKjp4MHzmM3D66dmtonPn\nwogR8KUvwYEHZusEjh7tKKCkDs8gqBZ7sfFF7n3mXk4dfCqdO1Vw5ZEf/QjOOQeOOQZuvx26dq3c\ntSRJ1aNfP/jtb+H882H8eNhzzywYPvkkTJyYPYf+oQ/lXaUktQmDoFrs+unXszKtZOyQsZW7yKWX\nZs9mHH98dutOly6Vu5Ykqfp06ZItDH/vvfDaa9lo4MyZcOqpjgJKqiouKK8WSSlRqi8xvN9wBvQa\nUJmLfPe78PWvZ7fsTJoEnf3rKUmqkEMPhRdfhE7+TlxSdfK7n1rksYWPMefVOdTtWaFJYr71rSwE\nnnxytnC8IVCSVGmGQElVrEXfASNip4jYrPx6RER8MSK2qmxpak9K9SW6d+nOcQOPa90TpwTf+AZ8\n85swdmy2cHxNTeteQ5IkSdK7tPRXYXcCKyNiZ2A80A+4uWJVqV1pXN7IrTNv5fjdj6fnZj1b78Qp\nwYUXwsUXw+c+B7/4hSFQkiRJagMtDYKrUkpNwKeAq1JKXwV6V64stSd3zLqDxuWNrXtbaEpw3nnZ\nA/tnnJEt9OstOpIkSVKbaOmDWCsiYhRwKnBEeZvTOVaJUn2JXT6wC8P7DW+dE6YEX/kKXHEFnHUW\n/OQnztQmSZIktaGWDsGMBT4GfDelND8idgBuqFxZai+eefUZHl3wKHVD6ojWCGspwdlnZyHw7LMN\ngZIkSVIOWjQimFKaBXwRICLeD/RMKV1aycLUPkyon0BN1HDK4FM2/WSrVmUjgD//eTYi+MMfGgIl\nSZKkHLR01tCpEfG+iNga+CtwbURcXtnSlLemVU1Mmj6Jkf1H0rvnJj4SumpV9izgz3+ePRtoCJQk\nSZJy09JbQ7dMKS0FPg1cn1LaG/j39R0UEYdExJyImBsR56/h8y0j4u6ImB4RMyNibLPPzilveyoi\nbomIzVv6Ral1PDD3ARY3LqZuyCZOErNyZTYr6LXXZrOEXnKJIVCSJEnKUUuDYOeI6A0cD9zTkgMi\nogYYB4wEBgKjImLgarudCcxKKQ0GRgA/ioiuEdGH7FbUYSmlPYAa4MQW1qpWUmoosc0W23DYLodt\n/ElWroS6umx9wIsuypaKMARKkiRJuWppEPw28ADwvymlaRGxI/Dseo75KDA3pTQvpbQcmAwctdo+\nCegZ2SwkPYDXgKbyZ52BbhHRGdgCeKGFtaoVLHlzCVPmTOHkQSfTtabrxp2kqQlOOQWuvx6+/e1s\n0XhDoCRJkpS7lk4Wcztwe7P384Bj1nNYH+D5Zu8XAnuvts9PgSlkIa8ncEJKaRWwKCIuAxYAbwEP\nppQebEmtah03PnkjTauaNn7twKYmGD0abr0Vvvc9uOCC1i1QkiRJ0kaLlNL6d4roC1wF/HMhuUeB\ns1NKC9dxzLHAISmlz5XfnwzsnVI6a7V9hgNfBnYCHgIGk90KeidwAvA6WQi9I6V04xqucxpwGkBt\nbe1ekydPXu/X09YaGxvp0aNH3mW0WEqJusfr6FbTjZ8N/dkGHx9NTez2ne/wwd/9jv89/XSeP7FY\nd/UWrV+yZ0Vkz4rFfhWPPSsee1Y87bVn+++//xMppWHr26+lC8pPAG4Gjiu/H13edtA6jlkE9Gv2\nvm95W3Njge+nLI3OjYj5wK7Ah4H5KaUlABHxS2Bf4D1BMKU0HhgPMGzYsDRixIgWfkltZ+rUqbTH\nutZm2qJpPPf757jm8GsYsdeIDTt4+XI48UT43e/g8svZ6Zxz2KkiVVZO0fole1ZE9qxY7Ffx2LPi\nsWfFU/SetfQZwW1SShNSSk3lPxOBbdZzzDSgf0TsEBFdySZ7mbLaPguAAwEiohYYAMwrb98nIrYo\nPz94IDC7hbVqE11Xfx3dOnfjhN1P2LAD33kHjj0W7rorWyj+nHMqU6AkSZKkTdLSEcFXI2I0cEv5\n/Sjg1XUdkFJqioizyCaZqQFKKaWZEXFG+fOrgYuBiRExAwjgvJTSK8ArEXEH2ZqFTUA95VE/Vday\nFcu45albOHbgsWy5+ZYtP/Dtt+GYY+C++2DcOPjP/6xckZIkSZI2SUuDYB3ZM4JXkM30+UdgzPoO\nSindB9y32rarm71+AfjkWo69CLiohfWplfxy9i9Z+s7SDZsk5q234NOfhvvvh2uugdNOq1yBkiRJ\nkjZZS2cN/RtwZPNtEfEl4MpKFKX8lOpL7Pj+Hdnvw/u17IBly+Doo+E3v4HrrsvWDJQkSZLUrrX0\nGcE1+XKrVaF2Yd7f5/Hb537L2CFj6RQt+Kvx5ptwxBFZCJwwwRAoSZIkFURLbw1dE1cG72AmNkwk\nCMYMGbP+nRsb4fDD4dFHswXjR4+ueH2SJEmSWsemjAiufwFCFcbKVSuZ2DCRg3c+mL7v67v2HVOC\nX/0K9tkH/vAHuOkmQ6AkSZJUMOsMghHxRkQsXcOfN4APtVGNagMPz3+Y55c+T92QtdzemRLccw8M\nGwaf+lS2XuCvf52tGShJkiSpUNZ5a2hKqWdbFaJ8XVd/HVt325ojBxz57g9SymYDvegimDYNdtwR\nJk6Ek06CzptyZ7EkSZKkvGzKraHqIF5d9iq/evpXjP7IaDbrvFm2MSV48EHYd1849FBYsiSbFfTp\np+HUUw2BkiRJUoEZBMXNM25m+crl2dqBKcHDD8O//RscfDAsWpStDThnTjYraJcueZcrSZIkaRM5\nrCNKDSWG9h7K4Dmvwwkj4Pe/hz594Gc/y8LfZpvlXaIkSZKkVmQQrHL1i+vp/ucGJs/YBR4fAb17\nw1VXwec+B5tvnnd5kiRJkirAIFjNHnuMzc88gT/Uw6oPvg5XXAGnnw7duuVdmSRJkqQK8hnBavSX\nv8DIkbDvvvR6ZiE3njKETvPnw5e+ZAiUJEmSqoBBsJr89a9wxBGw994wbRrTz/kM25+d2PaiH8IW\nW+RdnSRJkqQ2YhCsBg0NcPTRsNde8D//A9/9Lsyfz1f3XEKvbbbjgB0OyLtCSZIkSW3IINiRzZgB\nxx4Le+4JU6fCt78Nzz0HX/saf1v5Gr+Z9xvGDhlLp/CvgSRJklRNnCymI5o1C771LbjtNnjf++Ci\ni7Ln/7ba6l+7TJo+iURizJAx+dUpSZIkKRcGwY7k6aezUb/Jk6F7d/j61+Gcc2Drrd+126q0igkN\nEzhwhwPZfqvt86lVkiRJUm68J7AjePZZOOUU2H13mDIFzjsP5s+Hiy9+TwgEmPrcVJ57/Tk+u+dn\ncyhWkiRJUt4cESyyefOysHfDDdC1K3zlK/DVr8I226zzsFJ9ia0234qjdz26jQqVJEmS1J4YBIvo\nuefgO9+BiROhSxf44hezUcDa2vUe+vrbr3Pn7DupG1JHty6uGShJkiRVI4NgkSxYkC39UCpBTQ2c\neSacfz707t3iU0x+ajJvN71N3Z51FSxUkiRJUntmECyChQvhkkvg2mshAk4/HS64APr02eBTXVd/\nHYNqBzG099AKFCpJkiSpCAyC7dnixVkAHD8eVq2Cujr42tdgu+026nRPvvQkj7/wOFcefCUR0crF\nSpIkSSoKg2B79NJLcOml8POfw4oVMHYsXHghbL/9Jp12Qv0EunTqwkmDTmqdOiVJkiQVkkGwPVmy\nBH7wAxg3DpYvz5aE+PrXYccdN/nUy1cu54Ynb+CoXY+i1xa9WqFYSZIkSUVlEGwPXn0VLrsMrroK\n3noLRo+G//ov2HnnVrvE3XPu5tW3XnXtQEmSJEkGwVy99hpcfjn8+Mfw5pswahR84xswYECrX6rU\nUKJPzz4ctONBrX5uSZIkScViEMzD66/DFVfAlVfCG2/A8cdnAXDgwIpcbtHSRdw/934u+PgF1HSq\nqcg1JEmSJBWHQbAt/eMf2ejf5Zdnr489Fi66CPbYo6KXvX769axKqxgzZExFryNJkiSpGAyCbaBm\n2TL43vey5wD//nc4+mj45jdh8OCKXzulRKmhxCc+/Al23rr1njmUJEmSVFydKnnyiDgkIuZExNyI\nOH8Nn28ZEXdHxPSImBkRY5t9tlVE3BERT0fE7Ij4WCVrrYjGRrj0UvYZNSpb/uHjH4cnnoC77mqT\nEAjw6IJHmfvaXOr2rGuT60mSJElq/yoWBCOiBhgHjAQGAqMiYvWH4M4EZqWUBgMjgB9FRNfyZz8G\n7k8p7QoMBmZXqtaKWbwYLryQpbvtBn/5C0yZAkOHtmkJpfoSPbv25JjdjmnT60qSJElqvyp5a+hH\ngbkppXkAETEZOAqY1WyfBPSMiAB6AK8BTRGxJbAfMAYgpbQcWF7BWiujf3945hlmLFjAiP/3/9r8\n8kvfWcrts25n9EdG071r9za/viRJkqT2qZK3hvYBnm/2fmF5W3M/BXYDXgBmAGenlFYBOwBLgAkR\nUR8Rv4iIYiaZVlgMfmPdNvM2lq1Y5m2hkiRJkt4lUkqVOXHEscAhKaXPld+fDOydUjprtX2GA18G\ndgIeIrsNdBfgT8DwlNKfI+LHwNKU0n+t4TqnAacB1NbW7jV58uSKfD2borGxkR49erT5dc+qP4vG\npkYmDJtANuiqlsirX9p49qx47Fmx2K/isWfFY8+Kp732bP/9938ipTRsfftV8tbQRUC/Zu/7lrc1\nNxb4fsrS6NyImA/sCiwAFqaU/lze7w7gPZPNAKSUxgPjAYYNG5ZGjBjRal9Aa5k6dSptXdfsJbOZ\n+buZXHbQZey/7/5teu2iy6Nf2jT2rHjsWbHYr+KxZ8Vjz4qn6D2r5K2h04D+EbFDeQKYE4Epq+2z\nADgQICJqgQHAvJTSi8DzETGgvN+BvPvZQq3HhIYJdO7UmdGDRuddiiRJkqR2pmIjgimlpog4C3gA\nqAFKKaWZEXFG+fOrgYuBiRExAwjgvJTSK+VTfAG4qRwi55GNHqoFVqxcwaTpkzh8l8Op7VGbdzmS\nJEmS2pmKLiifUroPuG+1bVc3e/0C8Mm1HNsArPfeVr3Xfc/ex8tvvkzdECeJkSRJkvReFV1QXvko\nNZTYtse2jOw/Mu9SJEmSJLVDBsEO5sXGF7n3mXs5dfCpdO5U0QFfSZIkSQVlEOxgbph+AyvTSsYO\n8ZFKSZIkSWtmEOxAUkqUGkoM7zecAb0GrP8ASZIkSVXJINiB/Gnhn3j6laep29NJYiRJkiStnUGw\nAynVl+jepTvHDTwu71IkSZIktWMGwQ6icXkjk2dO5vjdj6fnZj3zLkeSJElSO2YQ7CDumHUHjcsb\nvS1UkiRJ0noZBDuIUn2JXT6wC8P7Dc+7FEmSJEntnEGwA3jm1Wd4dMGj1A2pIyLyLkeSJElSO2cQ\n7AAmNkykJmo4ZfApeZciSZIkqQAMggXXtKqJSdMnMbL/SHr37J13OZIkSZIKwCBYcA/+74O88MYL\n1A1xkhhJkiRJLWMQLLhSfYltttiGw3Y5LO9SJEmSJBWEQbDAlry5hClzpnDyoJPpWtM173IkSZIk\nFYRBsMBufPJGVqxa4dqBkiRJkjaIQbCgUkpcV38de/fZm90/uHve5UiSJEkqEINgQT3+wuPMXDLT\n0UBJkiRJG8wgWFCl+hLdOnfjhN1PyLsUSZIkSQVjECygZSuWcfNTN3PswGPZcvMt8y5HkiRJUsEY\nBAvortl3sfSdpd4WKkmSJGmjGAQLqNRQYsf378h+H94v71IkSZIkFZBBsGDm/X0ej8x/hLFDxtIp\nbJ8kSZKkDWeSKJiJDRMJgjFDxuRdiiRJkqSCMggWyMpVK5nYMJGDdz6Yvu/rm3c5kiRJkgrKIFgg\nD89/mOeXPk/dECeJkSRJkrTxDIIFUqovsXW3rTlywJF5lyJJkiSpwAyCBfHaW69x19N3Mfojo9ms\n82Z5lyNJkiSpwAyCBXHzjJtZvnK5awdKkiRJ2mQGwYIo1ZcY2nsog7cdnHcpkiRJkgquokEwIg6J\niDkRMTcizl/D51tGxN0RMT0iZkbE2NU+r4mI+oi4p5J1tnf1i+upf7HeSWIkSZIktYqKBcGIqAHG\nASOBgcCoiBi42m5nArNSSoOBEcCPIqJrs8/PBmZXqsaiKNWX2KxmMz7zkc/kXYokSZKkDqCSI4If\nBeamlOallJYDk4GjVtsnAT0jIoAewGtAE0BE9AUOA35RwRrbvbeb3uamGTfx6d0+zfu7vT/vciRJ\nkiR1AJUMgn2A55u9X1je1txPgd2AF4AZwNkppVXlz64EzgVWUcV+/fSv+fvbf3eSGEmSJEmtpnPO\n1z8YaAAOAHYCHoqIR4H9gJdTSk9ExIh1nSAiTgNOA6itrWXq1KkVLXhjNDY2bnRdlz15GbWb1dLp\nb52YumDjzqENsyn9Uj7sWfHYs2KxX8Vjz4rHnhVP0XtWySC4COjX7H3f8rbmxgLfTyklYG5EzAd2\nBYYDR0bEocDmwPsi4saU0ujVL5JSGg+MBxg2bFgaMWJEq38hm2rq1KlsTF0L/rGAJ373BN/4xDc4\nYMQBrV+Y1mhj+6X82LPisWfFYr+Kx54Vjz0rnqL3rJK3hk4D+kfEDuUJYE4Epqy2zwLgQICIqAUG\nAPNSSheklPqmlLYvH/fImkJgRzepYRKJxJghY/IuRZIkSVIHUrERwZRSU0ScBTwA1ACllNLMiDij\n/PnVwMXAxIiYAQRwXkrplUrVVCSr0iomNEzgwB0OZPutts+7HEmSJEkdSEWfEUwp3Qfct9q2q5u9\nfgH45HrOMRWYWoHy2rWpz01l/uvz+e4B3827FEmSJEkdTEUXlNfGK9WX2GrzrTh616PzLkWSJElS\nB2MQbIdef/t17px9J5/Z4zN069It73IkSZIkdTAGwXZo8lOTebvpbdcOlCRJklQRBsF2qFRfYlDt\nIIb2Hpp3KZIkSZI6IINgOzPjpRlMe2EadUPqiIi8y5EkSZLUARkE25kJDRPo0qkLJw06Ke9SJEmS\nJHVQBsF2ZPnK5dzw5A0ctetR9NqiV97lSJIkSeqgDILtyN1z7uaVZa/w2T0/m3cpkiRJkjowg2A7\nUmoo0adnHw7a8aC8S5EkSZLUgRkE24lFSxdx/9z7GTNkDDWdavIuR5IkSVIHZhBsJ66ffj2r0irG\nDBmTdymSJEmSOjiDYDuQUqLUUOITH/4EO2+9c97lSJIkSergDILtwB8W/IG5r82lbs+6vEuRJEmS\nVAUMgu1AqaFEz649OWa3Y/IuRZIkSVIVMAjm7I133uC2mbcxao9RdO/aPe9yJEmSJFUBg2DObp15\nK8tWLPO2UEmSJEltxiCYs1J9iYHbDOSjfT6adymSJEmSqoRBMEezl8zmsYWPUTekjojIuxxJkiRJ\nVcIgmKMJDRPo3KkzoweNzrsUSZIkSVXEIJiTFStXcP306zl8l8Op7VGbdzmSJEmSqohBMCf/Pfe/\neenNl6gb4iQxkiRJktqWQTAnpfoS2/bYlpH9R+ZdiiRJkqQqYxDMwYuNL3LPM/dw6uBT6dypc97l\nSJIkSaoyBsEc3DD9BlamlYwdMjbvUiRJkiRVIYNgG0spUWooMbzfcAb0GpB3OZIkSZKqkEGwjf1p\n4Z94+pWnqdvTSWIkSZIk5cMg2MZK9SW6d+nOcQOPy7sUSZIkSVXKINiG3lz+JpNnTub43Y+n52Y9\n8y5HkiRJUpUyCLahO2bdQePyRm8LlSRJkpQrg2AbKjWU2OUDuzC83/C8S5EkSZJUxSoaBCPikIiY\nExFzI+L8NXy+ZUTcHRHTI2JmRIwtb+8XEb+NiFnl7WdXss628Oyrz/L7v/2euiF1RETe5UiSJEmq\nYhULghFRA4wDRgIDgVERMXC13c4EZqWUBgMjgB9FRFegCfhKSmkgsA9w5hqOLZQJDROoiRpOGXxK\n3qVIkiRJqnKVHBH8KDA3pTQvpbQcmAwctdo+CegZ2RBZD+A1oCmltDil9FeAlNIbwGygTwVrraiV\naSWTpk9iZP+R9O7ZO+9yJEmSJFW5SClV5sQRxwKHpJQ+V35/MrB3SumsZvv0BKYAuwI9gRNSSveu\ndp7tgd8De6SUlq7hOqcBpwHU1tbuNXny5Ip8PZti6qKpfGvut/j27t/m33r9W97laD0aGxvp0aNH\n3mVoA9iz4rFnxWK/iseeFY89K5722rP999//iZTSsPXt17ktilmHg4EG4ABgJ+ChiHj0n4EvInoA\ndwJfWlMIBEgpjQfGAwwbNiyNGDGiLereIBeNu4htttiG8z51Hl1ruuZdjtZj6tSptMe/R1o7e1Y8\n9qxY7Ffx2LPisWfFU/SeVfLW0EVAv2bv+5a3NTcW+GXKzAXmk40OEhFdyELgTSmlX1awzopa8uYS\n/vjqHzl50MmGQEmSJEntQiWD4DSgf0TsUJ4A5kSy20CbWwAcCBARtcAAYF75mcHrgNkppcsrWGPF\n3TTjJppSk2sHSpIkSWo3KhYEU0pNwFnAA2STvdyWUpoZEWdExBnl3S4G9o2IGcDDwHkppVeA4cDJ\nwAER0VD+c2ilaq2UlBKl+hK79dyN3T+4e97lSJIkSRJQ4WcEU0r3Afettu3qZq9fAD65huP+ABR+\nsb0nX3qSGS/P4Mv9v5x3KZIkSZL0L3lPFtOhDaodxLT/mMZLs17KuxRJkiRJ+pdKPiNY9SKCdMhq\nbAAABwpJREFUYR8aRvfO3fMuRZIkSZL+xSAoSZIkSVXGIChJkiRJVcYgKEmSJElVxiAoSZIkSVXG\nIChJkiRJVcYgKEmSJElVxiAoSZIkSVXGIChJkiRJVcYgKEmSJElVxiAoSZIkSVUmUkp519BqImIJ\n8Le861iDXsAreRehFrNfxWPPiseeFYv9Kh57Vjz2rHjaa88+nFLaZn07dagg2F5FxOMppWF516GW\nsV/FY8+Kx54Vi/0qHntWPPaseIreM28NlSRJkqQqYxCUJEmSpCpjEGwb4/MuQBvEfhWPPSsee1Ys\n9qt47Fnx2LPiKXTPfEZQkiRJkqqMI4KSJEmSVGUMghUUEYdExJyImBsR5+ddj9YtIvpFxG8jYlZE\nzIyIs/OuSesXETURUR8R9+Rdi9YvIraKiDsi4umImB0RH8u7Jq1bRJxT/p74VETcEhGb512T3i0i\nShHxckQ81Wzb1hHxUEQ8W/7v+/OsUe+2lp79sPy98cmIuCsitsqzRv2fNfWr2WdfiYgUEb3yqG1T\nGAQrJCJqgHHASGAgMCoiBuZbldajCfhKSmkgsA9wpj0rhLOB2XkXoRb7MXB/SmlXYDD2rl2LiD7A\nF4FhKaU9gBrgxHyr0hpMBA5Zbdv5wMMppf7Aw+X3aj8m8t6ePQTskVIaBDwDXNDWRWmtJvLefhER\n/YBPAgvauqDWYBCsnI8Cc1NK81JKy4HJwFE516R1SCktTin9tfz6DbJ/oPbJtyqtS0T0BQ4DfpF3\nLVq/iNgS2A+4DiCltDyl9Hq+VakFOgPdIqIzsAXwQs71aDUppd8Dr622+ShgUvn1JODoNi1K67Sm\nnqWUHkwpNZXf/gno2+aFaY3W8v8YwBXAuUAhJ10xCFZOH+D5Zu8XYqgojIjYHtgT+HO+lWg9riT7\nBrwq70LUIjsAS4AJ5dt5fxER3fMuSmuXUloEXEb22+7FwD9SSg/mW5VaqDaltLj8+kWgNs9itMHq\ngP/OuwitXUQcBSxKKU3Pu5aNZRCUVhMRPYA7gS+llJbmXY/WLCIOB15OKT2Rdy1qsc7AUODnKaU9\ngTfxdrV2rfxc2VFkIf5DQPeIGJ1vVdpQKZsivpAjFtUoIi4ke1zlprxr0ZpFxBbA14Bv5F3LpjAI\nVs4ioF+z933L29SORUQXshB4U0rpl3nXo3UaDhwZEc+R3Xp9QETcmG9JWo+FwMKU0j9H2u8gC4Zq\nv/4dmJ9SWpJSWgH8Etg355rUMi9FRG+A8n9fzrketUBEjAEOB05KrvHWnu1E9guy6eV/h/QF/hoR\n2+Za1QYyCFbONKB/ROwQEV3JHq6fknNNWoeICLJnl2anlC7Pux6tW0rpgpRS35TS9mT/fz2SUnKk\noh1LKb0IPB8RA8qbDgRm5ViS1m8BsE9EbFH+HnkgTvBTFFOAU8uvTwV+nWMtaoGIOITscYcjU0rL\n8q5Ha5dSmpFS+mBKafvyv0MWAkPLP+cKwyBYIeWHfc8CHiD7oXlbSmlmvlVpPYYDJ5ONLDWU/xya\nd1FSB/MF4KaIeBIYAnwv53q0DuXR2zuAvwIzyP7dMD7XovQeEXEL8BgwICIWRsRnge8DB0XEs2Qj\nu9/Ps0a921p69lOgJ/BQ+d8gV+dapP5lLf0qvHDUWZIkSZKqiyOCkiRJklRlDIKSJEmSVGUMgpIk\nSZJUZQyCkiRJklRlDIKSJEmSVGUMgpIkARGxstnSMQ0RcX4rnnv7iHiqtc4nSdKm6px3AZIktRNv\npZSG5F2EJEltwRFBSZLWISKei4gfRMSMiPhLROxc3r59RDwSEU9GxMMRsV15e21E3BUR08t/9i2f\nqiYiro2ImRHxYER0K+//xYiYVT7P5Jy+TElSlTEISpKU6bbaraEnNPvsHymljwA/Ba4sb7sKmJRS\nGgTcBPykvP0nwO9SSoOBocDM8vb+wLiU0u7A68Ax5e3nA3uWz3NGpb44SZKai5RS3jVIkpS7iGhM\nKfVYw/bngANSSvMiogvwYkrpAxHxCtA7pbSivH1xSqlXRCwB+qaU3ml2ju2Bh1JK/cvvzwO6pJS+\nExH3A43Ar4BfpZQaK/ylSpLkiKAkSS2Q1vJ6Q7zT7PVK/u85/cOAcWSjh9Miwuf3JUkVZxCUJGn9\nTmj238fKr/8InFh+fRLwaPn1w8DnASKiJiK2XNtJI6IT0C+l9FvgPGBL4D2jkpIktTZ/6yhJUqZb\nRDQ0e39/SumfS0i8PyKeJBvVG1Xe9gVgQkR8FVgCjC1vPxsYHxGfJRv5+zyweC3XrAFuLIfFAH6S\nUnq91b4iSZLWwmcEJUlah/IzgsNSSq/kXYskSa3FW0MlSZIkqco4IihJkiRJVcYRQUmSJEmqMgZB\nSZIkSaoyBkFJkiRJqjIGQUmSJEmqMgZBSZIkSaoyBkFJkiRJqjL/HxuFoa7pA2lgAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x17501978>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(15, 5))\n",
    "plt.plot(fitted_model.history['acc'], 'g', label=\"accuracy on train set\")\n",
    "plt.plot(fitted_model.history['val_acc'], 'r', label=\"accuracy on validation sete\")\n",
    "plt.grid(True)\n",
    "plt.title('Training Accuracy vs. Validation Accuracy - VGG16')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many deep learning pioneers encourage to use pre-trained networks for classification tasks. In fact, this usually leverages the training of a very large network on a very large dataset. So the motto is \n",
    "\n",
    "> Don't be a hero. Don't reinvent the wheel."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This article was an opportunity to go through the theory behind convolutional neural networks and explain each of their principal components in more details.\n",
    "\n",
    "This was also a hands-on guide to setup a deep learning dedicated environment on AWS and develop an end-to-end model from scratch as well as an enhanced model based on a pre-trained one. \n",
    "\n",
    "Using python for deep learning is extermely fun. Keras made it easier for preprocessing the data and building up the layers. \n",
    "Keep in mind that if you want someday to build custom neural net components, you'll have to switch to another framework. \n",
    "\n",
    "I hope this gave you a practical intuition about the convnets mechanisms and a strong appetite to learn more.\n",
    "\n",
    "Convnets are amazingly powerful. Anyone working on computer vision believes in their robustness and efficiency.\n",
    "\n",
    "Their applications are getting a larger scope. NLP practictioners are now the ones who are switching to convnets. Here are some of their applications:\n",
    "\n",
    "- Text classification using CNNs : <a href=\"https://chara.cs.illinois.edu/sites/sp16-cs591txt/files/0226-presentation.pdf\"> link </a>\n",
    "- Automated image captioning (Image + Text): <a href=\"https://cs.stanford.edu/people/karpathy/sfmltalk.pdf\">link </a>\n",
    "- Text classification at **character level**: <a href=\"https://papers.nips.cc/paper/5782-character-level-convolutional-networks-for-text-classification.pdf\"> link </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "Here's a list of some references I used to learn about neural nets and convnets:\n",
    "\n",
    "1. <a href=\"http://neuralnetworksanddeeplearning.com\" style=\"text-decoration:none;\"> neuralnetworksanddeeplearning.com </a>: By far the best notes notes neural networks and deep learning. Recommend this to anyone who want to start learning about neural nets.\n",
    "\n",
    "2. <a style=\"text-decoration:none;\" href=\"http://cs231n.github.io/convolutional-networks/\"> CS231n Convolutional Neural Networks for Visual Recognition </a>: Andrej Karpathy's lectures at Stanford. Great mathematical focus.\n",
    "\n",
    "3. A Beginner Guide to Understanding Neural Networks: A three-part post explaining CNNs, from the basic high level inuition to the architecture details. Very interesting read. Learnt a lot from it.\n",
    "    + <a style=\"text-decoration:none;\" href=\"https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/\"> part 1 </a>\n",
    "    + <a style=\"text-decoration:none;\" href=\"https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/\"> part 2 </a>\n",
    "    + <a style=\"text-decoration:none;\" href=\"https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html\"> part 3 </a>\n",
    "\n",
    "4. <a style=\"text-decoration:none;\" href=\"https://blog.keras.io/running-jupyter-notebooks-on-gpu-on-aws-a-starter-guide.html\"> Running jupyter notebooks on GPU on AWS <a>\n",
    "\n",
    "5. <a style=\"text-decoration:none;\" href=\"https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html\"> Building powerful image classification models using very little data </a>\n",
    "\n",
    "6. <a style=\"text-decoration:none;\" href=\"https://www.kaggle.com/jeffd23/catdognet-keras-convnet-starter\"> CatdogNet - Keras Convnet Starter </a>\n",
    "\n",
    "7. <a style=\"text-decoration:none;\" href=\"https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/\"> A quick introduction to Neural Networks </a>\n",
    "\n",
    "8. <a style=\"text-decoration:none;\" href=\"https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/\"> An Intuitive Explanation of Convolutional Neural Networks</a>\n",
    "\n",
    "9. <a style=\"text-decoration:none;\" href=\"https://hackernoon.com/visualizing-parts-of-convolutional-neural-networks-using-keras-and-cats-5cc01b214e59\"> Visualizing parts of Convolutional Neural Networks using Keras and Cats </a>\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}