21237189_DeepLearning_Assessment1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Learning - Assignment 1 Report\n",
    "\n",
    "#### Submitted By: Kalyani Prashant Kawale\n",
    "#### Student ID: 21237189\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Assignment: Implement Shallow and Deep Neural Network\n",
    "\n",
    "The given assignment has 5 parts, the solutions to each part are presented in following sections,\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 1 Solution: Implement Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Package imports\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn\n",
    "import sklearn.datasets\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import itertools\n",
    "from tabulate import tabulate\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "# Display plots inline and change default figure size\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following section consists of two methods, LogisticRegressor and Predict. The LogisticRegressor implementation is based on material covered in [1],"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def LogisticRegressor(X, y, alpha=0.01, max_iters=1000):\n",
    "    '''\n",
    "        Method to create a model for any data-set based on Logistic Regression, \n",
    "        using Stochastic Gradient Descent optimization.\n",
    "        \n",
    "        X: Input features for fixed number of samples\n",
    "        y: Output labels for each sample in X\n",
    "        alpha: Learning rate to control downhill step size for gradient descent\n",
    "        max_iters: Number of iterations for training weights and biases\n",
    "        \n",
    "    '''\n",
    "    # Getting the size of number of samples and number of features in X\n",
    "    (nsamples, nattributes) = np.shape(X)\n",
    "    # Initialising threshold to small value to check convergence\n",
    "    threshold = 1e-6\n",
    "    # Initialising weights and biases\n",
    "    w = np.random.rand(nattributes)\n",
    "    b = np.random.rand()\n",
    "    # Initialising cost value to 0\n",
    "    J_prev = 0\n",
    "    # Running logistic regression for max_iters iterations\n",
    "    for i in range(max_iters):\n",
    "        # Randomly selecting a sample from X\n",
    "        idx = np.random.choice(nsamples, 1, replace=False)\n",
    "        random_X = X[idx[0]]\n",
    "        random_y = y[idx[0]]\n",
    "        # Calculating prediction for random sample using sigmoid function\n",
    "        y_hat = 1 / (1 + np.exp(-1 * (np.dot(w, random_X) + b)))\n",
    "        # Calculating cost\n",
    "        J_curr = -1 * ((random_y * np.log(y_hat)) + ((1 - random_y) * np.log(1 - y_hat)))\n",
    "        # Checking if convergence reached\n",
    "        if np.absolute(J_curr - J_prev) < threshold:\n",
    "            break\n",
    "        else:\n",
    "            J_prev = J_curr\n",
    "        # Calculating gradients, derivatives of Cost w.r.t weights and bias\n",
    "        delta_w = []\n",
    "        for j in range(len(w)):\n",
    "            delta_w.append((y_hat - random_y) * random_X[j])\n",
    "        delta_b = (y_hat - random_y)\n",
    "        # Updating weights and bias using the gradients\n",
    "        for j in range(len(w)):\n",
    "            w[j] -= alpha * delta_w[j]\n",
    "        b -= alpha * delta_b        \n",
    "    return w, b\n",
    "\n",
    "def predict(data, weights, bias):\n",
    "    '''\n",
    "        Method to predict classes of new samples using LogisticRegressor\n",
    "        \n",
    "        data: Input data with fixed number of samples\n",
    "        weights: Weights obtained using gradient descent ran on training data-set\n",
    "        bias: Bias obtained using gradient descent ran in training data-set\n",
    "    '''\n",
    "    # Initialising predictions\n",
    "    predictions = []\n",
    "    # Calculating predictions for each sample using logistic regression\n",
    "    for sample in data:\n",
    "        prediction = 1 / (1 + np.exp(-1 * (np.dot(weights, sample) + bias))) \n",
    "        if prediction >= 0.5:\n",
    "            predictions.append(1)\n",
    "        else:\n",
    "            predictions.append(0)\n",
    "    return predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2 Solution: Easy Tasks\n",
    "\n",
    "Following section performs training, validation and, testing on blobs300 and circles600 data-sets using LogisticRegressor defined in Part 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating a list of file names for storing,\n",
    "# names of files consisting blobs300 and circles600 data-sets\n",
    "files = [\"blobs300.csv\", \"circles600.csv\"]\n",
    "\n",
    "# Setting a list of different learning rates\n",
    "alpha_s = [0.1, 0.001, 0.0001]\n",
    "\n",
    "# Setting a list of different number of maximum iterations\n",
    "iterations = [1000, 10000, 20000]\n",
    "\n",
    "# Initialising dictionaries to store train, valid and test data-sets for both datasets\n",
    "X_train = {}\n",
    "y_train = {}\n",
    "X_valid = {}\n",
    "y_valid = {}\n",
    "X_test = {}\n",
    "y_test = {}\n",
    "# Initialising dataframe to store results for different parameter configurations\n",
    "results = pd.DataFrame(columns=['File Name', 'Learning Rate', 'Iterations', 'Accuracy'])\n",
    "# Setting random seed to reproduce the same results\n",
    "np.random.seed(200)\n",
    "\n",
    "# Note: Reading of files into X and y is based on codes given in [2]\n",
    "for file in files:\n",
    "    # Using pandas to read the CSV file as a dataframe\n",
    "    df = pd.read_csv(file).sample(frac=1).reset_index(drop=True)\n",
    "    # Extracting labels from column named Class\n",
    "    y = df['Class'].values\n",
    "    # Removing Class column from dataframe\n",
    "    del df['Class']\n",
    "    # Setting X with dataframe values\n",
    "    X = df.values\n",
    "    \n",
    "    # Splitting data into train (70%), validate (15%), and test data (15%),\n",
    "    # the use of train_test_split is based on codes given in [3],[4]\n",
    "    X_train[file], X_validate_test, y_train[file], y_validate_test = train_test_split(X, y, train_size=0.7)\n",
    "    X_valid[file], X_test[file], y_valid[file], y_test[file] = \\\n",
    "        train_test_split(X_validate_test, y_validate_test, test_size=0.5)\n",
    "    \n",
    "    # Training and validating for different hyperparameters\n",
    "    for alpha, iters in itertools.product(alpha_s, iterations):\n",
    "        row = {'File Name': file, 'Learning Rate': alpha, 'Iterations': iters}\n",
    "        # Getting trained weights and bias\n",
    "        w, b = LogisticRegressor(X_train[file], y_train[file], alpha, iters)\n",
    "        # Predicting on validation data using trained weights and bias\n",
    "        predictions = predict(X_valid[file], w, b)\n",
    "        # Using Scikit Learn accuracy_score method to calculate prediction accuracy\n",
    "        accuracy = accuracy_score(y_valid[file], predictions)\n",
    "        # Setting results for current pair of hyperparameters\n",
    "        row['Accuracy'] = accuracy\n",
    "        results = results.append(row, ignore_index=True)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training and Validation Results:\n",
      "\n",
      "|    | File Name      |   Learning Rate |   Iterations |   Accuracy |\n",
      "|---:|:---------------|----------------:|-------------:|-----------:|\n",
      "|  0 | blobs300.csv   |          0.1    |         1000 |   1        |\n",
      "|  1 | blobs300.csv   |          0.1    |        10000 |   1        |\n",
      "|  2 | blobs300.csv   |          0.1    |        20000 |   1        |\n",
      "|  3 | blobs300.csv   |          0.001  |         1000 |   0.933333 |\n",
      "|  4 | blobs300.csv   |          0.001  |        10000 |   1        |\n",
      "|  5 | blobs300.csv   |          0.001  |        20000 |   0.977778 |\n",
      "|  6 | blobs300.csv   |          0.0001 |         1000 |   0.577778 |\n",
      "|  7 | blobs300.csv   |          0.0001 |        10000 |   0.577778 |\n",
      "|  8 | blobs300.csv   |          0.0001 |        20000 |   0.911111 |\n",
      "|  9 | circles600.csv |          0.1    |         1000 |   0.655556 |\n",
      "| 10 | circles600.csv |          0.1    |        10000 |   0.477778 |\n",
      "| 11 | circles600.csv |          0.1    |        20000 |   0.677778 |\n",
      "| 12 | circles600.csv |          0.001  |         1000 |   0.522222 |\n",
      "| 13 | circles600.csv |          0.001  |        10000 |   0.588889 |\n",
      "| 14 | circles600.csv |          0.001  |        20000 |   0.333333 |\n",
      "| 15 | circles600.csv |          0.0001 |         1000 |   0.7      |\n",
      "| 16 | circles600.csv |          0.0001 |        10000 |   0.711111 |\n",
      "| 17 | circles600.csv |          0.0001 |        20000 |   0.522222 |\n"
     ]
    }
   ],
   "source": [
    "# Printing results for different hyperparameters\n",
    "print(\"Training and Validation Results:\\n\")\n",
    "print(results.to_markdown())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test data-set using alpha 0.1 and 1000 iterations:\n",
      "100.0%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAEICAYAAABCnX+uAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd5xU1dnA8d8zdWeXpTdpYm8Y24pYY40NNbHEHqNGk6iJLRqNppjYEhV9o0ZFzau+KvaCGEWNWBBUQLEgFhRQEASEXbZPe94/7l0Ydmdhd+fO3p2d5/v5zOeze+/MOc/cvfvMmXPPPUdUFWOMMYUr4HcAxhhjcmOJ3BhjCpwlcmOMKXCWyI0xpsBZIjfGmAJnidwYYwqcJfIuTESiIlIjIkP8jsUrIrJURPZqx/NHiciH7nE424P6rxeRe3KNT0QOEZF5ucbTnYjI2yJyit9xFCNL5B3gJpWmR1pE6jN+PzmHctf5R1DVRlXtoarfehP5OnVdLyIJN+ZKEZkqIhVe19OGGFpNqq7Lgefd4zC+M+LKJxHZUUTeE5FVIrJSRCaLyJYZ+wMicrO7f4WIXN3s9buKyGwRqRORd0VkVOe/i9y19wO9q9fjN0vkHeAmlR6q2gP4GjgiY9tDfsfXDve772EA8DbwqM/xZLMxMKcjLxSRkMexeOFr4CdAX2Ag8ArwYMb+3wAHAdsCOwPHi8jPAUQkBjwLjAf6AI8DT3fR92k6kSXyPBCRoIj8UUS+cltVD4lIb3dfmYg84rbGKkXkHRHpIyI3AbsC97it5JtEpEREVESGua99RERucVtx1SLylohsnFHv4SLyhVvuLW39qquqCeBhYFMRKc8o7ydut0aliLwpIttm7PujiCwRkdUiMldE9s6I8cqM52XtghCRHwMXAae57/fdLM+ZBuyecUxGiEhfEXlYRJaLyHwRuVRExH3+r0TkVRG5XURWAZe18pbLRORJ9xjOEJHtsj1JRGJuWUtEZJGI3CAi4WbPucr9W34lIsdlbD9KRD516/hGRH7rHuuVqrpQ195SnQI2zyjyNOAfqrpEVb8GbgF+7u47CGhQ1X+paiNwE1AOZG1xishIEXnWPQdXuOcYIrK1+w2syj2OD7jb78vyDWCyiJzTSvmZ59u4Zvu2FpHX3GOzXETubzq3RORxnA+xl9y/629FJOT+Tb5zy5siIltt6Hi6+7Kep9nqyfY+ugVVtUcOD2ABcGCzbZcBbwJDgBLgPuB/3X3nA08AMSCEk7zL3H1vA6dklFMCKDDM/f0RYBlOSy3slnOfu28wUAOMdfddCiQyy2sW4/XAPe7PUeBmYAkg7rYx7u+7AEHgbOBzN+YdgK+AQYAAmwKbZMR4ZUY9hwDzMn5fCuzVPIb1HN/mx+QxnJZoD5wEOB842d33KyAJnOXGHGvlfceBI93jdCXwGRDMEt8/3L9jf/e9zgCuyHhfSeA6IAIcCNRlHIfvgdHuz/2AnTJiiAKVOEk8BVySsa8B2CHj972A5e7PlwNPN3s/rwDnZnmfYWCu+35Lcc63Pdx9TwO/c/92MWBPd/uPmv2tBgL1QP8s5Q8GajOO4+Xu8TjF3b81sL97bAa7f8frs50H7u8hnA+xHjjn/R3A2xn7sx5P1nOeZqunuz6sRZ4fvwQuU9VvVbUBuArnK7LgJNcBwGaqmlTVGapa246yH1PV93RtK3pHd/uRwAxVneTuuxFYtYGyThWRSpwEdDJwrLpnv/seblPVWaqaUqd/OorzD5PESQDb4iTAr1R1fjveQ4eISBQ4Bvi9qtao6jycFuupGU/7SlXvdmOub6Woaao60T1O1+Mk6p2zPO9k4M+qukJVvwOublZXErhKVeOq+gpOUj02Y992IlKuqt+r6vtNL1Ln2kdvoDdwIfCB+/7COMe4KqOOKpxWNzhJLnNf8/2Z9gJ6An9Q1TpVrVfVae6+BDASGOxuf8vd/l+gh4iMdn8/HpiiqiuylN90vjUdx38AKzPe46eq+qp7bJbi/J1+mKWcpucnVfV+9+/a9D8zWkRK3Ke0djzXd54WDUvkHnOT9XDgP+5XvUrgfZxj3Q+4F3gdeML9un6tiATbUcXSjJ/rcP65wWn9f9O0Q1XTwOINlPV/bkLZCPiStR8K4PRN/6HpPbjvYwAwVFXn4HzruAZYJk7X0aB2vIeOGoxzHL/O2LYQGJrx+zdsWOZxSgLf4hy/Ndy/42C3/NbqWu4mncz9TeX8GOdD52u3u6fFhWRVrQbuBB4VkT5uQmzEScBNegLV7s81zfY1359pODDfPQ+auxCnlf6+2yVxihtPCucbz4nu804CWrvm0/x8S5FxvonIEBF5XEQWi8hq4B6cD8ys3K6VG90uqtXApzjfGPq5T2nteLZ6nrZWV3dkidxjbot2MbC/qvbOeJS4LbtGVf2Tqm4N7AMcB5zQ9PIcql4CDGv6RUQCtPFkVtVlOC2b60Sk6Z/tG+BPzd5Dqao+5b7mflXdA6dbpQSntQrO1+3SjOIHr6/qtsSXYSmQBkZkbBvBuh9YbSlzeNMP7ofoEJxkvrYQ5++4FCdRtFZX/4wWY9P+b93XT1fVsThdMi8BE1qJJYDTom46Tp/gdF012YG1F3vnZO5z/8ajyH4x+BtgpPucdajqYlU9A+cD/LfAv0Wk6ZhOwPn2uDmwPfBMK3EvYd3j2Px8uwHnXBilqj2BX+Ak5jVhNCvvdJyunf2AXjhdMzS9Zj3Hc73naZZ6uiVL5PlxJ3C9iAwHEJGBInKE+/OBIrKte+KvxvnKmHJf9x1OYuyIicBuInKYOKMYLsIZ2dAmqvoh8AZwsbtpPPAbEakQRw8ROVJESt34f+h2ddS7j6b3MBsYKyK9RWQoziiM1nwHbOK2ftsSYyNO/+614lw03gznmsOD639lC3uIyFi3K+NSnP7X97I8bwLwZxHpJyIDgSua1RUG/igiERHZH+di5JNubCeISE+cboxq3OMjIoeKyA/EuSDeC6fL4VvgC7fMB4BLRGSwe/5cgHONBeBlICbORd0oTsu6FpiaJfapbr1/c/9mMRHZw43heBEZ4n5YVbrPT4KTMHH66e8AnltPt99EYNeM43gJzkicJuU43yBWux8SFzV7ffNzvdyt93ugjLUNA9Z3PFnPedpKPd1TZ3TEd+cH2S92BoHf4/xzVgPzcPpawbmg8wXOP+BSnJEHAXffD93nrsLpc8x2sXN9FxKPdF9fiZMg3gOOayXuFhca3fpXA30zypuF0w/7rVt/DKf/cab73lbitNoGuq8pA55yy3kf56Jaaxc7BwPT3fc7rZU4m1/s7OfGsQKnK+Ny1l6g/RXwygb+XtfjJOincBLNTGD7VuIrxUloS933Pw6IZB57nL7cle55cHzGMXjJfV+rgXeA3dx9J+NcjKvFuXA9Edg2o/4AzoXnVThJ7Zpm8e+K82FZj3PxdVTGvqvIuBgKbAJMcuNbDtzobr8Fp0Vdg3Mu/rxZHdfgnHdHbOBYHpFxvo3L/FvhdNPNduuYhfP/kHkeHIfTmq4EzsNphT/vPn8+zkgdxfmW2erxXN95mq0ev/NFvh5N/wCmm3Fb5Utx/hmn+x2PMSZ/rGulG3G/tvdy+23/jHMxdJbPYRlj8swSefeyD87X0mXAAcBPVDXub0jGmHyzrhVjjClwnszRICILWHslOamqnTr5kjHGFDMvJ9vZT7PfAdZC//79deTIkR5WbYwx3d+sWbNWqOqA5tt9mTVt5MiRzJw504+qjTGmYInIwmzbvbrYqTgzjM0SDyb/N8YY03Zetcj3VNVv3bvfXhaRT1X1jcwnuAn+bIARI0ZkK8MYY0wHeNIiV3cFG3Xm7HgaGJ3lOeNVtUJVKwYMaNHFY4wxpoNyTuTuPAhNE8aX4Ux883Gu5RpjjGkbL7pWBuEsN9VU3sOq+qIH5RpjjGmDnBO5qn7FutNumiLy5QcLmHD90yz8+Bu2rNiMEy77McO3KqqpoI3xnS3aajrsg9fmcMXY64g3xNG08vWni3njibe5+Y2/svlOm/gdnjFFw+ZaMR32z3PvprGuEU070zykU2kaahu446L7/A3MmCJjidx0SCKe4JvPvs2679N3vsi63RiTH5bITYcEQ0GisUjWfT369Mi63RiTH5bITYcEAgEOP/ugFsk8WhrlmAsO9ykqY4qTJXLTYWdedxJ7HzuGcDRMac8YkZIwh565P8defITfoRlTVHyZj7yiokJt0qzuo3J5Fd8tWM6QzQdTbt0qxuSNiMzKNk24DT80Oes9oBe9B/TyOwxjipZ1rRhjTIGzRG6MMQXOErkxxhQ4S+TGGFPgLJEbY0yBs0RujDEFzoYfGmOyWjDnG96eNItINMzex45hwLB+fodkWmGJ3BjTwt2XPcizt75AMpEiEAxw7x8e4qJ7fs0BJ+3td2gmC+taMcasY+47X/DsbS/SWB8nlUyRaEwQb0gw7hd3snpltd/hmSwskRtj1jFlwlTiDfEW24OhAO88/54PEZkNsURujPGEqvL4uIkcO+hMfhT8KWeOupCZL33gd1hFwRK5MWYd+56wJ5GSlnPNp5Ipdjts51Zf98BVj/HAnx6javlqVJWvP1nEX47+Bx9PnZvPcA2WyI0xzWw7ZkuOOvdgIrEIwVCQSEmYSEmYi+75NT37lWd9TbwhzhM3PUdDXeM62xvr4tz3p0c7I+yiZqNWjDEtnPX3UznoZ/vy9nMziZRE2Oe4MfQf2vrww8plVbQ2I/bXcxflKUrTxBK5MQVu4dxFTLrzJb5fvJJdD92J0vIYk++fQmNdnP1P3Isf/XxfwpFwu8sdud1wRm43vE3P7T2oNyLZ943Ydli76zbtY4ncmAL25pNv8/fTbiUZT5JKppk2cQbplNK0YMznM77klQff4MZX/0IwFMxbHJFomGMvPpLHb5xIY0b3SjQW4edXHZ+3eo3D+siNKVCJeIKbfnEHjXVxUsk0AKlkmsxVvxrqGvly9gKmP5f/Fbl+9ufjOP1vJ9B7YE8kIGy83TCueuZSRu21TU7lfvvlUt54Yjqfz/oSP1Y0KwTWIjfGB8lEkukTZzL3nc8ZPHIQ+5+0Fz16l7WrjC9nL0DTG05s9TUNvPvC++z1k906Gm6biAjHXDiWYy4c60l5qWSK60/9J9OenUEoEiKdSjN866FcP/lKevbNftG1WHmWyEUkCMwEFquqN39JY7qh2tV1XLDXlXy3YDn1NQ1ES6P8+4qHGff6X9n0Bxu3uZxYjxLSqfQGnxcKB+kzqPCW4nv8pueY/txM4g3OnaUA8z9cyE1n3sFVT1/qc3Rdi5ddK+cDNmDUmA14+JqnWPzFEuprGgBorGuktqqO607+n3aVM2KbYQwY0R9p7SqjKxgKcsjp+3c4Xr88d+dkGuvWvcM0mUjx7gvvU1/b4FNUXZMniVxEhgGHA/d4UZ4x3dmUR94i0ZhssX3xvKWs+q6yzeWICH+b+Hv6D+tHrDxGaXmMUCRErEcJsR4llPZ0tl3+0PlstOkgL99Cp2ioaWx1X8JtoRuHV10rtwCXAq12XInI2cDZACNGjPCoWmO6nlQyxfPjX+H58S+TTCQ54OS9+cn5hxMrKwEgEGylBa1KINi+ttXQzTfiwfm389Ebc6lcVsW2e2xFvyF9+GLWV8QbEmw1enMi0fYPPewKRh+2E68+PLVF99FGmw5q9cakYpVzIheRscAyVZ0lIvu29jxVHQ+MB6ioqLBLz6bb+tvx45g5+YM1w/Aeuvop3nzyHW575zq3m2M/Jlz39Jp+XwAJCJvuOJJe/Xu2u75AIMAO+263zratdt08tzfhsXhDnM9nfUVJWZTNdhi5we4ggDOvPYlZL31A3ep6GuvjhCIhQuEQv7v3150QcWHxokW+J3CkiBwGlAA9ReRBVT3Fg7KNKSjz3p/PzMmz1+nbjTfEWfzFEqZNnMneR+/GTy85ivdf/ZjPZ35JMpEiHA0RKyvhiocv8DHy/Hn9sWmMO+tOEEin0/QZ1JtrJl3O8K2Grvd1/Yf2495PbuGFe/7LR1PnMmLrYRx5zsEM2nhAJ0VeOMTLcZlui/x3Gxq1UlFRoTNn5n9cqzGd7dnbX2T8Jf+XdRrYo847lPP+eQbgzBQ4561P+WzGlwwc0Z8xR+zSobsvu7oFc77hvNGX0Vi/9niIQN8hfXlowb8IBvN3k1J3JCKzVLWi+XYbR26Mh/oN6UMwHIBmgyoiJWEGjlg7V4mIMGqvbXK+Waare378yyTi617YVYW61fV88Non7HzA9j5F1r14emenqr5mY8hNMdvt8J2JxqIt5h0JhIIcdOoP/QnKRyuXVrYy1l1ZvWJ1p8fTXdkt+sZ4KBwJM+71qxix7XAisQjR0igDhvfj+hevpM+g3n6H1+l2O2xnSsqiLbanEilG7bW1DxF1T9a1YozHhm81lHs+GseS+d+RSqQYusVGbRql0d1ULq9CVek9sBcrl64iXu+M0ikpi3LUuYesd1pc0z6WyI3Jk402KbybcLwy8V8vctfvHnBmXBRIp5QR2wxl4IgBHHnOwYwZu4vfIXYrlshN0Ugmkrzy4Ju8+vCbRGMRDj/7IHY7fOeibC3n0/yPv+auS/7PHSe/dqz8sq9XcOvb11FaHvMvuG7KErkpCqlUissOvprPZsyjoda5UWf2lI857BcH8OubT/c5uu7l5QdeIxlvOQWBBIR3nn+P/U7Y04eouje72GmKwjuT3uPzmV+uSeIADbWNTLrrZZZ89Z2PkXU/DbWNWUeqaFrXGU9uvGOJ3BSFd194b81sg5kCwQCzp3zsQ0T+W72ymvdf/YivP13sWZmpVIp4YwKy9FalkmkqDt7Bs7rMWta1YopCz/49CYWDJBOpdbYHAgHK+/bwKSp/qCr3/flRnrhxIuFomGQiySajRnD1pMs7NNdLplvPvYcpE96CZjeMR0rC/OwvP6X/kL45lW+ysxa5KQqHnL5f1jUrg6EAow/dyYeIWpdOp5kz7TNmvvQB9TX1npf/xuPTeermScQbEtRW1dFYF+eL9+dz9fE3t4jj3Rfe57bf3sv9f3l0g11QlcureOmB14k37z4RGH3Yzhx/6Y+9fivGZS1yUxSGbDaY3z/wG244/XYkIKAQLYty9XOXESmJ+B3eGvM/Wsjlh15LXXUdIkIqmea8W8/wdGGIx8c9t861AnBu0Jkz7TNWLl1F38F9SKVS/PGIv/PR1Lk01DQQCgd5/IaJXHr/eexz7O5Zy1302bdEouGWc4UrLJ2/zLP4TUuWyE3R2PuYMex2+M58Mv1zIiVhthq9eZeatCmVTHHpQX+jclnVOttvO+9etth5UzbbYaQn9VR/X511eygcpHpVLX0H9+H1R6fx0ZufrEn4yUSKZCLFDaffzujDdqaktOXdmoM3HbTO1LxNAsEAm2xvaxDkk3WtmKISKYmw436j2Hb3rbpUEgd4/9WPW3ZLAInGBP+5+xXP6tnt8F0IhVu+91AkxLAtNgLgvw9PbdFqBycpfzz106zl9h/Slz2OqiASW/cbTjga5qeXHOVB5KY1lsiN6SJqK2ubXyMEIJ1WKpd7N8HUiZf/hPJ+5URKnGlzJSBEYxHOv+PsNdcRmvZlE460/kX+0vt/w2Fn7k80FkECwsbbDee6F65g5HbDPYvftGRdK8Z0Edvvs03WG2lKyqLsedSuntXTZ1Bv7vloHM/c9gLvvfIhgzYewNEXjGWris3WPOewXxzAzMmzW7TKQ+Hgeie7ikTDnPvPM/n1LaeTTKQKdpm5QuPpwhJtZQtLGJPdg397nEf//iwN7jJxJWVRNtl+Y8a9fhWhcOe1u1SVu353P8/d8RIiQiAUICABrn3xCrYds2WnxWHW1drCEpbIjeliZk/5mOfumExtVR37Hr8n+5+8t28t28XzlvD+fz+mR+9SxhxRkfUip+k8lsiNMabA2VJvxpguadWyKqY/O4NUMsVuY3dh4PD+fodUcCyRG2N88+qEN7npzDsIBAKoKndefD9nXHsSx1xgK0a2hw0/NMa0S+XyKmZMns38jxbmVM6qZVXcdOYdxBsSNNQ10lgfJ96Q4N9XTGDh3EUeRVscrEVujGkTVeWeyx7kmVtfcCfbSjFim6Fc+58/0HtAr3aXN/3ZGQQCLduSqUSS1x+dxs/+8lMvwi4K1iI3xrTJlAlTmXj75IzJthr56sOFLSbbaqtUMkW2wRbptJJMtBxPb1pnidwY0yZP3jJpzfj2JqlEik+mf87KpavaXd5uY3fJmsgjJWH2PmZMh+MsRpbIjTFtUr2yNuv2YChAbVVdu8sbOLw/Z1x7EpFYhGAo4EwVUBrhiF8fzBY7b5pruEXF+siNMW0yZuwuPHfH5BaLc0RjEYZsPrhDZR5zwVgqDt6R1x+dRjKRZO9jxlgS7wBL5MaYNjnxD0fzxhPTqV5ZQ7whQSAghEvCXDj+VznNJLnxNsPswmaOck7kIlICvAFE3fKeUNU/51quMaZr6TOwF3d/NI7n7pjMrJc/ZKNNBvKT8w/3bJ5003E536IvIgKUqWqNiISBqcD5qvp2a6+xW/SNMab98naLvjqfBDXur2H30fkTuBhjTJHyZNSKiARFZDawDHhZVd/J8pyzRWSmiMxcvny5F9UaY4zBo0SuqilV3REYBowWkVFZnjNeVStUtWLAgAFeVGuMMQaPx5GraiXwGnCIl+UaY4xpXc6JXEQGiEhv9+cYcCCQfXVWY4wxnvNiHPlGwP0iEsT5YHhMVSd5UK4xxnQL8z9ayPTnZhEKB9nnuN0ZPHKgp+V7MWrlQ2AnD2Ixxphu5+7LHuTZW18gEU8SCAa4/y+Pcd6tZ3DoGQd4Vofd2WmMMR6pr6nnnsse4r8PvUkykWLb3bdkzlufEW+IA5BOpQG47bx7GTO2gj4D2z/9bzY2aZYxxnhAVbnkwL/ywr3/XTPN7/uvfrQmiWcKBAO8M2mWZ3VbIjfGGA/MeetTFs75hkRjxlzq67s1UsSzui2RG1OkVny7kqtPuJkjyk/hx31O49bf3Et9Tb3fYRWs+R9/g6bbdlN7OpVm9yN28axu6yM3pgjV1zZw3ujLWPVd1Zp+2xfueYXPZ37JP6ddg3jYWiwWw7bciECoZds4GA6iqgQDASQYAFUuvPtX9Orf07O6LZEbU4RefehNaqvq1iRxgERjkgUff82caZ8xas+tfYyuMO2w73YMHN6fxV8sWTNnuwSEsp4xrnvxSj547RPCkRB7HbMb/Yf09bRuS+TGFKEv3ptPQ21ji+2aVuZ/9LUl8g4IBAKMe/2v/M85d/PWM++iaWX7vbfhgrt+ybAtNmLLXTbLW92WyI0pQiNHDSdaGqWx2RqcgWCAoVts5FNUha9nv3L++OhFpNNpNK0EQx1fcKM97GKnMUXowFP2IRILr9MXHgoHGTiiPzvut52PkXUPgUCg05I4WCI3pij16F3GP6ddy/b7bOMknXCQ3Y/alZteu4pAwNJCobGuFWOK1LAtNuKmKVeRTCSRgOS07qbxlyVyY4pcKGxpoNDZdyhjjClwlsiNMabAWSI3xpgCZ4ncGJM38cYEqWTK7zC6PbvKYYzx3II53zDurDv5bMY8AgFhjx+P5vw7zqJn33K/Q+uWLJEbYzxVubyKC/a+krqqOlQhnYJpz85g0effcud7N9iEXHlgXSvGGE+9+O8pJBoSaMaMrsl4km+//I450z7zL7BuzBK5McZTCz7+mnhDIuu+xV8s6eRoioMlcmOMp7YavTnR0miL7ZpWNv3Bxj5E1P1ZIjfGeOpHp+1LaXkJgeDa9BIpCbP1bpuzxc6b+hhZ92WJ3BjjqbKepdw+4+/sfcwYSnqUUN63B0eddyjXTLrc79C6LRu1Yozx3IBh/bjykQv9DqNoWIvcGGMKXM6JXESGi8gUEZkrInNE5HwvAjPGGNM2XnStJIGLVfU9ESkHZonIy6r6iQdlG2OM2YCcW+SqukRV33N/rgbmAkNzLdcYY0zbeNpHLiIjgZ2Ad7LsO1tEZorIzOXLl3tZrTHGFDXPErmI9ACeBC5Q1dXN96vqeFWtUNWKAQMGeFWtMcYUPU8SuYiEcZL4Q6r6lBdlGmOMaRsvRq0IcC8wV1XH5R6SMcaY9vCiRb4ncCqwv4jMdh+HeVCuMcaYNsh5+KGqTgVsgmFjjPGJ3dlpjDEFzhK5McYUOEvkxhhT4CyRG2NMgbNEbowxBc4SuTHGFDhL5MYYU+AskRtjTIGzRG6MMQXOErkxxhQ4S+TGGFPgLJEbY0yBs0RujDEFzhK5McYUuJynsTWmO9DkPGicClIGJT9CAr38DsmYNrNEboqaqqLVV0Pd40AaJAjVV0PvfyHRPf0Oz5g2sa4VU9ziU6H+CaABiIPWg9ajlb9BtdHv6IxpE0vkpqhp/VNO8s4m/k7nBmNMB1kiN8VNUx3bZ0wXYoncFDWJHQlSmmVPCqJjOj0eYzrCLnYWKE18CullENoOCfbLf33p1dAwCU1+g0R2hOj+iITzXm/eRfd3Hg3/xeknDwMB6HkDIjGfgzOmbSyRFxhNfY+uOgOSC0BCoHG09BSk/FJEJD91Jj5DV54MmgDq0fpSCA6Fvo8igR55qbOziASg101Q+h7a+DpIORIbiwQ32uBrVRNA0hK+8Z0l8gKjlb+F5BdAEtTdWPcwhLeD2Nj81Fl1MejqjA11kFyI1v4LKb+04+VqCgh48gGkmoLGV9CGF0BiSOynSGSnNr1WRCCyCxLZpW11pWvR6r9C/fNACg1tivS8us31GeM16yMvIJpaBokPgGSzPfVo3f/mqc7vndZ/C3Gon9SxMhunkl5+MPrdNuiyCtI1t6Ga7niMmkZX/RKt+j00/Afqn0JXnka6ZnyHy1xvfZXnuUk8DqQg+QW66udocmFe6jNmQyyRFxKtdrpTsklX5adOEdY2/ZsLtrs4jb+PrjoHUvPdDdVQczda/fcOh0jjFIjPdL4pOIUCDVDzTzS1vOPlZpFufB3ib+Mk8QyaQGvv87QuY9rKEnkhCY4Eoll2hCF6QF6qlEBfCG9Ly1MlCrGj212e1tyKc1ExUz3UTUDTtR2KURtfAuqy7AlBfHqHyswmXXsvrDoHyDYsMel2eRnT+U3SZLsAABCLSURBVDxJ5CLybxFZJiIfe1GeyU4kCD2vBkpY+6eLQqAv0uOX+au3100Q6OfMQ0LYGa4X/gHS4+z2F5b8spVKAs4onA4FWE7WU1mklaGF7afJRVB9C5Bo5RkRsD5y4xOvLnbeB9wGPOBReaYVgdhBaOgxtO5+SC6C6F5I6Ql5neRJQiNgwBSnCyO1GMI/gPAuHbtIGdoK4ktp0V2jaQgO7lh8sWPRusdo2dIPQHTvDpXZQuOr64sApAQpPdWbuoxpJ08Suaq+ISIjvSjLbJiEt0Z6Xde5dUoESg7OvZzy36Lfv806SVdiUHpah4fxSXhrtPxyqL424xpCEOlzNyLZuqI6IgC08sEV3ATpcycSHOhRXca0T6cNPxSRs4GzAUaMGNFZ1ZouRsKjoO+96OrrIPkpBPpA2VlI6Wk5lRsoOxGNHeZciJQSiOzufPh4peQgyHpBNor0uQMJjfSuLmPaSVRbG5HQzoKcFvkkVR21oedWVFTozJkzPanXmM6SrnsSVv+FtS1zhfLfESjL7UPImLYSkVmqWtF8u90Q1EGqCWfIG3EIVyCBMr9DMnkWKD0Gje4Nja84E2qVHIAEh/gdljGWyDtC47PRVWfjjGAQ0BTa62oCsSP8Ds3kmQQHQulJfodhzDq8Gn44AZgObCUii0TkTC/K7YpU6525TrQStBa0BqiHqivQrHdAGmNMfnk1auVEL8pZbx3pSrT2f50hcIE+SNnpSHTffFfbUuPrZL/TMYnWP4WUX9TZERljilxBdK1ougpdcRSkv6fp1mhNzEbLziXQkZtScpGudsY8t5DM323yxhizHgVxi77WPQTplawzv4XWQ82taLq6c4OJ7A5kSeRSipTs17mxGGMMBZLIne6MLAvhShgSczo1FAkNg9KfARk3r0gMwrtAZJ9OjaUjNPU9Wv8c2jAZbW2tyi5INY7WP4/W3IU2vpnTbInGdDcF0bVCYDDO2N3mt3WnINi/88PpeQka3QOtfxy0Hik5AkoOcRYp6MLStQ9A9Q1A0J3VEOh9B9LFlzTT5DfoyhOc2Q21ASTqTCDW9yEb9mkMBZLIpeznaOMU1p1LIwihTZDQ5v7EFN0Tie7pS90doYm5UH0ja77ZuJ+JWvkrGDANCXgzuVQ+aNXv3esjbitc6yA5D625Fel5ma+xGdMVdO0mpEsiO0HPPzuz70kPoMRZq7JPfhYO6I60/ilazKENgED8jc4Op800XQOJ2bS8LhGHhomdF0fj66RXnkl6xTGka+5y4jKmiyiIFjm4d9XFxkLyc5Bezox8pu20nqwXaVF3X47Fp6ucD4vkZxAahcR+3AnreXozvcSGpGv+BbV3rT1ONZ+j9U9Cv6eta8d0CQXRIm8iEkXC21sS7wAp+VH2ubk1BZG9cipbkwvQ5QdC9c1Q/xRU34CuOAhNfZtTuYDzYRAeRctTNQIlh+dc/oZoehXU3NHsw64RUkvR+ifyXr8xbVFQidzkILK381iTzANACZRfgAQH5FS0rv6juzhz0zWMekivQldfk1O5TaTXP0B6AW7sUgbBEUiP8z0pf70SHzijo1pogMbX8l+/MW1QMF0rJjciAr3/B+Jvog2TgRhSejQS3i6nclVTEJ9By26ONDR60/cuoZHOwhYNL6KpRUh4G4juhzRbv1S10Rmqmq6CyG7efHML9CV7l1QAAjb/uOkaLJEXEZEAGhzprGQT2qJFIuxgqTiLMGe7SSpbS7aDtQRKofTo1pZ2QBOfoCtPA5LunbdptPR4pPyKjq1k1CS0vZOwU1+z7nuMIGWndLxcYzxkXStFQpNfk15+OLriCHTlieiyPdCGKTmXKxKAkkOA5kk7AiVH5Vx+W6imndkotcqZyIx6oBHqH9/AEm0bJiJI3/+F4GZAzBk1JaXQ8y9IeHsvwjcmZ9YiLwKqKXTlqZD+Dki7vSB1aOX50H9izqvbSM8/ocl5kFoITQuVhLdCyn+XY+RtlPjITeDNaD1a9yhSckBOxUtwKDLgeec9pldDeDsPl5AzJneWyItB/F33YmTz7o+kk+h6/j6n4iXQC/o9A4lZkJwPoS0h/IPcujTapZFW19P0cBoCv24+M2ZDLJEXg/T3a1vK60hCeoknVYgIRCqcR2cL70DWMeUSQ2JHdno4xnQ26yMvBuEdgVTL7RJDInt3ejheE4lCz78DJaxpm0gphEZBrHP66Y3xk7XIi4CEhqGxY6D+aZwLgQBRCAyF2Fg/Q/NMIPYjNDzJueMyvRKJ7gPR/REJ+h2aMXlnibxISM8/Q6QCrXvQuTBYcihS+rNuddFOQiOQ8gv9DsOYTmeJvEiICMTGIt2kBW5ANQnxtyC1DMI7IOEt/Q7J+MQSuTEFyJmj/WTQame+HECj+yK9b7bupCJkFztN3mhqMelVvya9dBTp73YiXXUVmq7zO6xuQSt/C+ll7vj5BufR+Dpa94jfoRkfWIs8TzRdidY/A8kFznzqJYd0q/7oDdF0NbriGNBKnJuQ4lD/OJqcC30ndOIY8+5HU0shOY+W9wXUQ/0EKDvZj7CMjyyR54EmPnW/9iaABrThGai5Ffo9gQR6+x1ep9D6p7LMgR6HxKeQ+BAiO/gVWuHTOK3fANWQfbvp1qxrJQ+06lKn77JpWletg9QStPp/fI2rUyXmsHaoYyZ1W5Omw4LDIdAny44IlNjF7GJkidxjmq6E5JdZ9iSg8cVOj8c3oa1wbtBpRgRCm+SlSk18RLryAtIrjiNdfSOaWpGXevwmIkjvm9y55SPuxlIIDkfKfuFrbMYf1rXiuRCtL0Hm3bSuXZ2UHoPW3gHayNrjEYbgJhDeyfP60vUvQdXvcOZdUUjOReseh/7PIsHBntfnN4lUQP/Jzg1QqcVIZDf3OkzE79CMDzxpkYvIISLymYjME5GiXtbcWZpsF1oe2ijEjvUjJF9IoDfS73EI74pzLMJQchjS935EBNUkmnX+l/ZTTcHqP+F0ZTWVGQddjdbc5kkdXZEEBxHocQ6BXtcgsSMtiRexnFvk4gxavR04CFgEzBCRiar6Sa5lFyrpfQO68kRIV4ImgYBzw0aPX/odWqeS0CZIvwdRTQOCiJCufxmtuRZS34L0RMvOQsrOym0US2pxK7McpqDxzY6Xa0yB8KJrZTQwT1W/AhCRR4CjgOJN5MHB0P8ViE91kkxou06e1rVtVJPOKAcpy2tsIs63E218C6ouZu1F4CqovR2lEenxm45XECgn66Rg0MpFQWO6Fy+6VoYC32T8vsjdtg4ROVtEZorIzOXLl3tQbdcmEkSiP0RKT0IiO3SpJK6aIL36GvS7ndFlo9Hl+5KufyX/9dbcwtoFmps21kPtv1FNdLhcCfSByB60vAYRQ8rO7HC5xhQKLxJ5tgzVovNTVceraoWqVgwYkNuq7SY3uvoqqHsUJ6m6c5JXXYTGZ+a34uTCVgJKOt1QOZDeN7oXUUuc5diIQtlpNhzPFAUvulYWAcMzfh8GfOtBuSYPNF0N9c8A8WZ7GtCa2531KfMltJmzilBzEsm5C0QCPZ3++OTXzpJ2oS2dlYt8pOmVaPVN0PCSsxB17Gikx3mIZBmWaUwOvGiRzwC2EJFNxLlsfgIw0YNyTT6kv2t9dftUKy1mj0j5RbQcWx6DHucg4s1IWAmNQCK7+p/EtQH93p0DXqsgvQJq70dXnuHZaB1jmuScyFU1CZwHTAbmAo+p6pxcyzV5EhwK2nyODoCAs6JOHklkV6TPnRDaBghDYAj0/ANSekZe6/VF/X8gvQpIZmxshOQcSMz2KyrTTXnSDFLV/wD/8aIsk18iMbTHWVBzN+veQh/NbeRIW+uP7oFEn817PX7TxAfO1AwtdqQh+SlEvL8pyhQvu7OzCEnZuWhgENTe5SzMHP4BUn4pEt7C79C6j9AmON1IzUbpSAiCw/yIyHRjlsiLkIggpcdB6XF+h9JtSezHzl2l60xREIRAP3eopDHesUmzjMkDCfRG+k5wrzsEgRBE9kD6Pmwr+BjPWYvcmDyR8BZI/yfRdC1I0IYdmryxRG5MnkmgzO8QTDdnXSvGGFPgLJEbY0yBs0RujDEFzhK5McYUOEvkxhhT4CyRG2NMgbNEbowxBc4SuTHGFDhL5MYYU+Dszk5j8kQ1DfHpkPgQgkOg5EeIxPwOy3RDlsiNyQPVenTlzyD5BWgDSAxWXwv9JiChTf0Oz3Qz1rViTB5ozXhIfOouLpEGrQWtRCsv9js00w1ZIjcmH+qfARqbbVRIfo6mvvcjItONWSI3Ji9aW2BZ1rPPmI6xRG5MPsSOAqLNNgqENkOC/f2IyHRjlsiNyQMp+yWENgcpdbeUgvREet/ka1yme7JRK6YF1ZQ70qIUEfE7nIIkgVLo9wTE34TERxAYAiWHOtuN8ZglcrOGagqtvhnqH3QWDQ4MRMuvJBA7yO/QCpJIEKL7Og9j8si6VswaWn091D3gDplLQXoJVF2MNr7td2jGmPWwRG4A0HQd1D0CNDTb04DW3OZHSMaYNsopkYvIcSIyR0TSIlLhVVDGB+kVQDD7vtSCzozEGNNOubbIPwaOBt7wIBbjp+BgZ4hzCwKhbTs7GmNMO+SUyFV1rqp+5lUwxj8iESg7B2g+qVMUKT/fj5CMMW3UaX3kInK2iMwUkZnLly/vrGpNO0jZWdDzSggOdyZ5Clcgff8PCW/nd2jGmPXY4PBDEXkFGJxl1xWq+mxbK1LV8cB4gIqKCrtHuQsSEaT0OCg9zu9QjDHtsMFErqoHdkYgxhhjOsaGHxpjTIHLdfjhT0RkEbA78LyITPYmLGOMMW2V0y36qvo08LRHsRhjjOkA61oxxpgCZ4ncGGMKnKh2/khAEVkOLMxD0f2BFXkot1DZ8VjLjsW67HisVUjHYmNVHdB8oy+JPF9EZKaq2pwvLjsea9mxWJcdj7W6w7GwrhVjjClwlsiNMabAdbdEPt7vALoYOx5r2bFYlx2PtQr+WHSrPnJjjClG3a1FbowxRccSuTHGFLhul8ht+TkQkUNE5DMRmScil/kdj59E5N8iskxEPvY7Fr+JyHARmSIic93/kaJeMURESkTkXRH5wD0eV/kdU0d1u0ROkS8/JyJB4HbgUGBb4EQRKea12u4DDvE7iC4iCVysqtsAY4Bzi/zcaAT2V9UdgB2BQ0RkjM8xdUi3S+S2/ByjgXmq+pWqxoFHgKN8jsk3qvoGsNLvOLoCVV2iqu+5P1cDc4Gh/kblH3XUuL+G3UdBjv7odoncMBT4JuP3RRTxP6vJTkRGAjsB7/gbib9EJCgis4FlwMuqWpDHI6dpbP3i1fJz3ZRk2VaQrQyTHyLSA3gSuEBVV/sdj59UNQXsKCK9gadFZJSqFtz1lIJM5Lb83HotAoZn/D4M+NanWEwXIyJhnCT+kKo+5Xc8XYWqVorIazjXUwoukVvXSvczA9hCRDYRkQhwAjDR55hMFyAiAtwLzFXVcX7H4zcRGeC2xBGRGHAg8Km/UXVMt0vkxb78nKomgfOAyTgXsx5T1Tn+RuUfEZkATAe2EpFFInKm3zH5aE/gVGB/EZntPg7zOygfbQRMEZEPcRpAL6vqJJ9j6hC7Rd8YYwpct2uRG2NMsbFEbowxBc4SuTHGFDhL5MYYU+AskRtjTIGzRG6MMQXOErkxxhS4/wfFIwm9u+3bZgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Predicting class for samples in test data using,\n",
    "# best alpha and best iterations for blobs300 data-set\n",
    "np.random.seed(200)\n",
    "file = 'blobs300.csv'\n",
    "# Training logistic regressor for data-set using best alpha value and best iterations\n",
    "w, b = LogisticRegressor(X_train[file], y_train[file], 0.1, 1000)\n",
    "# Predicting classes for Test data-set\n",
    "predictions = predict(X_test[file], w, b)\n",
    "# Calculating and printing accuracy\n",
    "print(f\"Accuracy on Test data-set using alpha 0.1 and 1000 iterations:\")\n",
    "print(f\"{round(accuracy_score(y_test[file], predictions), 4) * 100}%\")\n",
    "# Visualising the predictions for data-set\n",
    "plt.scatter(X_test[file][:, 0], X_test[file][:, 1], c=predictions) \n",
    "plt.title(f\"Testing Result for {file} dataset\")\n",
    "plt.show()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on Test data-set using alpha 0.0001 and 1000 iterations:\n",
      "65.56%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEICAYAAABcVE8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydd5wkVfHAvzV5Zndvdy9xwHFESSIoHEmigIDknBHJIkgSEX4STAQRlSCgJMkccCASJSlR0LsjKVHyHenC3m2c3PX74/XebZjZODuzob6fz3x2+73u96p7eqpf16tXJaqKYRiGMfoJVFoAwzAMozyYwjcMwxgjmMI3DMMYI5jCNwzDGCOYwjcMwxgjmMI3DMMYI5jCH4GISFREWkRkhUrLUipE5AsR2bIf+68nIq/71+G4IZRrwNdaRNYWkdxQyDUaEJGLReT6SssxljCFX0J8xdD+8UQk2WH70EG0+5KIHNa+rappVa1W1c9KI3mnvi4Wkawv8xIReV5Eppe6nz7I0JsiOBt42L8O1w6VLEN5rfuDiJwhIh/738sbIrJKh7ojROQTv26miNR2qJskIg+KSKuIfCgi+1VC/sEiIjNE5JzR0k+lMIVfQnzFUK2q1cAnwO4dym6vtHz94Gb/HCYBLwF3VVieQqwMvDGQA0UkVCohRCRYqrZ66OMk4GBgR6AG2AtY4td9A7gCOBBYHhDg8g6HXwssBiYDRwM3ishXhlpmY5iiqvYZgg/wEbBDl7IgcC7wAbAQuB2o8+uqgBlAA+7H/C+gHvgtkAdSQIu/HQMUmOofOwO4DHgMaAZeAFbu0O+uwP/8di/DKfHDish9MXB9h+0N/b5qOpTtDbzut/ccsG6HunOBz4Em4C1gqw4yntNhv52B9zpsfwFsiVNmGSDrn++/C8j4zy7XZBowHrgDWAB8CJwJiL//94G/A1fhlN85BdoMAef7300TMAuYUuRaXwE8DrT6Mlf5ZXOBRuAZv721gVyHPsYDt/jnOtfvL+DXrQ087x+/ALjFLw/7+29R5Pv6HXBjh+2vAklf7nog1+VeuAf4WZG2ip1HsXvze8DzXdo4G7i7SPtr4O7NZuBR4E/t95rfz73Al34f/wDW8utO9u+HtP993+OXn+d/183Af4FdO/RV8Hr6dev598Ni3D26V0/9jKZPxQUYrR8KK/yzcApyBf8HeRPwZ7/uFGAmEPdv/o2BKr+uk4KmsBKaj1POYb+dm/y6Kf7Nu5tfd6Z/U/eq8IEo8HucAm9Xnpv52xvhHmDHAe/6Mm+AU5jL4UaaqwGrdpCxV4XfVYYerm/Xa3I3TplV+4rlQ+BQv+77OMV3rC9zvEB75wKv+McGgG8AdUWudQOwqb9fFLgB9wCY4re/lf+3q8J/FLgSSOBG468AR/h1fwHO8K9bHF/BA2v6/Z8MzPOv7zkdvo/HgFO6nEsGp/g3BxZ3qTuHIoqsh/MoeG8C44A2YFqHNv6Dr0ALtP8ycBEQAbb3j+2o8I/wv78YcA3wUodjO90/fln7W00AOByn+Cf2cj3H4e7fQ/1z29j/Ptco1s9o+phJp7wcD5ylqp+pagr4OXCgiAhOCU8CVlfVnKrOUtXWfrR9t6q+rKpZ3Ej36375HsAsVX3Ir7sUN7LpicNFZAnuB3kosJ/6vwb/HP6gqnNUNa/Ofh7FPQByuB/XukBQVT9Q1Q/7cQ4DQkSiwL7AT1S1RVXfw73JHN5htw9U9Tpf5mSBZo7BfTfvqaqnqq+o6pIiXc5U1X+pqgd4wHeBH6rqF377z6lqvouMKwNbA6erapuqfo4bTR/k75IFVgGmqGpSVV/wy6f6f7fDXddvA0cB7XM61bhRbEeacaafQnWNfl0nRCTcw3kUvDdVtQl4pP0cRGQ9X95HCrS/pi//z1U1o6pPAX9rr/fbvdn//tp/G5uISKxrWx2OuUtVP/e/r1uBT3H3IRS/nnsD/1XV2/1znAU8iLt/Rj2m8MuEr9RXAh7xJ0OX4EZ4AWACbnT1DDBTROaJyIX9tA9/0eH/NtyPHdzbxNz2Cl9JfdpLW7eqah1u9PQ+yx4e4Gzn/9d+Dv55TAJWVNU3cG8xFwDzReR2EVmuH+cwUKbgruMnHco+BlbssD2XIvjfzYq4c+0LHdtaHjc6/aCXY1bGjVwXdLhul+PehgBOw438X/G9j9oVevvD6SJVbVLV93H3yi5+eQtu1NqRGpzSL1Q3zq/rSk/n0dO9eQdufgHc4GCmqmYKtLECsMBX5u183P6PiIRE5FIR+UBEmoC3caPzCQXaaj/maP9atV/PNYCJfnWx67kysHWX+3df//xHPabwy4Q/Qv4U2E5V6zp8Yqq6UJ03yHmqujZuJLg/y0Z/gwlp+jnLRomISIDOirAnmefjRvQXiUj7D2kucF6Xc0io6n3+MTer6jdx5pwY8Cv/uFbcD7CdKT113Rf5OvAFbqQ9rUPZNDo/2Iq22eG7Wb2P/XVs63Pcm81qvRwzF6eA6ztct3GquqEvw6eqehRO8ZyMm1ydBryJm68oJv8bOFMaACKyrr//+zilOc5vp50NKDzZXfQ8erk3HwZWFZF1/LI7isj5OTCxy4i9o1xH4ialvwXU4sxh4JQ+dDl//43hSpxJcbw/QHmvff8erudc4PEu92+1qp5aqJ/Rhin88vJH4GIRWQlARCaLyO7+/zuIyLq+Qm7C/fjazQJf0rtCKcYDwKYisovvnXI6bsKtT6jq68CzwI/8omuBH4rIdHFUi8geIpLw5d/GN7Ek/U/7ObwK7CYidSKyIvDDHrr9EqdEpId9OsqYxtlsLxSRKhFZHWd3vq2v5wlc7x+/mn9e3xCRuj70ncVNxF4uIsuJSFBEtuz6duabtl4CLhGRGhEJiMhXxF97ICIHisgK/sOn3ZSUU9VG4D7gJ/65rYwz6Tzk73MbsK+IbCYi1ThTyF2qmlLVxf5+v/C/n21xcyfdPMZ6Oo+e7s0O1/4KnG3+mSKX6l3cA+hcEYmIyLd8WdqpwU3CL8LND/yqy/FdfwPVuIf8AiAgIt/HjfDp6XoC9wPf8OvDviyb+Q+QQv2MKkzhl5dLgCeBv4tIM87bZEO/bkXgryzzOHgENxEJbuL0uyKyWEQu6U+Hvq34YNwPciFutP8fnCdCX/kNcKKIjPdtoSfjPCyW4H7Ih+BGRnGcF9FC3IiuGudJAXAjbgT2CU4J3dlDfzNwbwMNIvLPPsp4vP/3Y5wHxvUUUGw9cDFutPp3nFL7I25uoi+cjBtRv4JTWL9k2ci0IwfjJoLfxk0U3sUyk87mwBwRacFNPh+ny3z/28/tC9yk//Xqu/mq6svAqbhJ1S9xv+lTOvR5LM7MsRDnJHC0qv4Plg4yFvbhPHq6N8GN6ncAZvgmw274ivdA3Ai+Aec80PGBfANOeX+Buz+f79LEtcDGvhlmhn/efwRm4+61Vf3/2yl4Pf2H4E64N4rPgc9wD5dwoX4KnctIpn2m3xgj+KP8L3BrBF6stDyGYZQPG+GPAUTkOyJS69tPz8dN6s6psFiGYZQZU/hjg61xfunzcf7PexfxpDAMYxRjJh3DMIwxgo3wDcMwxgglCyJVaiZOnKirrLJKpcUwDMMYUcyZM2ehqk4qVDdsFf4qq6zC7Nmze9/RMAzDWIqIfFyszkw6hmEYYwRT+IZhGGMEU/iGYRhjBFP4hmEYYwRT+IZhGGMEU/iGMQhU86jXgAs2aRjDG1P4hjFAvNY70fmbofO3RudPx2v+HUWCRRrGsGDY+uEbxnBGk49A80W4EO644NCtN6MEkZpTejrUMCqGjfANYwBoy5UsVfZLSULbTajmKiGSYfSKKXzDGAjeF4XLNQ0Fc6QbRuUxhW8YAyG0VuHyQC1IdeE6w6gwpvANYwBIzY9xOdo7EoPqM+ljKl7DKDum8A1jAEhkI2T8zRDeBKQWQusgdb8nkNi70qIZRlHMS8cwBohEvoFMuK33HY0hQb3FkH4OCEB0GyRQU2mRhj2m8A3DGHF4bfdC08+AIIiA5tHa3xKIf7vSog1rSmLSEZEbRWS+iPy3SL2IyBUi8p6IvC4iG5aiX8Mwxh6am+sr+zTQBtoKpKDxR6jXUFnhhjmlsuHfBOzcQ/13gK/4n+OAa0rUr2EYYwxNPQIUWtEskHq83OKMKEqi8FX1WaCnR+uewC3qeAmoE5HlS9G3YRhjDE0C+QIVebcOwihKubx0VgTmdtie55d1QkSOE5HZIjJ7wYIFZRLNMIyRhMS2B6KFaiC6bZmlGVmUS+EXckzWbgWq16rqdFWdPmlSwRy8hmGMcST8NYjvDcRxqiXg/q86BgmtXFnhhjnl8tKZB6zUYXsq8FmZ+jbGAKo5yM+FQC0SGF9pcYwhRsadD/Fd0eRDQBCJ74lENqi0WMOecin8B4CTRGQGsCnQqKqfl6lvY5TjJR/2vTayoDk0shlS91skUFtp0YwhQkQgsjES2bjSoowoSqLwReROYFtgoojMA84HwgCq+kfgEWAX4D2gDTiyFP0ahmZehcaz6RS5MvMiuvgkZMKtFZNroGh+EZq8C3LvQngDJL4vEhhXwva/AAQJLleyNo2RQ0kUvqoe3Eu9AieWoi/D6Ii23oDzx+5IFrKvorm5SGilQocNSzT7DtpwCGgGSEPqH2jrtTDhXiS4wiDbfhddchrkP3HboVWQut8joTVKIPnQofkv0NabIPsqhNZEqo5EQqtWWqwRi8XSMUY2+U8pMP8PEgZvftnFGQza9FPQZpY9wJLgLUabLh5cu14L2nAo5P/nt52G3LvookPQYRzKWXMfoAt3hbZbIfsyJO9BF+2FZuZUWrQRiyl8Y2QT3QzfetgZzUJozbKLM1BUM5AttFDdg8yzg2s89Td3PTr3CGQg9djg2h5CtOki0BagXfY8aBJtOq+SYo1oTOEbIxpJHOnHn+9onYxD9fEjLJhWAAgWqSvkc94PvM9xU2dd0BTkh7HvRObfFHx7y72PatdsY0ZfsOBpRjdU85D6G5p6GCSGxA9AoptVWqyCSHASTPwr2nI1pJ+HYD1SdQwS+06lResXIiE0ugOkn2TZiBYgCon9Btd4eAOQBGgXpS8xVzdcCVSDV8jkFKbgW53RK6bwjU6oeuji4yE7e6mC0NRTaNX3CNScVmHpCiPBKUjtLyotxqCR2l+gDR9D/iNXoAqRbyDVJw+u4ciWEFoDsu+wbH4g5rJ2RTYfXNtDSeJwaLmazrmDoxDfA5Fib0P9R/OLIDsLpBoNb4hk/uVMSZFNkeDkkvUzHDCFb3Qm/UwnZe9IQuuNaOLAQXuLGMWRQC1M+AtkX3NKP7QWEl5n8O1KAMbf6jyakn8BBOL7IFVHDevsXFJ1LJr7CFIPgUSd91Jkc2TcOSXrw2u5FlqucJP8mgfSKDGciS2LVv+AQPUPStZfpRHnMTn8mD59us6ePbvSYow5vMZzIXlX9wpJIDXnIIM1LxhGP9H8F5B7H4IrgcTR5IPgLUCi34TIFu6BNpB2M7PQhmOAnjyV4sj460fUAi8RmaOq0wvV2Qh/DKFeK9rye0g+AOQhtiNS8+POoQgC43C3Ra7L0eJsqoZRZiQ4BYJT0PSL6JLvg3pAGk3e6eYg6q9HpP82fW27k87mokKk0LYZI0rh94R56YwRVBVtOBzaZoAucf7eyb+ii/ZzLoE+Et+XwuMAi0RoVA7VHLrkFD80sj8PoW1ugV3bvQNr1GumoBdQ555919DRgSn8sULm35D/AMh0KMyB1+D8tH0ktBqM+yUQc+6OUg1Si9TfgEis3FIbhiP7Bp29l3w0Can7B9SkxL+Di7jZEwkkttuA2h+OmElnrJB7G7SrmQbQNjT7BhLfY2lRILEnGtseMrNAIhDZZECvzIZRMiRE8dH4ANVYbDdoucFfgVyozwSE1ocR5uLbE6bwxwrBab4nQqZLRbxgbBIJVEPsW+WRzTB6I7SOe9vstpYgjiT2H1ibmoH8vAIVQYh8C0nsCdHtERk9atJMOmOF6FYQqKfzas6Ac3cbRa+sxuhEJIDUXwNSA1IFRIAYRLeH2O79bk81iTYcRmEPHYHwWkhsp1Gl7MFG+GMGkRCMn4E2/hQyLwAK4W8gtRe60bxhDHMk/DWY9Bykn3JzT5GNkfC6A2pLm37uQlAXJOcHsRt9mMIfQ0hwMjL+Ot8rx7NJWGPEIYEExPs/ou+IagaSD9Hd9bidBBLdflB9DFdM4Y9BRCIlaUdVnZeExAa8+MUwyo6mAa94fXQLiAzP2FGDxX6lxoDwkk+gC7ZF52+Ezt8Qr/l3LuiaYQx3pBqCyxeuC62P1F05rENODAZT+GVAVcnnRo8y1PRL0PgjP+xu3nlOtN6ENv+60qIZRq+ICNK+1mSpCgyD1CB1l4zqt9XRe2bDgEw6y1Wn/pndaw7nO9GD+P6GP+bNl4pNFI0ctOVKui9JT0HbjGGdQckw2pHoN5EJ9zgPn9DXIHEQMvFBt/BwFGM2/CHk4sOv4F8Pv0wm6Xzf33/1I37y7V9wzZxLmLrmCI46mf+4cLkI5BfCCMoja4xdJLwWUvebSotRVmyEP0QsmLeIlx6as1TZt5NJZbn70gcqJFWJCK1dpCIAweXKKophGH3HFP4Q8dl7XxCJdg9H4OU9PnityAh5hCA1p+Lsnx2JQ9UJJfMAMgyj9JhJZ4iYutYKZNPdgz0Fw0HW2mT1CkhUOiS8Hoy/xU3SZt+E4ESo+j4St1j5hqG5D9HWP0PuHQivj1QdOWwSB5nCHyImLF/PNgd8k2dnvki6bZlZJxILs9/pg1s4MhyQyNeRCXdWWgzDGFZo5lW04QhcVNo8ZP+DJmfChHuQ0BqVFs9MOkPJj64/gQPP3IvaiTWEIiE22ParXPbcr1h+VbNzG8ZoRJvOw8XnaXfDzrmItE0XVFCqZViKQ8MwjBKgmkG/XJ/Cq3ijBKb8pyxy9JTi0Eb4hmEYJSEEFMkbIVVllaQYpvANwzBKgEgA4vsC0S41Mag6vBIidcMmbQ3DMHpBNQWpJ8D70iVOD08vGG9Hxp2Fel9C+nmXLU7TENsZqTq+AlJ3xxS+YRhGD2juPXTRIUDWKXCJOKVff123dSciMaT+GjQ3D/KfQGg1JDilMoIXwEw6hmEYPaCLTwZtBG2l3euGzCto661Fj5HQVBevZxgpezCFbxiGURTNfwb5uXRPoJ6C5D2VEGlQmMI3DMMohnpAsdj4PSRRGaaUROGLyM4i8o6IvCciZxWo/56ILBCRV/3PMaXo1zAMY0gJrlgkIGAU4nuVXZzBMmiFLyJB4CrgO8C6wMEiUiiz8F2q+nX/c/1g+zUMwxhqRASpu9xlySLuFyYgvBZSdVRFZRsIpfDS2QR4T1U/ABCRGcCewJslaNswDKOiSHhdmPQ0pB5G818gkW9AZCvcWHdkUQqFvyIwt8P2PGDTAvvtKyJbA+8Cp6nq3K47iMhxwHEA06ZNK4FohmEYg0cC4yBxcFFr/kihFDb8Qteg65T2g8Aqqro+8CRwc6GGVPVaVZ2uqtMnTZpUAtEMY/ihXguavB9tvQXNvVdpcYwBoppBU0+hbXeNmO+xFCP8eUDHnHZTgc867qCqizpsXgdYtmtjTKKZ2ejiY0EVyEFzAI3vg4w7v+DKTWN44hZjHQ6kQfOAh8Z2QWovGtZJ0Esh2SzgKyKyqrhlZwcBnXL4icjyHTb3AN4qQb+GMaJQzaKLf+Av4GnDxUxPQep+SP+jwtIZfUVV/e+xAbQFFw45Dem/QeqvlRavRwat8FU1B5wEPIZT5Her6hsi8gsR2cPf7WQReUNEXgNOBr432H4NY8SRfQXIdS/XNpckwxgZ5D+A/Jd0s1xrEm2bURGR+kpJYumo6iPAI13Kzuvw/9nA2aXoyzAqjWZmo223QX4RxLZH4gcggUQfDuye8rJPdcbwQjMgge4zlQCaKrs4/cGCpxlGP/Bab4fmS4AUoJB9DW27CybMRAK9xDyPTKeglpAEEt9zCKQ1hoTQmiBR3zTXkRjE9ih4yHBh+M4uGMYwQ70WaP41zmbbrrhTkP8U7UNcFZEoUnspEAPaoywmILIpxL4zJDIbpUckiNT+FrcQy/8eJQGhNZCqQyopWq/YCN8w+kr2PyChAoP0FKQeh6rv9dqExLaHSY+hyQfAW4JEt4bIZuahM8KQ6Bbue2ybCd6XSPSbEN0BkSIZr4YJpvANo68Eav1gWoXqJvS5GQkuj1QPj4QYxsCR4BSk5qRKi9EvzKRjGH0ltA4EJ9P9ZxNHhkkKO8PoCVP4htFHRASpvwGCKzmbrVQDUag5DYlsUmnxDKNXzKRjGP1AQivBxMch9wZ4jRBeHwnUVFosw+gTpvCNMYFqHm39I7TeBNoEobWRcecgkY373ZaIQHi90gtpGEOMmXSMMYE2/QparnW5SVHIvYU2HI1mLcqHMXYwhW+MetRrguRMnP98R9Joy9WVEMkwKoIpfGP0k/8UCvpHK+TeKbs4hlEpzIZvjH6CU4vEqhEIrVV2cQaK5hdA5jkgCtFtkEB1pUUyRhijTuE3LWrmurNu47mZLyEBYbuDt+SoCw6mqraXOCfGqEUCNWjiAGi7BxcDp50oUv2DSonVL7zWm6D5tyBBXM4hhbo/INEtix6jqmjbHdB6NXgLIbgKUnMWEvtWucQ2hhmiWijkW+WZPn26zp49u1/HZDNZjl3vdL78eAG5bB6AUCTEtLVX5JqXLyEQMAvWWGWZl87NbuI2tI7vpTO90qL1imbfRhcdQOeHFSBxZNILRUf6XusN0HwFnecuYkj9NS40gDEqEZE5qlrwxh5VGvCff51Nw5dLlip7gFwmx+cffMmcx1+roGRGpREJEqg+kcBy/yYw5R0CE+8fEcoeQJN/wSVL6UoAbb0Fb9HBeF+sjzd/G7zW29zIXvPQcjXdJ6pTaPPvhl5oY1gyqhT+B699RLK5ezzqTCrL+699XAGJBs/zf/kXp219Lkd/9VSuP/s2mhY1V1oko9xoCigQw0fz0HoVZOcAKfA+h+bfoC2Xu7UGmi7cXv6jIRTWGM6MKhv+il9Znlh1lFRL5xs9Eg+z4leWL3LU8OWWn9/NPZc+QKrVnc/nH3zJU7c/x7Wv/ZaaepuwGytIbCc0dT9o19F6hu4PgiS03ohWHe3HbC/wZhBcpVuRah4y/wKvASLTkeCUEklvDCdG1Qh/6/03J5aIEQgsCzUbCAaorqti8903qqBk/aepoZm7fn3/UmUPkE3naFrYzIPXPFZByQz1GtDUE2hmFlosemYpiWwO0W+DxP2CAC6mfpGQDhJEvC+h6ge4mO0diSE1p3cq0dyH6IJt0SUnoo3nogt2wGu6iOE6v2cMnFGl8GOJKFe8eAHrb7MugWCAYCjIhjt8jctfuIBQeGS9zLz38oeEo919xzOpLLMfG5vzEeotRttmom13ovnPBt2el/0fXtPv8JouRbP/7dsxLX9A52+DNv4EXXwcumA7NPfRoGXpCRFBan+D1F0D8YMhcQQy4S6IrF/4AM1BYApSdRTU/BgCkwCB4KpI3eWdJmxdQu7jwZvvZ3BqBTKQnAHpJ4b0vIzyM7K0YB9YftXl+M1TPyOTdn7XkQJKcyRQP6WOfIfJ53YkIExaaWIFJBo6VBVN3gdtN4HXBNFtkeoTkeDkpft4yceh8Qw/l6gHXIhWn0qg+ugB9ektPg3SDy+Toe1GNP5dArVnFZcz/Ry0XAekl9nHtQ1dfAxMfGJIk5iICES/6RJttFN9Etowi87eOzGIbOKStUQ2JlB1GFQdVrzh3P/AK5aQ+3YktmMJz8KoNKNqhN+RSDQ8YpU9wKrrTWPqWisQDAU7lUdiYfY5ddcKSTU0aPOF0PQLt+rV+xySd6MLd8XLL3T1XqNT9qRA29xf0tByOZp9u9/9ea23dlL2jhwkb0GzbxSXs/U2unu9KOQ/c6aQ7Lv9lqWvqObQ5F/xGo7GW/wDNPUPCH8dqb/ShWsmAISBDGTmoEt+gM7fEs2+2UvDSSBYuM7rmrPVGOmMWoU/Grjwkf9j7U3WIBILE6+JUVWX4PTrvs9a01evtGglQ/MLoe1OOivSvPOVX7g9mnkZ0n+n8K2aQVMP9r/TlquKVOTQ5N96ELap6HGkZqKL9sNruab/8vSCqocuPh5tPM+ttE0/iS45FW2+AIluQ2DSUzD+Ttw18oBWZ57RBrThSLTgKmOf8Dq4hVxdiUFsl5Kfi1FZRp1JZzRRv1wdlz3/K+Z/soDmxa2svO7UETcX0Su5t4p7k2gSXXw0VP+IAolkXVlPyqwY2thDZQ9joNhOkH2DbgugAKdoU9ByNRrbBQmt3H+5ipF5zne97PhQTELbXWjiMCS0CiTvBwpdiwxkXoLoVgWbFomg4y6Exh/7x+fd5HBwGlJ1cOnOwRgW2Ah/BDB52iRW32CV0afsAQJT3CRjMVT9h0H3+QyIIbGd+99nsLgylsTuPdQdCKGV6e750hGF9JP9l6mnFlPP+qasbhJB5kV/p0YK+uoDeD2v3QjEd0Im/gUSh0H0O8i485EJ9yLS03kaIxFT+EZFkfBXIPQVCpsVADIIeag5E4jiXkoFiEN8bwh/o/99jjsLZ+/uQnQ3JLRG8eMkjky4Bxn3UwiuRuGfjyxtW702NP0imn19cC6OgdrC8koQZJz7N7qDS7vYFc1CZNNeu5DQ6gTG/ZRA/eVIfB9EIgOX1xi2jMIhozHSkPHXoQ3HQO4/BWrDEN2MQPhraPSbaPIh0AwS2xGJfH1g/UW3hfqr0KaLIf8xBOqg+ocEEof0fqzEIHEARDZHF+4CFFjNGtsRr+1eNxEtQcADqYfx1yOh/s+/SHwftPV6uptsBGLb+X3u5OZCsv/FmX4EiEL1SUhwQr/7NEYnoyp42nDh5Sdf548/uplP3ppH3XJ1HPrTfdjt+B2H1G1vpOP8wY+GzGyW2sglDpFvEai/rKKyFcNrvQOaL8IpVwE8GPczJLwOuuggOtv6BQKTkUlPI1LEK6anvpKPQ9OZuLcKBSJI/bVIZIOl+6hmIfUomnoEpAZJHIRERtaCQ2Pw9BQ8zUb4Jeb1Z9/kvD1/TTrpJiEXfdrAn864lbamJAeeuVeFpRueaPIhtPUayC+E0PcnYJQAACAASURBVGpACAIJJL4fxHartHhFCVQdgsa2972IBKLbI8FJeI3n0j3YmYK2uAdatHcTS7e+4juisa0hMwckAuFvINL55ysShvgeSHyPAZ+TMboxhV9i/nzujKXKvp10W5o7LriPfU/bbXROvA4Cr+Va303S90DJLQFJIHX3IaFVKypbX5DgcpDo4s3iLaLwBKqALinYjnotLnZ9+mkITkYSRyCRzvMTIjHotEo2C6mH0dSjIOOQxIEjJgLoWEc1g7bdCcn7AIH4fu77K5iZrXSY9ikxn7w5r2B5PpencWEzE5avL7NEwxfVlIv22MndUJ07ZsvVSN1vKiXaoJDo9mjmhe7BzjQD4e4KWb0mdNFekF8ApCEraOrv6LjzCCT2K9iHahZt+C5k36TdZq+px9DqHxKoPrbDfmnnky/1ZlIcJqh6aMPRkH2NpWa/5kvQ9D+g/voh/Z7MS6fETF2zcFROCQYYN2FsRbhUzaGpR/EWn4LX+FM082rnHXJzKXwLepB9uRwiDg3x3SG4Kp3dN+NQfULBCVRtu2WZsnclQAqaf+UeioVIPQbZt1j2sPSPabncBXfTNF7j/6FfboTO3xpdsBVe0mLjDAsyL/oOCh2/2xRkZw/5fW8Kv8Qc8YuDiCY6u7RFE1EOOGMPwpGRG+qhv6jm0cXHoo1nQ/pRSM5EG76L13LDsp2Ck4ovnAquVB5BhwCRCDJhBtT8BMKbQHQHpP5qAtUnFj4g9RQFvX0IQJHQEZp6Aijgmy9hyPzbXffkg7i5hIwLjtb4I7dy2agomnm58LoKzfhOC0OHKfwSs+H2X+OcGaezwhounvi4CdUc8fMDOOzcwq/mo5b0k5B9pcON3T4CvQzNLwJAAnXOnZBol4NjSNUJZRS29IjECFQdQmDCbQTqr+45pWBgfOFyzfk++IWOqaXYz1dVIfU43R8iKXQIQj8Y/UOCE3HhrbsS9SObDh0lseGLyM7A5bgoTNer6sVd6qPALcBGwCLgQFX9qBR9D0c2220jNtttIzzPG7N5dDX1eOFRjITcK23ced9I7YWoRP3RKBCohppzkAF4soxUpOoINDObznMZQQitVnTiWhIHoMn76R7mIQrBqc6Tp1C4ivwnAG5BWMuVbju0NlJzKhJerxSnY/RGbBdovqR7tBAJ+gOgoWPQ2kicU/FVwHeAdYGDRWTdLrsdDSxW1TWA3wO/Hmy/I4GxquwBfwVokZWoUrVsS6IEai9Elvs3MukpZNILBOKjKxpob0h0a6g+EYiC1Lj1B6HVkfo/Fj8mvJ4zGREFqXbXNDABGX8jElqtSLiKAES+jpd83MXAz852pp7Mc+iiQ7rPsRhDggRqkfqbILC8n9QmDoGpyPhbkUBVb4cPru/BLrwSkc2Bn6nqTv722QCqelGHfR7z93lRnPPwF8Ak7aHzkbzwygDNvllg8REgtcjkF8q+dF8151wYkw+AxJDE/hDZZlh5rqjX5FbKBsZDaK0+yaZeM2RmOYUfmb50UZfX8gc/dn/7W4O40Avj74PF33NhqLsS3ojAhDtLdj5Gz6gq5N93G8HVS3YvDvXCqxWBuR225wFd38eX7qOqORFpBCYAC7sIehxwHMC0adNKIJpRKSS8LlpzJjT/2plxAAgh9ddXQNl76OJjIfMy7QpQ089D4iBk3NlllaUnJDAOOiY46dMxNcvCK3QsrzoRDawArX/y89RuhNScAYHJqDe/cGO5XmLnGyVFRKCH2E1DQSkUfqHHUjfrVB/2QVWvBa4FN8IfvGj9Y+Gni7jrkr/y+jNvstwqkzjgx3uy3hZrl1uMUUOg6jA0vptLji1xiGxWmaBcmWfdBHK38MK34QVXRWJuhexg0dSjaPNlkP/M2d9rzuw2WauaRdtuh+RM0DzE90KqjnCLqkqIiCCJfSCxT5f+83446gLzK4HRlUnN6E4pFP48oKMP3VSga8LR9n3m+SadWqChBH2XjC8/XsAJG51JW3OSfDbPB69/zMtPvs7p15/AdgdtWWnxRizLPHEqh6b+USS8cBaaL0Cbf+Xiytf8ZMCv1S5Y2s9ZasLKvYUuPgHqr0ai7v5x8YKOc+ER2vdruQpNPwXjZyDSg9dN5iWXtjC4AsS+jfOD6D8iQTR+OLTdTGdzWxxGuGeU0TulmFWcBXxFRFYVN3w7CHigyz4PAEf4/+8H/L0n+30luPUX99Da2NYpj2y6LcNVP7yBfK5QLHZjxBCop/jYJo1L2n0npB4dUPOqCi2/pbvHTApt7rBaOPuyv7Cmy4Kb3LvuLaRg2ym04WB0yQloy2Vo07kuifoAE6er1+Jc/4JTcU51MWf/rz4Rie87oDaNkcOgFb6q5oCTgMeAt4C7VfUNEfmFiLRHcboBmCAi7wGnA8UzRVeIl598HS/fPf5Jui3Dvx99hWxmAJmVjGGBxPeh15dZTboVrwMiBd7iwlW5D5f9n3218EIzbSu6IEpbrnNZtrQNyPmpCxejS07vt5Sa/wxd+G33cMq/B4TdQq3xdxOoPm5YTWAbQ0NJ/AZV9RFVXVNVV1fVC/yy81T1Af//lKrur6prqOomqvpBKfotJfXLFV7gkk5muOjQy9lv8tE8dtM/yiyVUQokNA1qL/EThBRIEtKOVyxnbW/EOrmadiK4wrL/A5Od/bzA8RKcUvj45H10X0ClkHsH9fpnFdWmC/wHU/tcRspF8Gz+Rb/aMUYuY9hRvDMHnLEnsarCdtFkS4q2piRXnnQD/33+rTJLZpSCQHxnZPJLUHcFUEg5RyC244DaFhHf/t01JWAMqTmlw+YOFHzTkCDEiq09KGb5FNAiKQ2LkX6W7lE8FTKz3GRuP9HMLLzFP8BbuC9e8+VosbccY9hgCt9nmwO+yUFn7U00HiFWXVjxZ5JpZv7uwTJLZpQKkRiB2NZutE8MZ8PG/R+cjFQdOfC2q46CmpNBal27gUkuGUrsOx36jyPj7/ADq8X8fqci429BioVQiO8OdPVsEgit6i/R74+QxcxaQfqrCry2e1zEx/STLhBY63Xowt37/dZhlBfLeNWFtuYkT894gT+ecTPJ5u6RCtfcaDWumjUmFgoPS1TVKRhvCYQ3KK4oe2sn+y7adqvvQrm2U7yhVSGySVFvmT7LRwqIdbKJq9cK+U8hOMWtps3PAzwITuvRdq5eK9pwiEvFqG1AHCSMjL/T5QPuB17jzyF5D52Ts4QhtjOBut/24xzT6PzN3HxCJyJQdQSBmh/3Sy6jtFjGq36QqImz7UFbcPWpf+5WF46G2GjHDQocZZQDzc1FFx8F3gIgAJpFq08hUH1Mv9uS8Jow7nx0yWnQditIAEUgMAHG31bcpt5bu+InWG+XWRVtvhTabnEjbM1CfG9k3PndMlYVbC9QBRPug/QzaPZ1JLgCxHZBAv0PtS01Z6C5N/ywyjlAXa7d6n5OAOfeo/DSmgykn4ESKHz1Wl0WtORfXV/xPZGqE5BAD3MwRq+YSacAiZo4h5+/fyebfigSorq+mn1OHVtxXoYLzof9WMjPdSNdbQHS0Holmv7nwNpsvcUpKFJ+m24Urkt+VDq5226BttucrNqKcwH9K9r8+z63IRJEYtsRqDkVSRwwIGUP/sNjqXJXwANtgoaDl0Yw7ROBuuJhrQODT5iumkcbDoXWm8D7ErwvoPUmtOFQtL/zFkYnTOEX4cAz9+Knd57GBtt+lWnrrMieJ+3Mn169lLpJAzMhGIMk97Yf/6XLD16TzjQzEJJ30t13Pg/ZV3ucgNT8Ave20RdzaOv1dF7hi+szefuQKi/VrDNb5b/oUKbQeBbO66e97xR4i9CWq/rctgRXhPB6dDcQxN1cRjdZMmjqb2jrjWhmVu/XLf0s5D+is+kpDfkPIfNCn+U0umMmnR5oD3NsDAO0mWWTrF0YqHeIFko6As5c1L1O85+jS07x0woG3IKuukuRyMbF+/AK57BFUzizSulDTXjJh6HpfCAPmkPDX0PqrgTSfr7drmTd5Cvn9bkPqfsDuuT7kH3HN1XloPpkJLpNp/009wnacLD/BpVxfv+hdWH8n4uvFs690T09JLhrlv0vRLfqs5xGZ2yEb4wMQuu52DPdiEF0YO6UxL4NFMhCFlwOAst1KnJ5SA+F7Ou4kWcKvM/Rxceg+QKRJ9spFmM+uNKQxBXS7OvQeLYz1WgrLkfuqy6kg8QpnFyd4usIiiDBCQQm3INMfACpvxaZ/E8C1Ud3l6fxR+4ho61A1in+7H/QlmuLNx5cge4uroDEOq9rMPqNKXxjRCCBBIz7P5w7Y/uEoVMAkjhwYG1Wn+SU+1LlEgFJILW/6e45k/mX/ybR1aSUR9vuKt7HuLP99tt/agLEkHHnD0jm3tDWm+i+UCsHuf85xRvZiO4v9jFIHDag/iS0ChKZXnBeQb0G/22o60Mm7S8oK0J0Z3+BWsfvwF23SsdlGumYSccYMQQSB6Khtd1EqLcQotsj8f0G7LkhgXqY+DAkH0Qzs5yLZGL/wh463pcUXgSVWZpFqmAf4fVhwj1oyx9c+OHg6kj1D5DIEHl75T8rLKeEIL8Aqf0duvgI5yLqezoR/w6SOLj0sqhS2JsHoPhCLwkkYMKd6JIfu7kbgNA6SN2lJY8qOtYwhW+MKCSyARLpu894r+1JHBIHIIkDet4xvH4Rk1IciWzecx/hNZH6KwYuZH+IbOFi73Qd5WsWwuu46KUTHnKmKe9zCH0VCQ1NwngJTkBDq0Luna5C+gvKejg2tBoy8V5/8lyc3MagMZOOYfQBCa3mh17oaFuOQHDS0vy8pURVXbCz/rhLAlJ1uJ/gvOPcRByqjl2qNEXEPThjO3dT9pr7BG25Dq/5ajT77iDPAqT2UpfuUvzrJgkIrdLnJPUSqDdlX0Jspa1h9BHVPNo2A5J3gJeE+M5I1fEDXu1btJ/MbGfO8BYCCuH1kLrL+rwYTL0GNyma/gcE6pGqI5E+2L691hnQfAHO5u4BYYgfhMR3heCK/Q/lsFSeZkg9hObmOlNWdPs+LTozBkZPK21N4RvGMELzX6ALd+rilhh0Cnfi44MK+9Bzv/PRBdvTfcIX3FuNB7HtkdpLKpO1zOgzPSl8M+kYxjBC2+4qMFeQdx42mX8PXcfpf1BcHSSBNKSeQpsuHDoZjCHH3qsMow+oKqSfRpN3g6aR+O4Q272TaUI1CcmHnC98aHUkvlf/7c/5T+i8wnRp6/5K46FCnENNjy/8aUjei447x0wyIxT71gyjD2jzr6BtJu1hEjQzB5IPQP0NiATQ/EJ00b7+ytokEHOumP2NahneFFJP0i0cg3rOU2ioiG4HTb/sw445twrZFP6IxEw6htELmvsI2u6msxJOQvYVyDzn9mm51I/i2TGbVDPadHa/+pLE7hCcSOexWAyi2yGh1Qd8Dr32G5wI434BROm8uK0LwakuCJsxIrHHtGH0RuYlCipAbUNTT7v4MakncbFxOu0A2TdQr61Pi8PUa0GbL4T8fNzCpKhLi1j1XWSAK2H7QyCxNxr9JqQeR/OfQdvtQNaXJQBEkHE/G3I5jKHDFL5hdEA14+LFSwxCa7oQCzLOpSHsZt8OuwBqQPGfkrhje+1X0YYj3Wpc2kMPZ0Gb3VxAH9ooBRJcDqoOd+b8xEFo67WQfc1fIXw8El63LHIYQ4MpfMPw8ZJPQNNZLPVDD0yG+j9B7FvQVMjEEUTie7t/E/tC6y10dmsMQWTL4lEhO5J9HfLvskzZ42TQFNo2EykQmGyokdDKSO0FZe/XGDrMhm8YgOY+hMYfuTDM2ur84POfoA2HAxGk/s8g411USal2f2svXbpSVap/6E+qxoGYqw+uhNT20Y0x9z6F7eYpyL1VmpM0xjw2wjcMfP/3QjZ4bYPMS0h0C5j8gpuo1SxENuw0cheJwfjb3Eg99xYEp0Fks74vlAqt7gcb60rMxY83jBJgCt8wwI+G2VXht9c1AC7VIJGCCxj9eoHIBu7TX8LrQ3gtP5xwux9+ACSGJPbtf3uGUQAz6RgGOE8bKZB0Q3N+DPkh7l/EmY3ie/ty+Pb/CTNLHqvHGLvYCN8wAGK7QOufIfchS/PcShziByBlyrIkgSqk9pdQ23kBlGoabbkGkjOdOSn2baTmdCQwvixyGaMHU/iGAS4g2IQZaOudkHoEAlVI4pCBp08sIbr4OMi8zFIPoOR9aPp5mPSoi+dvGH3EFH6F+fjNufz3+bepn1LHxjt/nXCkQI5VoyyIxJHqo6D6qEqLshTN/geyr9LZ3TPnQjgkH4LE/pUSzRiBmMKvEJ7n8Zsjr+LZmS8hAsFgkEg8wm+f/jnT1l6x0uIZw4Xsm0UCmrWh2VcRTOEbfccmbSvEE7c8w/P3/YtMMkO6LUNbc5LGBY2cv9clDNccBUYFCE6Fgq6dMQiuWnZxjJGNKfwK8dCfniDV2jnZhCosmLeIee9+ViGpjGFHZHMITAK6hFaQMJLYpyIiGSMXM+kMEarKGy+8zYsPziZWHWP7Q7ZihdWXpajLpArFPIdAQMimi/iDG2MOkQCMvx1t/IkfxA0IfQWpvdi8dIx+Ywp/CFBVfnPkVTx370uk2tKEQkFmXHw/J199DDsd8S0Atj90Kz5993PSyc6KP5qIsvJXp1ZCbGOYIsFJyPgbUa8NyCOBmh7318zLaMtlkHsPgqshNacgkY3LI6wxrBmUSUdExovIEyLyP/9vfZH98iLyqv95YDB9jgTmPPG6U/ataVDIZfNkkhmuOOF6Wpa0ArDHD3Zm2rpTiVfHAAhHw0QTUc6+/RSCwfJERjRGFhJI9K7s0/9EG77n3ga8hZD9N9pwNJp+tjxCjkA0Mwuv4Wi8BTvhNZ6D5uZVWqQhY7Aj/LOAp1T1YhE5y9/+SYH9kqr69UH2NWJ4esbz3ezzAMFwgDmPv8Y2B3yTWCLKFf+8gBfun8UrT73OxKkT2Ol732LS1AkVkNgYLWjzRSxdOLaUFNp0ATJp60qINKzxkg9C409Zes2Sn6CpR2HCX5DQtIrKNhQMVuHvCWzr/38z8DSFFf6YIhQJISLdvG0EIRheNnoPhUNss//mbLP/5uUW0Rit5N4rXJ7/EFV18X4MAFTz0PQrOj8g86CtaMtlSN3vKiXakDFYL53lVPVzAP/v5CL7xURktoi8JCJ7DbLPYc8Oh29DJB7pVu55HhvtOIDAWobRV4pN5EqdKfuueF+Adn0bAvAg8++yi1MOelX4IvKkiPy3wGfPfvQzTVWnA4cAl4lIweScInKc/2CYvWDBgn40P7xYb4u12fe0XYnEwkRiYWJVUaKJKOfd8yPiVbFKi2eMZqqOw8Xk70gcqo6thDTDG6nFpW8sQGBiWUUpFzKYRT4i8g6wrap+LiLLA0+r6lq9HHMT8JCqzuxpv+nTp+vs2bMHLNtw4LP3v2DW314lVhVli702obrOkj8bQ4uqoq1XQev1bmGHAIkjkeqT+x6bfwzhLTnNz0fccc4tjtRehMR3KZscqnlIP4NmXoTAZCS+JxIsZjDpGRGZ4w+wu9cNUuH/BljUYdJ2vKqe2WWfeqBNVdMiMhF4EdhTVd/sqe3RoPAL8d4rH3Ltmbfy9r/+R93kcRz4k73Y5Zgd7HXbKCmqGeelE5jQtxSLYxT12tDGH0P6WZAwaB6qTyRQfVz5ZNAM2vBdyL3tEu4QBQkgddci0U373V5PCn+wk7YXA3eLyNHAJ+ACe4jIdOD7qnoMsA7wJxHxcCaki3tT9qOVj96Yy2lbn7vUgyfZkuKa025m4acNHPGzAyssnTGaEIlAmcI6j2QkkEDqr0LzC8GbD8FVkECirDJo2x1+4pv2+QTnzq2Np8KkF0r6ZjaoEf5QMhpH+L866Pc8O/NF1Ot8zaPxCPfMv8Hs+4YxDNDc+5B9G0IrQ+irQ/727S3cG3JvdK+QBDL+DiTcvxSXQznCN/rBu7Pf76bsAYKhIPM/XsDK665UAakMwwDftLLkFEi/ABIE9SC0Boy/cWizjkmxkOgKlDZcus3ilJGpay5fsDybyTFhBYuLYhiVRFv+BOnngRRoK5CE3Nto4zlD2q/ED6K7ZxXOUyi0Rkn7MoVfRg756b5Eu/jnR+MRdjhsK/PgMYxKk5xBZ28dgCyk/+4mwYeK+F4Q2w6IARGQKpBapO6qkpuTzKRTRtbbYm3Ouet0rvzh9Sz6dDGhSJBdj/s2x/76sEqLZhiGJotUeC6XsHRfTFkKRAJI3e/R7FuQmQ3BiRDdbki8q0zhl5nNdtuITXfdkLbmJLFElGDIAqUZxrAgug2kHgW8zuWhNZHA0L+BS3gdCK8zpH2YSacCiAhV4xKm7A1jGCE1Z4LU4Uwr4MwrCaT2gm77qibR7Luot6SsMg4WG+EbhmEAElweJj2Gtt3tEseH1kASByPBKZ3281quhZarXOpJzaKxnZDaC0fEAjdT+IZhGD4SqEWqi8cd0uRDTtmTXJZcPvU4KlGk9sKyyDgYzKRjGIbRR7T1GqDr5G4akg+iRSd9hw82wjcMw+iA5hdB9nVUal18m+QM0BzEd4N8D1F8vWYIFvCnH0aYwjcMw8CPNNpyGbTe6AdSa8PZbXzbTcs1vmumLCtrJ1A9IkIqm8I3DMMASD8JrTfhgpd1T1HqygWIAFmWuW/GoOacERF+2hS+YRgGoG230t0+35UURHdzsXayL0NwGlJ9AhLZpBwiDhpT+IYxhlBVyP0XvEYIb4AEaiot0vDBa+zDTlEIr13WePmlxBS+YYwRNPcJuvgolxgF34e85nQCVUdWWrThQXQHyH1A93g6HZAgEt+7bCKVmuFvdDIMY9CoKrr4aMjPc5OR2gKkofkydJQm7O4vUvU9CC7HspW24n/CQBwCU5D665HgpEqJOGhshG8YY4Hcm+AtoFucGJJo660jxgY9lEigBib8FU3OhPQzEJyCJA4DqQGyEFx1xKciNYU/yvE8jwVzF1Fdl6Cq1kIwj1m8RqBI7CavoayiDGckUIVUHQFVR1RalCHBFP4o5rl7X+KKE68n2ZzE8zw2220jzrjxRBI1w3txiDEEhNd3IX67EYPYjmUXx6gMZsMfpbz1r//x6yOuZMn8RtLJDNl0jpcemsOvDvxdpUUzKoAEqqHmJzj7dLtZIgbBFZH4/hWUzCgnNsIfpdx1yf1kkp2z9GTTOV57+g3mz13I5JWG/6pAo7QEqg5Fw+s4f/P8IojtgMT3QwKJIelP1YPUQ2jbXUAGYnsiiQOQIUokYvSOKfxRyucffIl2z5dOOBpm4acNpvDHKBLZEIlsWJa+tPFsSP9tWSap7Dto6mEYfxsilguiEphJZwSy+MslPHHrMzxzz4skW1MF91l/63UJhbv/qLLpHCuvO3WoRTTGOJr9n8se1SmCZApyb0H66UqJBYBqGq91Bt6i7+ItPglNv1hRecqJjfBHGPdd8TA3nHW7y5YlgnrKz+8/kw23/1qn/Q748Z48eduztDUl8fLOFS9WFWXf03ejatzQvMIbxlKyRXz7tQ1Nv4DEti+vPO3dawZddDDk3qc9jIKmn0OrjydQ/YOKyFRObIQ/gnj/tY+48ew7yKSyJFtSJJuTpFpTnL/XJd1G+pOmTuCaOZew3SFbMmGFelbbYGVO+9PxHPGzAyskvTGmCExw8Wa6EYHg5LKLs5TkQ5Bfpuz9Qmi5Bh0D7qk2wh9BPH7z02TT3V3rAgHh34+8wjb7b96pfMoqk/nJzT8sl3iGsYzotyisXoJIfK9yS7MUTT/ZxczkI2HIzILYTuUXqozYCH8EkW7L4HndZ2I91W4eOcOVfD5PU0Mz+Xy+0qIYQ4hIFBl/CwRWAEmAVIHUIfVXd8sRW1YCEyis9tRPYD66MYU/gthyn02JVXVPlOzl8my04/oVkKjvqCp/ufIR9pt8NAeucBz7TDiSGb/+i4veaIxKJLwOMukfyPgZyPhbkMn/RKJbVFamxEG4ePadSt0DKTK9EiKVFVP4I4iNvr0+m+660VKlHwgI0XiEoy48hPFT6issXc/87c9/54az76BlcSu5TI62piS3//Je7rv84UqLZgwhIoKE10bCX0Ok8hZkCX8Vxp0LxEGqnaIPTEHG3zQmXEVluI6wpk+frrNnz660GMMOVWXOE6/z3L0vEk1E2fGIbVnj66tWWqxeOXSVE5j/ycJu5eMm1nDv/BsrIJExllGvFbKvOqUf/tqIyFbVV0RkjqoWfF2p/CPX6BciwvQdN2D6jhtUWpR+sejzxQXLmxc5e34wOPpHV8bwQQJVUGHzUiUYPY81Y1gzbe0VC5Yvt/IkU/aGUSZM4Rtl4dhLDica7zxZFo1HOO433x1wm5r9D17DEXhfboS3cFc09ehgxTSMUc2gFL6I7C8ib4iIJyJFp7hFZGcReUdE3hORswbTpzEy2Xinr/PLB89irU3WIDEuzhrfWJXzZp7BVvtuNqD2NPsGuugwyLwI2gy5/6GNZ+G13l5iyQ1j9DCoSVsRWQeXQudPwBmq2m2WVdzU97vAt4F5wCzgYFV9s6e2bdLW6Amv4WjIPNe9QmqQyf8aFh4hhlEJepq0HdQIX1XfUtV3etltE+A9Vf1AVTPADGDPwfRrGOSKjBc06yfpNgyjK+Ww4a8IzO2wPc8vM4yBE+zhFgqM/hWThjEQelX4IvKkiPy3wKevo/RCWX8L2pFE5DgRmS0isxcsWNDH5o2xiFSfhMve1JEYJA5EpGu5YRjQBz98Vd1hkH3MA1bqsD0V+KxIX9cC14Kz4Q+yX2MUI9Ft0XE/h5ZLwGsGgpA4GKk5o9KiGcawpRwzW7OAr4jIqsCnwEHAIWXo1xjlBBJ7o/E9QZeAVFvqPMPohcG6Ze4tIvOAzYGHReQxv3wFEXkEQFVzwEnAY8BbwN2q+sbgxDYMh0gACYw3ZW8YfWBQI3xV/QvwlwLlnwG7dNh+BHhkMH0ZhtF/VBVtuw1a/wjeIgitjtT8X8WjVhqVwVbaN5odkAAABddJREFUGsYoRluvgeZLwVsAeG6B2uIT0MysSotmVABT+IYxSlHNQOt1dE7nB5BCmy+rhEhGhTGFbxijFa8BtEhmsdz75ZXFGBaYwjeM0UpgPBSL8x4a/jkUjNJjCt8wRikiEag6BiTepSaGVJ9aEZmMymIK3zBGIapZNPkI5D6H8IZArasIrobUX4lEN62ofEZlsJCChjHKUE2iiw6B3IdAGxAGQlB3I4HYlhWWzqgkNsI3jFGGtt7hT8q2+SVZIAmNZ6DFJnGNMYEpfMMYbaQeBFIFKtKQ6y2auTGaMZOOYXRA81+grTdCZg6EVkWqjkbC61RarP5RLMyEeiDR8spiDCtM4RuGj+Y+QRftA5oEspB7A009DvVXIdGtKi1en5HEIWjTO/55LC2F4HIQXK1ichmVx0w6huGjzZeCtuBs3uCyd6bQxnMYTCrQshPbA2I74/IFxEGqIDAeqb8akULpKYyxgo3wDaOdzEs4Jd8FbxHoYpDxZRdpIMj/t3d/L1aUcRzH35/dbXfZRPNXpfZTE8KrCgmpripCu9jtJ+VNFobsnyB5VxfhVRBEUBLZjUnetEEQpYVXVl5UamL+AGlZUfvlVVjZt4vzbBx2z+6Mq87M7vN5wTBzZh4evvvlme+Zec6cs+pCC7YT129pTU11LYW+h5Cuqzs0q5kLvtm4rgVw6Y/OxyZ9ean51LMKelbVHYY1iKd0zMYNvARMLOy90P8YmoUF32wiF3yzRAMbYeA5oA80r7XuXYfmv1Z3aGZXhad0zBJJaP4rxLzh1heXupej7hV1h2V21bjgm02grkXQOzs+oDW7HJ7SMTPLhAu+mVkmXPDNzDLhgm9mlgkXfDOzTLjgm5llQk39UShJ54HTdceRLAF+qTuIhnOOijlHxZyjYkU5uj0ilnY60NiC3ySSDkbE2rrjaDLnqJhzVMw5KnYlOfKUjplZJlzwzcwy4YJfzjt1BzALOEfFnKNizlGxGefIc/hmZpnwFb6ZWSZc8M3MMuGC34GkZyUdkfSvpCkff5K0XtIxSSckba0yxrpJWiTpc0nH03rhFO0uSfouLSNVx1mHonEhqU/S7nT8a0l3VB9lvUrk6EVJ59vGzst1xFkXSe9JOifp8BTHJenNlL8fJN1Xpl8X/M4OA08B+6dqIKkbeAvYAKwBNkpaU014jbAV2BsRq4G96XUnf0bEPWkZrC68epQcF5uB3yPiLuANYHu1UdbrMs6d3W1jZ0elQdbvfWD9NMc3AKvTsgV4u0ynLvgdRMTRiDhW0Ox+4EREnIqIv4APgaFrH11jDAE70/ZO4IkaY2mSMuOiPXd7gEckqcIY65b7uVMoIvYDv03TZAj4IFoOADdIWlbUrwv+zK0Afm57PZr25eKmiDgDkNY3TtGuX9JBSQck5fCmUGZc/N8mIv4BLgCLK4muGcqeO0+n6Yo9km6tJrRZY0b1J9t/cSjpC+DmDoe2RcTHZbrosG9OPeM6XY4uo5vbImJM0kpgn6RDEXHy6kTYSGXGxZwfOwXK/P2fALsi4qKkYVp3RA9f88hmjxmNoWwLfkQ8eoVdjALtVx23AGNX2GejTJcjSWclLYuIM+lW8twUfYyl9SlJXwH3AnO54JcZF+NtRiX1AAuY/vZ9rinMUUT82vbyXTL7nKOEGdUfT+nM3LfAakl3SuoFngeyeAolGQE2pe1NwKS7IkkLJfWl7SXAg8CPlUVYjzLjoj13zwD7Iq9vQBbmaMJ89CBwtML4ZoMR4IX0tM464ML4FOu0IsLLhAV4ktY76EXgLPBZ2r8c+LSt3ePAT7SuWLfVHXfFOVpM6+mc42m9KO1fC+xI2w8Ah4Dv03pz3XFXlJtJ4wJ4FRhM2/3AR8AJ4BtgZd0xNzBHrwNH0tj5Eri77pgrzs8u4Azwd6pFm4FhYDgdF60nnU6mc2ttmX790wpmZpnwlI6ZWSZc8M3MMuGCb2aWCRd8M7NMuOCbmWXCBd/MLBMu+GZmmfgPyf1RcF/g4FwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Predicting class for samples in test data using,\n",
    "# best alpha and best iterations for circles300 data-set\n",
    "np.random.seed(200)\n",
    "file = 'circles600.csv'\n",
    "# Training logistic regressor for data-set using best alpha value and best iterations\n",
    "w, b = LogisticRegressor(X_train[file], y_train[file], 0.0001, 1000)\n",
    "# Predicting classes for Test data-set\n",
    "predictions = predict(X_test[file], w, b)\n",
    "# Calculating and printing accuracy\n",
    "print(f\"Accuracy on Test data-set using alpha 0.0001 and 1000 iterations:\")\n",
    "print(f\"{round(accuracy_score(y_test[file], predictions), 4) * 100}%\")\n",
    "# Visualising the predictions for data-set\n",
    "plt.scatter(X_test[file][:, 0], X_test[file][:, 1], c=predictions) \n",
    "plt.title(f\"Testing Result for {file} dataset\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 3 Solution: Implement and Test a Shallow Neural Network\n",
    "\n",
    "Following sections contain different methods required to run the neural network for different datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.1: Sigmoid Function and First-order Derviative of Sigmoid Function\n",
    "f() and f_dash() methods calculate sigmoid function and first-order derivative of sigmoid function for activations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.1: Sigmoid Function and First-order Derviative of Sigmoid Function\n",
    "def f(z):\n",
    "    '''\n",
    "        Method to calculate sigmoid value of z\n",
    "    '''\n",
    "    return 1 / (1 + np.exp(-z))\n",
    "\n",
    "def f_dash(z):\n",
    "    '''\n",
    "        Method to calculate first-order derivative of z using sigmoid\n",
    "    '''\n",
    "    return f(z) * (1 - f(z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.2: Forward Propagation\n",
    "ForwardPropagation() method performs the forward pass of the neural network. The method is configured to work for both shallow and deep neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.2: Forward Propagation\n",
    "def ForwardPropagation(x, weights, biases):\n",
    "    '''\n",
    "        Method to perform forward propogation step of neural network, \n",
    "        to calculate sigma and activation values for each neuron\n",
    "        in all hidden and output layers\n",
    "        \n",
    "        x: numpy array of all features for a single sample\n",
    "        weights: initialised, updated, or trained weights for each layer, \n",
    "                 depending on the phase of calculations\n",
    "        biases: initialised, updated, or trained biases for each layer, \n",
    "                 depending on the phase of calculations\n",
    "    '''\n",
    "    sigmas = []\n",
    "    activations = []\n",
    "    # Initialising activations with input values\n",
    "    activations.append(x)\n",
    "    # Calculating sigma and activation values for each neuron in all layers\n",
    "    for layer in range(len(weights)):\n",
    "        # Initialising sigma and activations for current layer\n",
    "        sigma_layer = np.zeros(weights[layer].shape[0])\n",
    "        activations_layer = np.zeros(weights[layer].shape[0])\n",
    "        # For each neuron in layer calculating the sigma value\n",
    "        for i in range(weights[layer].shape[0]):            \n",
    "            sigma = 0\n",
    "            # Summing the product of all weights and activations \n",
    "            # in previous layer for a single neuron\n",
    "            for j in range(weights[layer].shape[1]):\n",
    "                sigma += weights[layer][i][j] * activations[layer][j]\n",
    "            # Adding bias of the neuron to the sum calculated in sigma    \n",
    "            sigma_layer[i] = sigma + biases[layer][i]\n",
    "            # Applying sigmoid function on sigma of the neuron\n",
    "            # to calculate the actiation of the neuron\n",
    "            activations_layer[i] = f(sigma_layer[i])\n",
    "        # appending all sigma values for current layer to list of sigmas\n",
    "        sigmas.append(sigma_layer)\n",
    "        # appending all activation values for current layer to list of activations\n",
    "        activations.append(activations_layer)\n",
    "            \n",
    "    return sigmas, activations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.3: Back Propagation\n",
    "BackPropagation() method performs the backward pass of the neural network to propogate the errors in predictions back to all layers. The method is configured to work for both shallow and deep neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.3: Back Propagation\n",
    "def BackPropagation(y, activations, sigmas, weights):\n",
    "    '''\n",
    "        Method to perform backward propogation step of neural network, \n",
    "        to calculate weight and bias gradient values for each neuron\n",
    "        in all hidden and output layers\n",
    "        \n",
    "        y: actual class value for the current sample\n",
    "        activations: activations of each neuron in all layers\n",
    "        sigmas: sigma values of each neuron in all layers\n",
    "        weights: current weights of each neuron in all layers\n",
    "    '''\n",
    "    # Initialising list of weight and bias gradients\n",
    "    delta_weights = []\n",
    "    delta_biases = []\n",
    "    \n",
    "    # Initialising list of delta sigmas with the delta sigma of output layer\n",
    "    delta_sigmas = [np.array(activations[len(activations)-1] - y)]\n",
    "    \n",
    "    # Calculating the weight gradients for output layer\n",
    "    for i in range(weights[len(weights) - 1].shape[0]):\n",
    "        delta_weight = []\n",
    "        for j in range(weights[len(weights) - 1].shape[1]):\n",
    "            delta_weight.append(delta_sigmas[i][0] * activations[len(activations)-2][j])\n",
    "        delta_weights.append(np.array([delta_weight]))\n",
    "    \n",
    "    # Reversing the weights and sigma lists for \n",
    "    # calculating deltas for hidden layers\n",
    "    reversed_weights = list(reversed(weights))\n",
    "    hidden_weights = reversed_weights[1:]\n",
    "    hidden_sigmas = list(reversed(sigmas))[1:]\n",
    "    \n",
    "    # Calculating the gradients of sigma for each neuron in each hidden layer\n",
    "    for layer in range(len(hidden_weights)):        \n",
    "        delta_sigma_layer = []\n",
    "        for i in range(hidden_sigmas[layer].shape[0]):\n",
    "            # Calculating the first-order derivative of sigma \n",
    "            # value of neuron in current hidden layer\n",
    "            delta_sigma = f_dash(hidden_sigmas[layer][i])\n",
    "            summation = 0\n",
    "            # Calculating sum of product of delta sigma values and weights of the next layer\n",
    "            for j in range(len(delta_sigmas[0])):\n",
    "                summation += delta_sigmas[0][j] * reversed_weights[layer][j][i]\n",
    "            # Multiplying the sum with the first-order derivative of sigma\n",
    "            delta_sigma = delta_sigma * summation\n",
    "            # Appending the gradient sigma to list of sigmas for current layer\n",
    "            delta_sigma_layer.append(delta_sigma)\n",
    "        # Inserting the delta sigmas calculated for \n",
    "        # current hidden layer at the beginning of delta_sigmas list,\n",
    "        # to be able to use it as the next layer delta sigmas \n",
    "        # for next hidden layer in the reverse order\n",
    "        delta_sigmas.insert(0, np.array(delta_sigma_layer))\n",
    "    \n",
    "    # Calculating the hidden layer weight gradients\n",
    "    hidden_delta_weights = []\n",
    "    for layer in range(len(weights)-1):\n",
    "        layer_delta_weights = np.zeros(weights[layer].shape)\n",
    "        for i in range(weights[layer].shape[0]):\n",
    "            # Calculating weight gradient for a neuron in current layer\n",
    "            # by multiplying delta sigma for the neuron with activations from previous layer,\n",
    "            # NOTE: activations list starts with input values in first position,\n",
    "            # thus, activations[layer] consists of activations of previous layer for current layer\n",
    "            for j in range(len(weights[layer][i])):\n",
    "                layer_delta_weights[i][j] = delta_sigmas[layer][i] * activations[layer][j]\n",
    "        hidden_delta_weights.append(np.array(layer_delta_weights))\n",
    "    \n",
    "    # Combining the delta weights for all hidden and output layers\n",
    "    delta_weights = hidden_delta_weights + delta_weights\n",
    "    # Setting the delta bias values for all hidden and output layers\n",
    "    delta_biases = delta_sigmas\n",
    "                \n",
    "    return delta_weights, delta_biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.4: Weight and Bias Update\n",
    "SGD_Parameter_Update() method updates the current weights and biases for all neurons in each layer. The method is configured to work for both shallow and deep neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.4: Weight and Bias Update\n",
    "def SGD_Parameter_Update(alpha, weights, biases, delta_weights, delta_biases): \n",
    "    '''\n",
    "        Method to update weights and biases using the weight and bias gradients\n",
    "        \n",
    "        alpha: Learning rate\n",
    "        weights: current weights for each neuron in all layers\n",
    "        biases: current biases for each neuron in all layers\n",
    "        delta_weights: weight gradients for each neuron in all layers\n",
    "        delta_biases: bias gradients for each neuron in all layers\n",
    "    '''\n",
    "    # Updating weights using gradient descent update\n",
    "    for layer in range(len(weights)):\n",
    "        for i in range(weights[layer].shape[0]):    \n",
    "            for j in range(weights[layer].shape[1]):\n",
    "                weights[layer][i][j] -= alpha * delta_weights[layer][i][j]\n",
    "    \n",
    "    # Updating biases using gradient descent update\n",
    "    for layer in range(len(biases)):\n",
    "        for i in range(biases[layer].shape[0]):\n",
    "            biases[layer][i] -= alpha * delta_biases[layer][i]\n",
    "    \n",
    "    return weights, biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.5: Prediction function for neural network\n",
    "PredictNN() function calculates predictions for test data-set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.5: Prediction function for neural network\n",
    "def predictNN(data, weights, biases):\n",
    "    '''\n",
    "        Method to calculate predictions for test-data set\n",
    "        data: fixed number of samples \n",
    "        weights: trained weights for each neuron in all layers\n",
    "        biases: trained biases for each neuron in all layers\n",
    "    '''\n",
    "    predictions = []\n",
    "    for sample in data:\n",
    "        # Performing the forward pass on sample using trained weights and biases\n",
    "        sigmas, activations = ForwardPropagation(sample, weights, biases)\n",
    "        # Getting the activation in output neuron\n",
    "        prediction = activations[len(activations) - 1][0]\n",
    "        if prediction > 0.5:\n",
    "            predictions.append(1)\n",
    "        else:\n",
    "            predictions.append(0)\n",
    "    return predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.6: Implementing neural network for a given configuration\n",
    "Stochastic_NN() function runs the neural network using stochastic gradient descent. Method is configured to support single hidden layer with different number of neurons and one output layer with single neuron (shallow network)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 3.6: Implementing neural network for a given configuration\n",
    "def Stochastic_NN(X, y, alpha, epochs, layers=2, hidden_layer_neurons=3):\n",
    "    '''\n",
    "        Method to run shallow neural network using stochastic gradient descent\n",
    "        \n",
    "        X: training data-set of fixed samples\n",
    "        y: labels for each sample of X\n",
    "        epochs: total number of epochs\n",
    "        layers: total number of layers of neural network, including output layer\n",
    "        hidden_layer_neurons: number of neurons in hidden layer\n",
    "    '''\n",
    "    # Getting number of samples and number of features in training data-set X\n",
    "    (nsamples, nattributes) = np.shape(X)\n",
    "    # Initialising threshold to small value to check convergence\n",
    "    threshold = 1e-6\n",
    "    # Setting the number of neurons in hidden and output layers\n",
    "    layer_neurons = []\n",
    "    layer_neurons.append(hidden_layer_neurons)    \n",
    "    layer_neurons.append(1)\n",
    "    # Setting the number of neurons for input layer\n",
    "    layer_neurons.insert(0, nattributes)\n",
    "    # Initialising weights and biases for each layer to small random values\n",
    "    weights = []\n",
    "    biases = []\n",
    "    for i in range(1, layers + 1):\n",
    "        weights.append(np.random.rand(layer_neurons[i], layer_neurons[i - 1]))\n",
    "        biases.append(np.array(np.random.rand(layer_neurons[i])))\n",
    "    \n",
    "    # Initialising cost value to 0\n",
    "    J_prev = 0\n",
    "    \n",
    "    for epoch in range(epochs):\n",
    "        # Randomly selecting a sample from X\n",
    "        idx = np.random.choice(nsamples, 1, replace=False)\n",
    "        random_X = X[idx[0]]\n",
    "        random_y = y[idx[0]]\n",
    "        # Implementing forward pass\n",
    "        sigmas, activations = ForwardPropagation(random_X, weights, biases)\n",
    "        # Getting the activation in output neuron\n",
    "        y_hat = activations[len(activations)-1][0]\n",
    "        # Calculating cost\n",
    "        J_curr = -1 * ((random_y * np.log(y_hat)) + ((1 - random_y) * np.log(1 - y_hat)))        \n",
    "        # Checking if convergence reached\n",
    "        if np.absolute(J_curr - J_prev) < threshold:\n",
    "            break\n",
    "        else:\n",
    "            J_prev = J_curr        \n",
    "        # Implementing backward pass\n",
    "        delta_weights, delta_biases = BackPropagation(random_y, activations, sigmas, weights)\n",
    "        # Updating weights and biases\n",
    "        weights, biases = SGD_Parameter_Update(alpha, weights, biases, delta_weights, delta_biases)\n",
    "        # Printing loss in intervals of epochs\n",
    "        if epoch % (epochs/10) == 0:\n",
    "            print(f\"Epoch {epoch} ==> Loss: {J_curr}, Error: {random_y - y_hat}\")\n",
    "    return weights, biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.7: Testing Shallow Network on blobs300 Data-Set.\n",
    "The best parameters found for the given data-set for neural network with one hidden layer, after testing with different hyperparameters are,\n",
    "> 1. alpha: 0.1\n",
    "> 2. epochs: 1000\n",
    "> 3. number of neurons in hidden layer: 3\n",
    "\n",
    "> The neural network is able to handle linearly separable data as we get 99% accuracy on training data and 100% accuracy on test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0 ==> Loss: 1.483827904777161, Error: -0.7732320213659832\n",
      "Epoch 100 ==> Loss: 0.7303865490027355, Error: 0.5182772553505124\n",
      "Epoch 200 ==> Loss: 0.40916261175407, Error: -0.33579378419773886\n",
      "Epoch 300 ==> Loss: 0.21466863958574298, Error: -0.19319125963303552\n",
      "Epoch 400 ==> Loss: 0.1522224633927398, Error: -0.1412027914520617\n",
      "Epoch 500 ==> Loss: 0.145795564586872, Error: -0.13566561431883475\n",
      "Epoch 600 ==> Loss: 0.11590567411542381, Error: -0.10944077787782727\n",
      "Epoch 700 ==> Loss: 0.05489518354632233, Error: 0.053415639630393086\n",
      "Epoch 800 ==> Loss: 0.08879516656372713, Error: -0.08496701626977587\n",
      "Epoch 900 ==> Loss: 0.04806153973362341, Error: 0.04692486671707152\n",
      "\n",
      "EVALUATION ON TRAINING DATA:\n",
      "Training Accuracy: 99.0%\n",
      "\n",
      "PREDICTIONS FOR TESTING DATA:\n",
      "Testing Accuracy: 100.0%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEICAYAAACj2qi6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZgU1fXw8e/pnp6elR1FkMVdEBUVFfc94r7v0agxxMRXY1yiiVtM0BhN1OQXE0VjYty3aFyj4oa4g6KiaERQZAeHbWZ67/P+cWugZ+iBnpnqqVnO53l4mK7qvvdUdXWdqlu3bomqYowxxhQiFHQAxhhjOg9LGsYYYwpmScMYY0zBLGkYY4wpmCUNY4wxBbOkYYwxpmAdImmISFREakVkYNCx+EVEForIni14/0gR+dhbD+N8qP8GEbmrrfGJyFgRmdnWeNrT+pa9oxKRfUXkK28bGFvkurYWkXQrP1smIioiG3uvHxKRK/2NsGNr6e+7o2rNb2WdScPbeBv+ZUUklvP6tDYE+o6IfL/htaomVLVKVee3tsx11HWDiKS8mJeLyGQRGe13PQXEsL4v5pfAs956mNAecRWTiIwSkQ9EZJmI1IjICyKyZc78kIjc4s1fKiLjm3x+ZxGZJiL1IvKeiIwsUpztssMrsJ7rgBu9beC/PtQ5TESe9NbvCu+g5NS2ltvevAMXFZE/Npk+RURODiqu5njfdVZEts+ZNlJE4gV+/lwRmVi8CNtmnUnD23irVLUKmAMckTPt/vYJ0Rf3eMvQH3gHeDjgePIZCnzamg+KSInPsfhhDnAM0AfYAJgI3Jcz/3zgIGAEsCNwkoicCSAi5cB/gAlAb+BR4IkOupx+8nsbeBD4AhgM9APOApa2OrpgrQR+JCKDil2RT9vZMuC3PpRTNK1eTlUt6B/wNXBgk2lh4CpgFm5jvB/o5c2rBB4CaoDlwLu4HcAfgQwQB2q912WAAht7n30IuBV4AVgFvAkMzan3MOBLr9xbcYng+83EfQNwV87rHb26qnOmHQN87JX3BjAiZ95VwALcRjsD2Csnxitz3jcWmJnzeiGwJ3A0kARS3vK+lyfGt5qskyG4ne0DwBJgNvALQLz3nwu8AtyG2zivbGa5HwQe99bh+8A2TePz/i73yloAzAVuAiK5ywVc632Xs4ATcso5Cvjcq+Nb4II8sZQAFwE1OdM+AM7IeX0e8Jr395HArJx5IS/efZv5jjf3tpFVwPPAHQ3fuVf348Ai7/t9FdjKm3eB970kvPX+qDf9am+drwKmA4fl1LU1MBlY4X03/8qZN9L7XpZ528rR66qnyTLMBbJAPVDrTRsCPOet9/8BP2jy/T6AOwBaRZPtHxCvzq2bWWdbA2lcIpnrLculOfP3wP1mVwDzgVuAEm9evt9r7m/hPOAr4Dvg38CG3vTfAzflbHMJ4Dfe6x647b8qT6wN2+CdwN9ypk8BTs55/WNckqwBngUG5S5rkzJX7zPI83vyPvOaV9YS4B4a7zNW/37yxPsQcCNun7hrzrYRz3lPH+BfXjnfAtfgtvMdvPWQ9raVhcBwYHHOZ+8D5uS8fgw4tzXbDDn7R6AU91t5sOG7zrt8zc3IsyK+Zu2kcTluJzvQ25D+CfzDm/czb2HKcT/cnYHKpl/YOjbCxbgdfMQr55/evAHeyjzcm/cL3I9jvUkDiOI2/gWs2QGP8V7vhEuC47yVXQJsj9tJboj7EW4KbNLMDyVv0mgawzrWb9N18gjuCLsKt1OcDZyWs5GngR95MZc3s9xJ3A44gvshfAGE88R3o/c99vOW9X3gipzlSgO/8zaqA3E7tob18B2wi/d3X2CHnBiiuB11xvuXu1OKA9vnvN4TWOL9/UvgiSbLMxE4r5l190FOfAd48eUmjR9467EM+BvwTpMf+JVNyjsJ2Aj3Iz4d9wPr5817ArjE2x7KgT1ydnoLgNO872Rn3A938+bqybMcjXZEuJ32Ld56HO2V11DfDbid7qFenPm2gcnA68CJeL+tnHlb435zt3nrZWdve9nUm7+LNy0MbIbbaTfsmJpNGl48C4HtvPdNAF7Kmfe+9/f+uMTyes68d5tZLw1JY7D3XTRse6uTBnAyLlFvidvexwOv5izr+pJGo9+T95n9vW1qgPf+G5r7rpqU/RDu9/YLYKI3rWnSeB74P6ACt619iLeD9+KZ2KTMRXgHfbh9weyc9bAIGN6abcabdhfuIP9F7/sKrWs7beuF8B8Dl6vqfFWN445GTxKRhqOc/sBmqppW1fdVta4FZT+iqh+oagqXHUd504/EbXjPePP+gDs6WJfTRWQ5bmdyGnC8emvRW4a/qOpUVc2ou54QxSWRNG7FjsDtbGep6uwWLEOriEgUOA64TFVrVXUm7ozq9Jy3zVLVO72YY80U9ZaqPuWtpxtwSWHHPO87DbhGVZeq6iLcDy63rjRwraomVXUibgd+fM68bUSkWlW/U9UPGz6k7lpVL6AX8HPgI2/5Irh1vCKnjhVAtfd3VZN5Teev5l0nGZET38vA6usB3rZ3j7ceG7bRXUSkLO8ac595WFUXqGpWVe8F5uG2B3Db9TBggKrGVPVNb/oxwHRVvd/7Tt4HnsZ9jy0mIlvgDlp+5a3HKbij3dzv5XVVfc6LM982cDTwnrfMc7xrADs0ec81qhr34v0ct7NHVd/zfrMZVf0Kt2PZp4DQTwMmqOrH3vr+BXCAiAzAJbHtRKQHsDcugW/pfRf74BJcs1T1W+AfwK/zzP4xMF5V/+dt79cCe4rIhgXEDE1+T6r6uaq+4m1TC3G/v0KWP9dfcL+NfXMnishQ3PJfpKr1qroA+DMu8TVnErCPiAzDJc6nvdfDAVR1Rhu2mV7AS8BHqjpOVbPrWqhWJw0vMQwGnvMuMC/HZcsQ7ojz77iN4DERmSsi14tIuAVVLMz5ux63IwF3VvNtwwxvAeetp6x7vZ3XRrijm1E584YCv2pYBm85+uNObT/FnU1dBywWkftbsBG2xQDcepyTM+0bILc991vWL3c9pXHNDI16qHnf4wCv/ObqWuLtAHLnN5RzNG7HOEdEXsnXyUBVVwG3Aw+LSG/vR53AHZ036IH7MYA7k+zRuJRG83MNbCa+huUrEZE/iMgsEVmJ2zEKbhvNS0R+6F00btgeNsclXHDJrwL40HtPQ4eOocDeTbaj43DbXGs0LFduMmjRNuAdBFyqqsNx3/H/cM1FDTKqmnuNY/XvTERGiMjzIrLIW29Xs2YdrC/u1etfVZfjmnYHqepK4BPcWeXeuKbC94FdKSBpeK4DjhGRrZtMHwrcnrPul+AOaDYuoExosi5FZKCIPCoi87zlv4vCln81Va0HrvdibhprGbAkJ94/4c7ym/M6sC9uvb2Oazrbx/s3yXtPa7eZvXBnaH9Y70LRhqThHanPA/ZX1V45/8q8jTWhqler6ta4BT2BNZlUmyu3AAvI2RBEJETjlbKumBfjjkh+JyING8C3wNVNlqFCVf/tfeYeVd0d1zRVhjsKB6jD7TwaDFhX1YXEl2Mhrn17SM60ITROjoWUObjhDy9hD8QljjWFuO9xIW5Dbq6ufk2OzIc0lKOqb6vq4bgN/kVce2g+IdyZQsN6+gx3VNRge9ZcBP40d573HY8k/0XiBc3E1+As4HvAfkBPXLMDuMQBTdajd+byf7hmyj7ewcbMhver6jxVPRuXDC4A7haRIbjt6MUm21GVql6Yr54CzAf6e50CcperpdsAXtyLgZuBYSJSWcBH7sQ1+22mqj2A37Bmna3LfHK2JRHpiUv4DXG/jusAMRyY5r0+DHcgN7mA5VgE/NWLJ9e3wJlN1n+5qk7F/VbD3hl8g6a/16br8ibvcyO95T+Hwpa/qTtw+6tDm8RaC/TOibWHqja0AuT7Xl/HJYiGpDHJ+zs32bZ2m3kad6YzMWe/2Ky2Nk/dDtwgIoMBRGQDETnC+/tA72glhDvSSOPatcG1wW3ayjqfAnYVkUO9q/8X4S6wF0RVP8at8Iu9SROA80VktDhVInKkiFR48e/jbWwx71/DMkwDDheRXl6PjvPXUe0iYBPvqL6QGBO4tvPrRaRSRDbDXSO6b92fXMvuInK41xz0C9z1hw/yvO9B4BoR6SsiGwBXNKkrAlwlIqUisj/uR/+4F9vJXnNDCncmkAEQkUNEZDsRCXs7jltxG/WXXpn/Ai4VkQHe9nMh7poYuFPlcnFdD6O4o/s68u9U/oc7e2iIbz9cG3iDatz1k+9w7bbjm3y+6bZYhUvYS4CQiJyLO9PAW66TRGSgl2yXe5PTwJPADt78iBfLGFnTzbil2/xMXOeM8eLuY9oRd22m4F6L3hnWiJzv4FxcE1ohzcTVwApVrRWRbXDt/YV4ENfLaaSXyH8PvOI18YDbwf0Q+EBVM7gj5nOBT1W1aZNkc27EbYO56/N24EoR2QpARHqLSEPT4Hzc93maty5+yvoPNKtxO/aV3kHBRQXG1oiqJnEJ7rKcabNx10huFJFqcd3Pt5A1930sAgZ7v9sG03HXW04AJnlniPW4hNuQNFq9zajqb3D71pdEZJ3707YmjRtx7duviMgqXC+ghmw5CNdtsqEHynO4i7vgLtScIa6P/o0tqdBr/zsFlxmX4rL4J7jmjkLdBJwnIn28NukLcEcEy3E7oVNxGbkc17trKe6Itgp3mg5wN+5LmgM8Q/NH2OAujFUANSLyVoEx/tj7/xtcz467aMEOw/M4cDbums9xwHHeD7Wpq3FH/p/ikuGbuO+2wde4HeNC3HKfpaqzvHlnezGuAM7AbaTgeoc8hjtg+BJ3ljPWayYD9/29jLt4OQ3Xo+ifAN7p9VG4ncly3Bnq0TmfXc3beZ+EO5OowSXH3IT3d9wOYyFuO2maeCYAO3vNBA+p6ge4HdAU3He+ifd3g92AqSJSi+uoME7dNb1lwMG4M5sFuB3VeFzCXauepsvRzHKdiLtesxDX4+VSVX1jfZ/N0QO3I1iB21b7A8cW+NmfA+d4y3kbBXZTV9VncJ0SnsKtgwE0blN/A5e8G5pUpuGS9CQKpKo1uH1I75xpD+KuIfzba06ahksseNv8ObgeSktxZ+BT11PN1bhmtBW4A7jHC40vj3twBy25TsFdS/gct90+zJrmqf/ifnOLRWSutwyKW3fzvbNGcMkihdu/tnmbUdUrcL/JF72DjLwaehB1Wt7ZxkLcPSRvBx2PMcZ0ZR1iGJGW8po+enqnv9fgTtPWd+RgjDGmjTpl0sBdAJqNu5fjAOAYr+3QGGNMEXX65iljjDHtx/exfETka9b0okmrarsODmiMMaZ4ijUA3H5NbhrKq1+/fjps2LAihWCMMV3T1KlTl6pq/yDqDnTU0GHDhjFlypT1v9EYY8xqIvLN+t9VHMW4EK64fr5TxYeHCRljjOk4inGmsYeqzvfuLH5JRD5X1dU37niJZBzAkCFDmivDGGNMB+T7mYZ6T9/z7lp8AjfEcu78Cao6WlVH9+8fSJOcMcaYVvI1aXhjEVU3/I0bKG66n3UYY4wJjt/NUxviHsvZUPYD6sOzjo0xxnQMviYNbxC77df7RtPlpJIpnrtzIi/e8zoiwtiz9+eQH+5PuKQlj1AxxnR0gXa5NV2DqnLFodfz2Ttfkqh3gw1//em3vPPsVH77n8socER4Y0wn0FnHnjIdyIevTGfGezNXJwyARH2Cj16dzox3/hdgZMYYv1nSMG32yRufEa+NrzU9lUjz8aQZAURkjCkWSxqmzfps2ItoRela00vLIvQZ0CuAiIwxxWJJw7TZPiftTii89qYUCofY67hdA4jIGFMsljRMm/XoU80N/72SvgN7U1YVpawyygZD+nHjxKsprypffwHGmE7Dek8ZX4zYbSsemHM733z6LYgwbJvB1mvKmC7IkobxTSgUYpNthwYdhjGmiKx5yhhjTMEsaRhjjCmYJQ1jjDEFs6RhjDGmYJY0jDHGFMyShjHGmIJZl1tjzDol40kmP/Ee82cuZLNRw9jl0B0Ih23I++7KkoYxplmLvlnCBbtfQWxVjHhdwt3tP7Q/t77xW6p6VQYdngmANU8ZY5r1h7NvY/mi5cRq46gqsdo4879cwD+ufDDo0ExALGkYY/JKxBJ88sYMslltND2VTPPqQ28GFJUJmiUNY4zv5n+1kF+OHc/Y0pM4vPI0/njO36hbWR90WMYHljSMMXlFy6Nsu9dwQqHGA09GSkvY96Tdm/3cqmW1nD/ml0yd+DGZdJZELMnL90/i8oPHo6rNfs50DpY0jDHNuvjvP6XXBj0prypDRCivKmPg5gM4+7pTm/3MC/94lUQsieY0a6USab6ePocZ737ZHmGbIrLeU8aYZg0YtgH3zrqNyU+8x4KvFrHp9kPX2+V25rTZJOqTeefNmTGPEWO2LFa4ph1Y0jCmi0glU7z28Fu8/fQUem/Qkz2P2ZWpEz/ik0kzGLTFRpxw8RGtGrq+tKyU/U/Zs+D3b7HDJkx+/F0SsbUTx9ARG7e4ftOxSJBtjKNHj9YpU6YEVr8xXUUynuTne1/NnBlzidclkJCgWSVcEiKTzhIKh4hES7jm8UvZ+eBRRY1l1bJaztzyfFYtq1vdRBWJlrD5qE3401vX2cO5fCAiU1V1dBB12zUNY7qAF/75GnM+cwkDWL2zzqSzAGQzWRL1SW4Zd3vRL0ZX967i/975HaMP2p5wSYhoRZQDT9+HG168qk0JI51KM+XFj3jzyfeoXV7nY8SmJax5ypiALfpmCa888AZ1K2PseuiOjNxz6xbvXF9/9C3i9Yn1vm/5kpXULFxO3416tzbcggzcbADXP3+Fb+XNePdLrjjsejLpDADpZIaf/ulMDvvRQb7VYQpTlKQhImFgCjBPVQ8vRh3GdAWvP/IWN551G9lMlnQqzX/+8jy7Hbkzv7zvghYljsoeFYW9MatUVJe1MtpgJONJfnXIdWudXfztwn8yfNct2XQ7e8RweypW89TPgBlFKtuYLqF+VYybzr6NZCxJOpkGhXhdgrefep93n/2gRWUd+dODKauMrvM9kWgJY44cTXlVeVvCbndTXvyIbCa71vRUMs1/7345gIi6N9+ThohsDBwG3OV32cZ0JdNemU64ZO2uq/G6BC8/8EaLytrpoO054ZIjiUQjlFeXU1ZVRmlFKSWlJVT2rKC0vJRtdt+ai+/6iV/ht5vYqnje6zDZTJba5XaXeXsrRvPUrcAvgOp8M0VkHDAOYMiQIUWo3piOY87n87h//GN8/u6XDNpiI0791bGM3HM4AOGS5o/ZSiItH3r8jGtO5PAfH8Qnb3xOdZ8qtt93BCu/q+Xr6XPYYEg/Bm2+UauXI0ij9h9JOpVZa3pZVRl7HL1LABF1b74mDRE5HFisqlNFZN9871HVCcAEcF1u/azfmI5k9iffcMEeV5KsT5DNKvO/WsTHkz7j8nsvYM9jdmXU/iPzHkGXVUb53g/2bVWdfQb0Zp8Tdlv9uvcGPem9/7atXYSiWDB7EYvnLGWTkUPo0TfvsWUjfTfqzfevOo4Hrn+CZCyBqltH2+y+FWOO2KkdIja5/D7T2AM4UkQOBcqAHiJyn6p+3+d6jOnw7rzsfuK18UbTEvVJbrvgbvY4ehei5VGueewSfn3sTYBrbkGEw358EKP2GxlEyEVVvyrGtcf/gemTPydSWkIqkeLI88Yy7sbT13vR/9RfHcd2+2zD83e9TP2qevY9cQ/2PG5XexhUAIp2c593pnHJunpP2c19pis7us8PqMvT5l5SWsLD8yfQo487yq5dXsfkJ94jtirG6IO3Z/BWg9o71Hbx2xP/yNtPTyGVSK+eFq2I8tNbz+LQcw4IMLLOJ8ib++w+DWOKpFf/nnmTRijkBv5rUNWrkrFn7deeobW7+lUx3n5qCqlkutH0RH2Cx25+2pJGJ1K0O8JV9TW7R8N0ZydfdjRlFY27wUbLSzn47P2JlEYCiioYsdo4EsrfBLWqpradozFtYcOIGFMkB5+1HydedhTRiijl1eVEohH2PmE3fnLzD4IOrd31GdAr70XvUEjY8cCOdaHerJsNWGhMkcXq4iz6egl9Nuq1+jpGd6KqTJ/8OZOfeJenb3+BdDKDZpWS0hLKKqP8dcrv2WiTDYMOs1OxaxrGdGHllWUM22Zw0GEEYsXSlVx6wLUsnL0YBQSh1wY96TuwN6P2G8lxPz+cfgP7BB2maQFLGqZbmvu/+Tx+yzN8/dlcRozZkmN+dqjtvIrglnF38O3n8xrfnLe8jrFn7bfOp/+Zjsuap0y3M33yDC4fex3pZIpMOkuktITSilL+8u4NbLxF57xruiNKJlIc1eP0vHdz99qgJ48utJGGWsuep2FMO7rlx3eQqE+sftZEKpmmfmWMO39xb8CRdS3ZTLbZZ3ck46l2jsb4xZKG6VbqV8WY9+XCtaZrVpn2yvQAIgqeqjJz2mymvTqdWG3Mt3IXfb2YaMXaI++GS0I2/EcnZtc0TLcSiZYQCglrN5hARY/ONWS4HxbMWsSvDr2epfO+IxwOkU5nOfePZ3D4j7/XpnIXz1nCBbtfQf3KxkkoFA7Rs39PfvR7G1mos7IzDdOtREoj7HvKHkSijW+ui1aUcvT5hwQUVfOWLVrOe89/yKyPv/G9bFXl8rHjmTdzAfG6BHUrYyTqE9x+8T189vYXa8Xx8E1P8ufz7uS1h98klVx389JjtzxDMpb/PX945RrrdNCJ2ZmG6XbO/78fUjN/GZ94A+cl4yn2OXF3jr/4iKBDW01VuePie3jqby8SiUbIZjIM3moQ1z//K3r17+lLHf+b8hU1C5evfp54g2QsxZN/+S8jdtsKgM/e/oLLDx5PJp0hGU8x8d5JPHD9v/nTm+ObfaDTF+9/RTqVXmt6eVUZS+bWdNnxtboDSxqm2ymvKueGF65i3swFLJy9mKEjNqbfoL5Bh9XIxPsm8eydE0klUqQS7oh91sffcN3Jt3LTy9f4UsfKmlpCobUbG1SV5YtXrP77+lP/RCxntN5YbZx5Xy7gkT88xQ9+fVLesjcZOZjP3/1yrSfupRIpBm5mN/J1ZtY8ZbqtQZtvxE4Hbd/hEgbA47c8Q7wu0WhaJp3h07e+YJm3Q2+r4btuQTpPM1O0opTdjnS9ORfOXszyJWvXl4ynePXBN5st+/iLjqC0rHETYGlZhJ0O2p4BwzZoY+QmSJY0jOmA8o2OC67nUd0Kfx5xWtWrkjN/e3KjHk7R8lI2HNqfsWfvD7hh3Js2XzWIlDbfULHxlgP5/YtXsen2QwmFhNLyUr535n5c8dCFvsRugmPNU8Z0QGMO34ln7nhxrRvjyirLfG3eOeHiI9lix0158i/Ps2LJSvY8ZhcO/dGBlFe6odv7b9yXwVsNYtYn3zRKHtGKUg4dd+A6yx6x21bc8eEfSCZSlETCeZvCTOdjd4Qb0wEtW7yCn+x4KauW1ZGMJQmFQ0SiJVzx4M/Z7Yj2vRF43swFXLTPNcRq42TTGRBhp+9tx9WPXEy4xJ6cF4Qg7wi3pGFMB7VqWS3P3P4iH0z8hAGbbsCxPzuMTUYOCSSWdCrNe89/yHfzahi+25ZsPmqTQOIwjiUNY4wxBbOh0Y0x3Z6q8vHrn/HVtK/ZaLMN2eWQHaz5qwOypGGMCVysNsalB1zLnBnzSKfSlJSW0KNPNbe+Od7uHu9grDuDMaZVVJUv3p/JlBc/om5l27oB33PNw8z6eA6x2jipRJrYqjhL5n7HH8/+q0/RGr/YmYYxpsXmzVzAL8eOZ/nilUhIyKQy/OjG73PUea0bv+uleyetvvO9QTaT5cNXppOMJyktK/UjbOMDO9MwxrSIqnLZ937LwtlLiNXGqV8ZIxFLcudl9/HpW1+sv4A8mg43klMb2WZuLjTBsKRhjGmRz9+bycqlq9Z6wFIyluKpv/63VWXudeyulEQaX/QWEbbeZQvK8jyTwwTHkoYxpkVql9UiIVlreu5Ahy31w9+dRv/B/Sivcneil1VGqe5TySV3/7RNsRr/2TUNY0yLDB+zJenk2sOeRyui7HnMrq0qs2e/Htz16S1Mfvwd/vfBLAZvOZD9TtmTiuru92Csjs5u7jPGtNjjtz7DP658iGQsgaobi2rgZgP489vXW3NSO+gyN/eJSBkwCYh6ZT+mqv4M/m+M6TCOu/BwttxpM/5z2/OsWLKKvY4bw/fO3NcSRjfgd/NUAthfVWtFJAJMFpHnVfUdn+sxxgRs272Gs+1ew4MOw7QzX5OGurauWu9lxPtn/eWMMaaL8L33lIiERWQasBh4SVXfbTJ/nIhMEZEpS5Ys8bt6Y4wxReR70lDVjKqOAjYGdhGRkU3mT1DV0ao6un///n5Xb4wxpoiKdp+Gqi4HXgPGFqsOY4wx7cvXpCEi/UWkl/d3OXAg8LmfdRhjjAmO372nNgLuEZEwLiE9oqrP+FyHMcZ0SolYgjcef5f5MxeyybZD2O3I0ZREOtc91n73nvoY2MHPMo0xpitY9M0SLtjtV8Rq48Rq45RXldFno978+a3r6NG3OujwCta5UpwxxnQSH77yCXdc8i/mzJhLnwG9iVZEWbZ4BeqN2hurjbPo68Xcefl9XHznTwKOtnA2YKExxvjskzdmcNURN/DVtK9JJdIs+mYJc2bMXZ0wGqRTGSY9+nZAUbaOJQ1jjPHZ3Vc8QCKWLOi9ImuPGNyRWdIwxvDKA29wxhb/j0PLT2XcqIt5/4VpQYfUqX396bcFva+ktIR9T9qjyNH4y5KGMd3c83e/zM3jbmfBV4tIJVLM/ngO1x57E1Ne/Cjo0DqtgZsNyD9DoKwqSigcory6jEGbD+CcG05r3+DayC6EG9ONqSp3/+pBEvWNm1ISsSR//+X9jP7e9gFF1rmd+duTufa4mxqt12hFlOMvPpxNtx3KvC8Xsul2Qxg9dhThcHgdJXU8ljSM6cbidXFW1dTmnfftF/PaOZquY+eDR3HZPedz+yX3sGTOd1T2rOCky47mpF8c1emuYTRlScOYbixaEaW8qoza5XVrzdtw6AYBRNR17HXcGPY6bgypZIqSSEmnTxYN7JqGMd1YKBTi1CuOpayy8cOTohWlnDX+5ICi6loipZEukzDAzjSM6YAGLMkAABWXSURBVPaOv+gIQuEQ949/nNrldfQZ0Itzfv/9Vj/v23Rt9oxwYwzgLoqnkmkipV2nKaWr6jLPCDfGdF4iQmk0EnQYpoOzaxrGGGMKZknDGGNMwSxpGGOMKZglDWNM0WWzWeL1CYLseGP8YUnDGFM02WyWB65/nGP7nsVRPc/g1CHn8spDk4MOy7SBJQ1jTNHcP/5xHrj+CepW1JPNZFk6r4abz/kb7z47NejQTCtZ0jDGFEUmneHRPz5Foj7RaHqiPsk/r344oKhMW1nSMMYURe3yOtLJTN55C2YvaudojF8saRhjiqKqdyXRitK884aOGNzO0Ri/WNIwxhRFOBzmzN+cRLSiyWCI5aWc87vO9eAhs4YNI2KMKZqjzjuEyp6V3PubR/luXg1Dt9mYcTeewbZ7DQ86NNNKNmChMcZ0MkEOWGjNU8YYYwrma9IQkcEi8qqIzBCRT0XkZ36Wb4wxJlh+X9NIAxer6gciUg1MFZGXVPUzn+sxxhgTAF/PNFR1gap+4P29CpgBDPKzDmOMMcEp2jUNERkG7AC822T6OBGZIiJTlixZUqzqjTHGFEFRkoaIVAGPAxeq6srceao6QVVHq+ro/v37F6N6Y4wxReJ70hCRCC5h3K+q//a7fGOMMcHxu/eUAH8HZqjqzX6WbYwxJnh+n2nsAZwO7C8i07x/h/pchzHGmID42uVWVScD4meZxhhjOg67I9wYY0zBLGkYY4wpmCUNY4wxBbOkYYwxpmCWNIwxxhTMkoYxxpiCWdIwxhhTMEsaxhhjCmZJwxhjTMEsaRhjjCmYJQ1jjDEFs6RhjDGmYJY0jDHGFMyShjHGmIL5OjS6MZ2dagLiL0N2AUS2g8ho3LPFjDFgScOY1TT9NVpzCmgcNAFSCiUjoc/diJQGHZ4xHYI1Txnj0eUXQrYGtA5Ig9ZD6iO07q6gQzOmw7CkYQygmSWQnglokzkJiP07iJCM6ZAsaRgDQJZmn1Ss2XaNxJiOzJKGMYCEN4TwxnnmRKH8iHaPx5iOypJGJ6fZOjQxGU1+iLbDEbGqosmPyK66meyqv6LpOUWvs71Ir5tBqkHKvQkVULIpUjku2MCM6UCs91Qnlq1/FFb+FqQEULfD63M3UrJ5UepTVXTl1RB7CogDYbTub2iPawhVHF+UOtuTRIZD/1ch/iyamYdERkF0X0TW/zPRbD1IaUHvNaYzsy28k9LUpy5hEF9z7Vbr0JofQP9JiIT9rzT5HsSfAmLehLT7t/JatOwAJNS7VcWqKpD1LWbNLEDrH4D0VxDZCak4AQn1KOizEuoBFac0d3Vj7boSb6Irfw2ZuUAJWn480uOX1kXXdFnWPNVJaf1DQDLfDEi+X5w648+5exjWEobEGy0vT2NkV1yDLtoOXTSC7HcnoqkZbYsx+RG69BCouxsSE6H2T+jSsWhmUZvKzVtX6jN02U8g8w2QwfW0ehxdcZnvdRnTUVjS6Kyy3+F6/OShK4pUaZi8PYxEvHkto8vO87qzJgCF1DS05lQ0s6DVEeqKy13iJOVNiUN2GVp7c6vLzFuPJtEVv3blNxKH+EQ0s9TX+ozpKCxpdFJSduCaC7a5NA2lOxenzvKjgDzNLpqB6D4tKkvTsyA5BZcwcmck0fr7WhWfZpdDJt+F+QzEX21VmfnrqUGXjoX0R/nfIBHIzPOtPmM6El+ThojcLSKLRWS6n+WaPMoOh/DmQG7iKIeq85BQn6JUKaXbQ+U5QNT9k3KgDHrejISqWlZYerZ3Ab+pFLS6iWod1xHyJdhW0pW/h8xC1r4RsOENSSgZ5lt9xnQkfl8I/yfwF+BfPpdrmhAphb4PovVPQuI5kB5IxalIdExR6w1VX4CWHw2J10DKoOyg1l0AL9kcNJVnRilEtm1VbBKqQKN7eddXcssug4pTWlVmXomXcJ0A8imDipOQUE//6jOmA/E1aajqJBEZ5meZpnkipUjliVB5YvvWWzIESs5oYxlD0egekHiTNU1UAhJFKk5rfbk9f4fWnOldnBbXXBfdB6k8p03xNraOE/SqC5HKM32sy5iOpd273IrIOGAcwJAhQ9q7etOBSK8/o6tuhdgjoDEo3RXpcQUS3qD1ZYZ6Q98nIf0JpOdCZDhSsomPUeOaBmOP0vhsJgylexKqOtvfuozpYMT1kfexQHem8Yyqjlzfe0ePHq1TpkzxtX5jik2ztWjNae6iuybdEOqh3kifh9qU8IwplIhMVdXRQdRtN/f5QDPz3cXb8CAksnXQ4Zgik1AV9H0Ckm9D+nMID3NNYHY3uOkGbCtvA9UMuvIKiD3rullqBi3ZAulzFxLqFXR4pohEQhDdw/0zphvxu8vtg8DbwFYiMldEfuhn+R2N1t8HseeBBGgtEIP0DHTFL4MOzRhjisLv3lM+9mtcRz2Jt9G6v0N2EZTugVSeg4T7tUfVjdX/izXjMDVIQWISmq1DQpXtH5MxxhRRp2ueytY/DCuvY/XwDelZaOxJ6Pc0Eu7fvsFofTMzxBujyZKGMaZr6VTDiKgmYNUNNB7vJwW6Cq27s/0DKt2bvGMuhQdAke7KNsaYIHWuM430V83MSLVqlNW2kuqfo4nXQetwN6iVgESQntcjUujg2sFQVUhNgcy3UDLcPUuik9D0bEi8CpRA2cHuqXvGmHbRuZJGqI+7wzef9m6aAiQ8APo/74YpT74PJZsgFWcgJUPbPZaW0GwNWvN9yMxvmICW7oz0/luHfw5EtvYvUHsHbtynEKy6Ce0xnlDFUUGHZky30KmapyQ8AEp3BCJN5pT7PExE4STUm1DVTwj1uZtQj6s6fMIAXO+u9NfumozWA3FIvo/W/i3o0NZJU59B7QTcWV0S10yZgJVXotmaYIMzppvoVEkDQHr9CSI74kZZrXKjl1ZfhET3Djq0TkE1DonJrD3gXtwbGqPj0viz5H3wlIQg/kr7xJCtIbvqFrJLjyW77CdokR54ZUxH1bmapwAJ9UL63uvuws4uhZItEB+Hve7yNE3zQ3on8k9vaRXJD9DY04Ag5UcgpTv4Ui7rHPLG3+Fw8taQrUGXHgHZFUAS0tPd4157XN0lnpFuTCE63ZlGAwkPRCLbWcJoIQlVQckWeeaUQNn+bS4/u/J36LKzIPYAxO5Ha84ku/LGNpcLIOWHkP8hUFmI7udLHevi7g3yEsZqcVh1nevZZ0w30GmThmk96fk7kErcw5QAyiHUB6m6pE3lauoLqH/QjViLev9iUH8fmp7ZtqABiWwLFacDZbiuzhEgCj2ubJ+bO+Ovkrd5DAEfls+YzqDTNU+ZtpPICOj3Ehp7xD1BLzIKKT+q5U/fayrxKo2HC2+Qdjvcqs3bVj4Q6nEpWn4kJF5x431FxyIlG6/1Pk1/Dcn3XI+76N7+9AoL94NMnuSgKWjNg6iM6YQsaXRXUoFE94Hy4/y7z0GiuDOATJMZYW+eT9VEtoLIVnnnqSq68iqI/QcIuYvklEKfe5HIlm2rt/JsNPkRjYeOKYHISCQ8sE1lG9NZWPNUN5StnYAuHoPWnI4uOZBszTg0W9v2gssOAZq5qbHs4LaXX4j4cxB7GtctN+ZuvNRl6LJzaeuzYyS6L1RfAJS5nnuUuYTR+7a2x21MJ2FJo5vR+H+h9jYg7o3Mm4DkW76MzCvhAdDjOty1kgrc2FtR6HlDu921rfUPsvYgkoDWQPqLNpcfqvwhssFbSO87kX5PE+r7CGJDxphuxJqnuhk3RlfTnWoSEq+i2RVIqGebyg9VHImW7Q2JSW5CdJ82l9kizfZiEt+6FEuoCkp38qUsYzobSxrdTWZpMzPCrjupDzt4CfWC8iPbXE6rlB8Oq76g8aCW4K49bBNERMZ0KdY81d2UjiHvyLxSCl3gYq5UnOwukkuFNyUClCG9brLHsRrjA/sVdTNSdT6aeNkbc6phKJFyqL6qS+xURaLQ50FIvIwm3oBQf6TieCQ8KOjQjOkSOv9ewrSIlGwM/Z5GaydA8l0ID0KqxiGlOwcdmm9EvCHT26vHljHdiCWNbkjCGyE9rwk6DOMjzSyExJsQqoDovja8jikaSxrGdHLuGSO3g4RxlykFet/Rpc4eTcdhF8JN0akq2fpHyS7el+zCEWSXHuauN5g20+RUqL0TSLoxv7QOtNa7mTHfOFnGtI0ljSJTVTQ5hezK68mu+iOa+jLokNqd1t8LK8dDdj6QhvSX6LLz0MRbQYfW6WnsMdbuXgygkLT1a/xnzVNFtGYcpKdxP+wQWncPWn0Jocozgg6vXahmoPbPrH1DYRytvQWJ7h5EWF2Hxin281GMyWVnGsWUmuIljIahwjO45y/ciGYWBxtbe9GV3o4tj/RX7RtLFyRlh+CGbGlCU1C6W7vHY7o+SxpFpLEXyNt0IOE1w2x0ddLD3TiYT3hIUapUjZOt+xfZ704mW3M2Gn+5zYMVdljRAyG6W87NjGGgDHpcjYR6BBmZ6aKseaqYpBQ36muTHZaG3LMgugGRMFp5rjdIYm4TVRlS/XPf61NNot+d4p3FuIStyalQcSrS4zLf6wuaSAh63QbJyWh8IoSqkfJjkJK2P7vEmHx8P9MQkbEi8oWIzBSRy/0uvzOR8iPJ+3hSMhBt+6NVOwup/BFUXwjiPagovDH0vAmJ7uOu+2jT52+0Qfx5SM+i8RleDOrvRTML/KunAxEJIdG9CfX8DaHqSy1hmKLyNWmISBi4DTgEGAGcIiIj/KyjM5HI1lD9M9xQ4eW4tucypNetSKg62ODakYgQqjyL0IbvIhvOINT/FSS6B9nll6KLRqKLtiH73am+PBJWE6+Rd2h0IpCc2ubyjenu/G6e2gWYqaqzAETkIeAo4DOf6+k0QpU/RMsOg8TrrrkqekCHbGvWbB1IxJ/Hoq6DSNidXdScCenPWf142NRU9LuToP9LbXs+Rag/eZ8eKECoV+vLNcYA/jdPDQK+zXk915u2moiME5EpIjJlyZIlPlffMUl4AFJxkmtr7mAJQ1OfkF16JLp4NLpoB7LLLkCzK4pbafoTSH8J5N58pqBJtP6RNhUtFSfiRrZtNNVdKC4d06ayjTH+J418z/psdBVYVSeo6mhVHd2/f3+fqzctoZmFaM3p3hF/Bki50WFrzi5ub6P0bJB8m0rCi6X1pGRz6Pl7kEr3SFapgPDGSJ97usQovsYEze9f0VxgcM7rjYH5PtdhfKL1D4Cmm0xNQWYmpD+FyMjiVFyyBeRNSmVQsm2biw+VH4KWHQCpT0DKoWQ4kjdJtR+NT0Rr/wSZeVCyOVJ9CVK6S6AxGdMafp9pvA9sISKbiGscPxl4yuc6jF/SM2ncRNQgDJlv80z3h0RGQGR7XAeBBiGQcqTieH/qkFKkdCckMiLwhJGtfwpdfpF7RrnWQmoaWnMOmngn0LiMaQ1fk4aqpoH/B7wAzAAeUdVP/azD+CgyisY7bo+moGSrolYtfSZAxckg1S6G6H5I38fa93ni7UBVofZG1r7JM46uujGIkIxpE98beVX1OeA5v8s1/pOKE9G6u12SIOtNLYPonkjJpsWtW8qQHldAjyuKWk/w4pD9Lv8sH7oYG9PebBiRbkxCvZB+/4ayQ91F41B/qPwR0uvWoEPrQqI5Q3w0ER7QvqEY4wPrTtLNSXgg0uvmoMPoskRCaOU4qPure97FamVQeX5gcRnTWpY0jCkyqRyHkoG6O0GTrjtw1UWEKo4IOjRjWsyShjFFJiJI1U/Ryh+73lNS7QYaNKYTsqRhTDsRCYN0rd5hpvuxwx1jjDEFs6RhjDGmYJY0jDHGFMyShjHGmIJZ0jDGGFMwSxrGGGMKZknDGGNMwSxpGGOMKZglDWOMMQWzO8KNaQea+Q4S/4VsHUT3QiLDgw7JmFaxpGFMkWn8VXT5z7xXaaj9C1p+DNLj14E/VdCYlrLmKWOKSDWGrvg57sl9cSDt/o8/Cck3gw3OmFawpGFMMSXeIe/PTGNo7Il2D8eYtrKkYUxRaSvnGdMxWdIwpphKxwCZPDPKkfKj2jsaY9rMkoYxRSShCujxB6AMiOJ+cuVQfhiU7h1scMa0gvWeMuukGgdCiJQGHUqnFSo/CC19GeLPgdZBdG8kMjLosIxpFUsaJi9NfY6uuALSnwGCRg9Aev4WCfUKOrROScL9ofIHQYdhTJtZ0jBr0cxStOZU9zzrBolX0Jo50PdJu7fAmG7MrmmYtWjsEdBUk6kpyHwDqWmBxGSM6Rh8SxoicoKIfCoiWREZ7Ve5JgDpL4FE/nmZb9o1FGNMx+LnmcZ04Fhgko9lmiBEtgfK156uWSjZqt3DMcZ0HL4lDVWdoapf+FWeCY6UHwehSiCcMzUKpTvZQHvGdHPtfk1DRMaJyBQRmbJkyZL2rt4UQELVSN9/Q9lYkEqQPlB5JtL79qBDM8YErEW9p0RkIjAgz6wrVPU/hZShqhOACQCjR4+2cRQ6KAkPQHrdEnQYxpgOpkVJQ1UPLFYgxhhjOj7rcmuMMaZgfna5PUZE5gK7Ac+KyAt+lW2MMaZj8O2OcFV9ArAHBBhjTBdmzVPGGGMKZknDGGNMwUQ1uF6vIrIEKNa4FP2ApUUqu7OxddGYrY81bF001lnWx1BV7R9ExYEmjWISkSmqamNgYeuiKVsfa9i6aMzWx/pZ85QxxpiCWdIwxhhTsK6cNCYEHUAHYuuiMVsfa9i6aMzWx3p02Wsaxhhj/NeVzzSMMcb4zJKGMcaYgnXppGGPoAURGSsiX4jITBG5POh4giQid4vIYhGZHnQsQRORwSLyqojM8H4jPws6pqCISJmIvCciH3nr4tqgY+rIunTSoJs/glZEwsBtwCHACOAUERkRbFSB+icwNuggOog0cLGqDgfGAOd1420jAeyvqtsDo4CxIjIm4Jg6rC6dNOwRtOwCzFTVWaqaBB4Cjgo4psCo6iSgJug4OgJVXaCqH3h/rwJmAIOCjSoY6tR6LyPeP+sh1IwunTQMg4Bvc17PpZvuGEzzRGQYsAPwbrCRBEdEwiIyDVgMvKSq3XZdrI9vQ6MHxY9H0HZhkmeaHUGZ1USkCngcuFBVVwYdT1BUNQOMEpFewBMiMlJVu/21r3w6fdKwR9Cu01xgcM7rjYH5AcViOhgRieASxv2q+u+g4+kIVHW5iLyGu/ZlSSMPa57q2t4HthCRTUSkFDgZeCrgmEwHICIC/B2Yoao3Bx1PkESkv3eGgYiUAwcCnwcbVcfVpZNGd38Eraqmgf8HvIC70PmIqn4abFTBEZEHgbeBrURkroj8MOiYArQHcDqwv4hM8/4dGnRQAdkIeFVEPsYdaL2kqs8EHFOHZcOIGGOMKViXPtMwxhjjL0saxhhjCmZJwxhjTMEsaRhjjCmYJQ1jjDEFs6RhjDGmYJY0jDHGFOz/A/6ljUOZqN/OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Part 3.7: Testing shallow network on blobs300 data-set.\n",
    "np.random.seed(200)\n",
    "file = 'blobs300.csv'\n",
    "# Training shallow neural network for blobs300 data-set, using the train data calculated in Part 2\n",
    "# with best parameters found.\n",
    "w_300, b_300 = Stochastic_NN(X_train[file], y_train[file], 0.1, 1000)\n",
    "\n",
    "# Evaluating the model on training data\n",
    "print(\"\\nEVALUATION ON TRAINING DATA:\")\n",
    "training_predictions_300 = predictNN(X_train[file], w_300, b_300)\n",
    "print(f\"Training Accuracy: {round(accuracy_score(y_train[file], training_predictions_300), 2) * 100}%\")\n",
    "\n",
    "# Using the model to predict classes of test data, using the test data calculated in Part 2\n",
    "print(\"\\nPREDICTIONS FOR TESTING DATA:\")\n",
    "predictions_300 = predictNN(X_test[file], w_300, b_300)\n",
    "print(f\"Testing Accuracy: {round(accuracy_score(y_test[file], predictions_300), 4) * 100}%\")\n",
    "\n",
    "plt.figure()\n",
    "plt.scatter(X_test[file][:, 0], X_test[file][:, 1], c=predictions_300) \n",
    "plt.title(f\"Testing Result for blobs300 dataset for Shallow Neural Network\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 3.8: Testing Shallow Network on circles600 Data-Set.\n",
    "The best parameters found for the given data-set for neural network with one hidden layer, after testing with different hyperparameters are,\n",
    "> 1. alpha: 0.5\n",
    "> 2. epochs: 5000\n",
    "> 3. number of neurons in hidden layer: 3\n",
    "\n",
    "> The neural network increases the accuracy from 65.56% in Part 2 to 97.78% for random seed 200, the data-set also gives 100% accuracy for different splits of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0 ==> Loss: 2.9354313213618783, Error: -0.946892192166351\n",
      "Epoch 500 ==> Loss: 1.0372935486156112, Error: -0.645587414452878\n",
      "Epoch 1000 ==> Loss: 0.4116173690166781, Error: -0.3374222496683901\n",
      "Epoch 1500 ==> Loss: 1.6551694923612907, Error: -0.808940330193622\n",
      "Epoch 2000 ==> Loss: 0.1008993101048937, Error: -0.0959759456094755\n",
      "Epoch 2500 ==> Loss: 0.3526855111393095, Error: 0.2972018191744421\n",
      "Epoch 3000 ==> Loss: 0.3068637662715999, Error: 0.26424917184157426\n",
      "Epoch 3500 ==> Loss: 1.387013142916528, Error: -0.7501796308837209\n",
      "Epoch 4000 ==> Loss: 0.1859814281851733, Error: 0.16971098518741912\n",
      "Epoch 4500 ==> Loss: 0.011382931525052572, Error: 0.0113183910786121\n",
      "\n",
      "EVALUATION ON TRAINING DATA:\n",
      "Training Accuracy: 99.52%\n",
      "\n",
      "PREDICTIONS FOR TESTING DATA:\n",
      "Accuracy: 97.78%\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEICAYAAAB4YQKYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOydZ5hURdaA39Pd02kCDFlEghgRRQVzQMWcc3bVNevqKua8BlzjfqhrwhxBxbBmESNiBEVUVERUomSGCZ37fD/qztAz0z2xZ3pCvc8zz3TfUHXu7bp1bp1z6pSoKhaLxWKxNBdXrgWwWCwWS8fAKhSLxWKxZAWrUCwWi8WSFaxCsVgsFktWsArFYrFYLFnBKhSLxWKxZIU2r1BExCciZSLSN9eyZAsR+UtEdm7E8UNFZKZzH87MQv23isgjzZVPRPYVkTnNlac1qe/a2yoispuI/Oa0gX1buK5NRCTexHP9IqIi0s/5PkFErsmuhG2bxj7fbZWmPCtNVihOw678S4pIKOX7Cc0o9wsRObHyu6pGVLVAVRc1tcw66rpVRGKOzKtF5FMRGZHtehogQ30/2pXAm859GNcacrUkIrKliHwjIqtEZKWIvCsiG6Xsd4nI/zn7l4vIzTXO30ZEZohIhYh8JSJDW0jOVukMG1jPGOB2pw28k4U6B4rIq879LXFeWI5vbrmtjfNSoyJyV43t00Tk2FzJlQnnt06KyLCUbUNFJNzA888WkcktJ2HzaLJCcRp2gaoWAPOAg1K2PZs9EVucJ51r6Al8ATyfY3nSMQD4sSkniogny7Jkg3nAYUA3oBcwGXgmZf/5wF7AEGBr4BgROQVARALA/4BxQDHwIvBKG73ObJLtNjAe+AVYD+gBnAosb7J0uWUNcIaIrNvSFWWpna0CbspCOS1GU6+zxUxeIuIWkWtFZK7zFvSsiHR19uU7mnqlMzL4UkSKnbeMbYBHnFHDXRmG0GOdt9pSEZkqIgNS6j1ARH51yh1bc8STCVWNAc8B64tIYUp5hzlvb6tFZIqIDEnZd62ILBaRNSLyk4jskiLjNSnHpTUNicihwGjgZOd6v0pzzGfADin3pL+IdBOR50RkmYj8LiKXiYg4x58tIh+IyH0isgq4IsMl54vIS849/FpENkt3kIgEnLIWi8gCEblDRPJqHHOD81vOFZGjUrYfIiI/O3XMF5ELnHu9UlX/1LVpGhLABilFnox5G1+sqvOAscApzr69gLCq3q+qEeAuoBBIa2IQkQ2cNlIqIm9jlFDlPo9zD5Y4v++HIrKxs+8C4AjgWue+v+hsv86556Ui8oOIHJBS3iZiRrklzm/zVMq+oc7vssppK4fWVU+Na1gA9AUmiUiZs62/iLzl3PfZInJyyvG3Ou3jeREpBY6tUZ4AI4DHVTWkqjFVna6qk2ocd6rzmy8TkUtTtu/kPLMlIrJIzGiyQR2QiJwnxnS3QkReFpHezvbbROQO53NARCIicqPzvUhEwiJSkKHYZZgXwYyjPBE5S0R+ce7Xm+IoH0lj3pOUPiPd8+Sc85FT1jIReVJS+owG8Ciwo4hsl0HWbiLylBjT2XwRuV7MqH0rzLOwm9NW/hKRTUVkacq5z4jIvJTvE0XkbOdzc9qM13lWxtf5W6tqs/+AP4A9a2y7ApiCeRD8wBOYBgzwT2AiEAA8GCWS7+z7AjgxpRw/oEA/5/sEYCnmzTXPKecJZ18foAw40Nl3GRBLLa+GjLcCjziffcD/AYsBcbZt73wfDriBM4HZjszDgLlAb0CA9YFBKTJek1LPvsCclO9/ATvXlKGO+1vznryAeTMvwHTEvwMnOPvOBuLAGY7MgQzXHQUOdu7TNZi3VXca+W53fscezrV+DVydcl1x4N+AF9gTqEi5DyuAbZ3P3YGtUmTwAasxyiQBXJqyLwwMS/m+M7DM+Xwl8EqN65kMnJfh3n2TIt8oR77K39yDUV4FmHb2APBFyrnVfkdn2zHAOpiXsZOAUqCHs+8V4BKnPQSAnZztRU47OsH5TbYBVgIbZKonzXVU/SbO9y8x7dWHUQ4rU+q7FYgA+ztypmsDnwIfA0fjPFsp+zbBPHP3OfdlG6e9rO/s39bZ5gYGA3OAs+t4Xq9xPu/vXMcWznHjgPdS9n3tfN4D+A34OGXflxnuy75O/es5v0Vl25sGHOt8Phb4CdgI095vBj5MudZ4pueNNM+Tc84eTpvq4xx/a6bfqkbZEzDP22XAZGfbUMxLUuUxbwP3AkFMW/sWODlFnsk1ylwCbOZ8/t35G5Syb9OmtBln2yNAPjDJ+b1cdbXTlnTKnwVcoaqLVDUM3IAxXQimk+8JDFbVuKp+rarljSj7BVX9RteOKrZ0th+MaZRvOPvuxAwv6+IkEVmN6WhOAI5U5w471/BfNW9vCTX+Cx9GwcQxN30IpiOeq6q/N+IamoSI+DBvtJerapmqzsG8tZyUcthcVX3YkTmUoajPVPU15z7dilEYW6c57gTgelVdrqpLMA9jal1x4AZVjarqZEznfmTKvs1EpFBVV6jqt5UnqfGNdQW6AhcB3znXl4e5xyUpdZRgRiFgOv/UfTX3VyHGLzMkRb73gSr/g9P2nnTuY2Ub3VZE/GnvmDnneTUjp6SqPg0sxLQHMO16INBHzZv/VGf7YcAPqvqs85t8DbyO+R0bjYhsiHmhucq5j9OAJ6n+u3ysqm85cqZrA4cCXznXPE+Mz2GrGsdcr6phR96fMYoAVf3KeWYTqvobptMZ2QDRTwDGqepM535fBowSkT4YBbeFiBQBu2KU+0bObzESo/wyoqrzgceBf6XZfRZws6rOdtr7DcDOlaOjBlDteVLVn1X1A6dN/YV5/hpy/an8F/Ns7Ja6UYy1ZVdgtKpWqOpi4B5qjBhq8AkwUkQGYpTq6873TQFU9admtJmuwHvAd6p6pqom67qoFlEojtJYD3hLjClhNUbLujBvqo9iGshEZ0h9i4i4G1HFXymfKzCdDJjR0PzKHc7FL6ynrKedjm0dzFvRlin7BgBXVV6Dcx09gXVV9UfMKGwMsFSMSa+hDbQ59MHcx3kp2/4EUu3H86mf1PsUBxZh7l8Vzu/Yxyk/U13LnM4hdX9lOYdiOs15jtmgVsCDqpYCDwLPi0ix88BHMG/1lRRhHhQwI9Ci6qVU259K3wzyVV6fR0TuFGOqW4PpNAXTRtMiIqfJWhPoaswIsYez+yLMW+W3zjGVptYBwK412tERmDbXFCqvK1VRNKoNOC8Il6rqppjfeDbwcsohCVVN9alUPWciMkRE3hZjKlwDXMfae1Cf3FX3X1VXY/wf66rqGuB7zGh0V+BDzGh4OxqgUBzGAIeJyCY1tg8AHky598swLzv9GlAm1LiXItJXRF4UkYXO9T9Cw66/ClWtAG5xZK4pqx9YliLv3RjrQCY+BnbD3LePgY8w92wkRtlA09vMLpiR3Z31XhQtpFCcN/yFwB6q2jXlz+805IiqXqeqm2BuwlGs1cDNSX+8mJRGIiIuqt+wumReinmT+beIVDaO+cB1Na4hqKovO+c8qao7YsxdfszbO0A5pmOppE9dVTdEvhT+ApJA/5Rt/amuOBtS5nqVHxxl3hejVNYWYn7HvzCNPFNdPWq80fevLEdVP1fVAzEPwySMIzgdLswIo/I+zcK8TVUyjLUO6R9T9zm/8VDSO6wXZ5CvklOBvYHdgS4YUwYYpQI17qMz4rkXY/rs5ryIzKk8XlUXqurfMYriAuAxEemPaUeTarSjAlW9MF09DWAR0FNMgELqdTW2DeDIvRT4DzBQRPIbcMrDGFPiYFUtAm5k7T2ri0WktCUR6YJ5GaiU+2OMj2xTYIbz/QDMS96nDbiOJcD9jjypzAdOqXH/A6o6HfOsup2RfyU1n9ea9/IO57yhzvWfTsOuvyYPYfqr/WvIWgYUp8hapKqV1oN0v+vHGOVRqVA+cT6nKuKmtpnXMSOkySn9YkZa0uT1IHCriKwHICK9ROQg5/OezluOC/OGEsfY0cHY/NZvYp2vAduJyP6O42g0KU7Y+lDVmZgf42Jn0zjgfBEZIYYCETlYRIKO/COdhhhy/iqvYQZwoIh0FeP8O7+OapcAg5zRQENkjGBs9beICW4YjPFJPVP3mbXYUUQOdExMl2H8Hd+kOW48cL2IdBeRXsDVNerKwziUvSKyB6ZDeMmR7VjHhBHDjCASACKyn4hsISZwowvGZLAI+NUp8yngUhHp47SfCzE+ODDD74AYZ6kPMyooJ32HMxsz6qiUb3eMzb2SQoy/ZgXGTnxzjfNrtsUCjDJfBrjEODurgglE5BgR6eso4tXO5jjwKrCVsz/PkWV7WRsq3dg2PweYCdwsZp7W1hhfUIOjK52R2ZCU3+BsjFmuIabnQqBEVcvEBHOc0cBqx2OisYY6Sv424APHbASm8zsN+EZVE5g37bOBH1W1ppkzE7dj2mDq/XwQuEbWBlwUi0iluXER5vc8wbkX51L/S2ghptNf47wwjG6gbNVQ1ShG+V2esu13jE/mdhEpFOOM31DWzmtZAqwn1QNjfsD4d44CPnFGlhUYZVypUJrcZlT1Rkzf+p6I1NmftqRCuR1jT/9ATNTAZ6y10a+LCf0sxdyMtzCOZjBOo7+JiYa5vTEVOvbG4zAadTlG+3+PMaE0lDuA80Skm2MDvwDzJrEa00Edj9HkAUyE0XLMm3ABZugP8BjmB5wHvEHmN3MwTrogsFJMRFdDOMv5/yfwAWbI3dhQ7ZeAv2N8TEcARzgPcU2uw4wYfsQoyqmY37aSPzCd5l+Y6z5VVec6+/7uyFgC/A3TgMGEC0/EvEz8ihkd7euY3sD8fu9jHKkzgBdV9QkAZ8h+CKajWY0Z2R6acm4VTsd+DGYEshKjOFOV4aOYzuQvTDupqZTGAds4pocJqvoNpnOahvnNBzmfK9kBmC4mEutF4Ew1PsRVwD6YEdFiTCd2M0YZ16qn5nVkuK6jMf6hvzARTpeq6pT6zk2hCNNJlGDaak/g8AaeexFwunOd99HAUHtVfQMTIPEa5h70oboNfwpGsVeaaWZgFPgnNBBVXYnpQ4pTto3H+CxedkxUMzBKB6fNnw5cj3mW1wOm11PNdRjTXAnm5e6lhsqXhicxLzSpHIfxXfyMabfPs9bk9Q7mmVsqJvqvsj1MARY5o00wiiSG6V+b3WZU9WrMMznJeQFJS2U0U4fEGaX8hZkj83mu5bFYLJaOTJtPvdJYHHNKF2dIfT1m6FffG4fFYrFYmkmHUygYZ9TvmLkqo4DDHFulxWKxWFqQDm3yslgsFkvr0RFHKBaLxWLJAW02oV6PHj104MCBuRbDYrFY2hXTp09frqo9c1F3m1UoAwcOZNq0afUfaLFYLJYqROTP+o9qGazJy2KxWCxZwSoUi8VisWQFq1AsFovFkhWsQrFYLBZLVrAKxWKxWCxZwSoUi6UZJBIJVi8rIR6rlZvSYul0tNmwYYulrfPGQ5N47OrxhMsjuD0uDrtgf0656VhcLvueZumcWIVisTSBj1/4jAcvfopIhVkZIRaBl+9+C5fHxSk31LVaq8XScbGvUhZLE3jqhheqlEklkYoIL//fmyTi6ZaVsVg6PlahWCxNYNmClWm3xyIxwuXhtPsslo6OVSgWSxNYf4v+abcXdisgWBRsZWkslraBVSgWSxM4/dYT8QW91bb5gl5Ov+1ERCRHUlksucUqFIulCQzdaRNuf+86thg5hILifAZvOZCrnruQvf+2W65Fs1hyRptdYGvEiBFqsw1bLJZMrFlRytfvzMDldrHtfluS3yU/1yK1CURkuqqOyEXdNmzYYrG0O9594kPuOfdh3B43CCTjSa545gJ2Pmy7XIvWqcmKyUtEHhORpSLyQ4b9IiL3iMgcEZkpIltno16LxdL5WPz7Eu457xGi4RihsjCh0jCRUJRbT7yH1ctKci1epyZbPpQngH3r2L8fsKHzdybwQJbqtVgsnYyPnv+MZJq5PiLC1Fe+yoFElkqyolBU9RMgfWC+4RDgKTV8AXQVkXWyUbfFYulcRCoiJBLJWtuTySSRUDQHElkqaa0or3WB+SnfFzjbqiEiZ4rINBGZtmzZslYSzWKxtCd2OHgbvH5v2n3bHWCt6bmktRRKusD8WuFlqjpOVUeo6oiePXu2glgWi6W9sfGIwex98kj8+T5EQFyCL+jjqEsPZt0NrOEjl7RWlNcCYL2U7/2ARa1Ut6UTkIgnWDx3CYXdCujSoyjX4lhamPP/ezq7HbMTH06YitvjYtQJu7LpdhvmWqxOT2splNeAf4jIBGA7oERVF7dS3ZYOzkfPT+We8x4hFomRiCfZco+hXPnMBRQWF+RaNEsLISJssesQtth1SK5FsaSQFYUiIuOB3YAeIrIAuB7IA1DVB4G3gP2BOUAFcGo26rVYZn0xmztPu59IxVpn7Lfvf88NR9zJnR/8K3eCNZFVS0t4+5HJ/P79PDbZdgP2OXUPCrpmb8Le8oUrQIQefbtlrUyLpZKsKBRVPa6e/Qqcl426LJZUXrzzNaI1Invi0Tg/ffkri39fwjqDeudIssbz+/d/ctGu1xGLxIiGY3z++jQm3PY/7vvq3/Tq3zyf4u8/zOOW48ay8Le/AOi34TpcPf5CBgxZr54zc8uyBSt4aewb/PzlrwzcrD9Hjj6Qfhv1zbVYlgzYXF6Wds2SP5eRLnuQx+thxaJVrS9QM7jrjAcpL6kgGo4BEKmIsmZFKQ9d+nSzyq0oDTF65HX88eN8YuEYsXCMP36Yx+iR1xGusaZLW2L+Lws5Y/PR/O+/b/Pj1F9457H3OWf4Zfww9edci2bJgFUolnbNlrsPJc9be6Adj8QYNLRtv32nEo3E+HX63Frbk4kkX739bbPK/uTFz4lHq695rwqxSJwpL33RrLJbkgcvfpKKNRXEo2YSYyKeJFwe4e5zxuVYMksmrEKxtGuOHH0gwaKAyenk4A/6OPaKw9pVskC324XLlT7tvdef16yyl81fQbi89kgkUhFh2fwVzSq7Jfnuo1lpR5/zflpIJNR2R1adGZsc0lKLRCLBlIlf8NHzU/EGfOx/+ii23H1orsVKS7c+xTz47R08c9NLTJ/0HV16FnLUJYcw8qgdci1ao3B73Ox02LZMffXraqMJrz+PfU/do1llb7LdBgQK/ITKqq8k6Qv62KQNh9rmFwVqLbMM4Mlz40kzKrXkHvurWKqRTCa59qDb+H7KrKq32s/+9zVHXHQAp95UZ+xFzuixbncufPDMXIvRbP75wJksnPMXC2YvRoBkUtlsx405+Yajm1Xu8L2HMWBIP+bOnEc0bAIYvAEvg7YYwFZ7tM0XBYBDzt+P58a8VC2Cz+vPY9QJu+J2u+s4s3GsWlrC95/MIlgUZOjOmzDjgx+oWFPBsN2H0n2d4qzV0xmw66FYqvHlm9O5+bixhGu8zXr9eTz+893Njjay1I2q8tOXv7Lw18Wsv8UABg8bmJVywxURJt71GpOe/BgE9jllN44cfRC+gC8r5bcEiUSC/5z+IB9OmIrXn0csEmOrUZtzzfOj8QezI/eE217hqX+9SJ7XQyKRJBqK4Av6EJcQjyY44erDOeGaI7NSV2uRy/VQrEKxVGPs2Q/x5rjJtbb7832cd/ff2ffvzTO/WCyNZfnCFfw5awF9B/fBF/Ty/nOfsuqv1Ww1anOG77UFLlfTXMEzP5nFVfvfktasVokv6OOWt65qVxMo7QJbllYhVBbi8Wsm8P6zU0jEE+x8+HacfusJdO3ZpeqYgq75uD1uEjXSg7tcLoJFgdYW2WKhx7rd6bFud7794HuuO/g2kskk0XCM1x+cxKbbbsAtb1+NJ6/xXdkbD71HtB7nfjQU4c1xk9uVQsklNsqrk6CqXLLHDbzx0CTWrCilvKSCyc98wvnbX0U0Eqs6bp9Td8eTV9s+LS6xmVwtOSMRT3DzMf9HuCJSNU8nXBZm1he/8s5jHzapzPKSirRRZKmoQkVpRZPK74xYhdJJmPnxLOb/vJBYZG0EUSKWoGTZGqZMXDsXYb2N1+XCh87EF/ASLAoQLApQ2K2AW966qk3b2y0dm1+/mUusxlwaMKHPk5/+uElljjxqB/z5dbdpf76P3Y/ZqUnld0asyauT8Nt3fxCP1X4gQ2Vhfv3mN0adsEvVtj1PHMmOh2zLzI9n4fXnscXIIU0yKVgs2cLtcZNpOOFOM6JuCLsftxMv3vUaf/wwP+1+f76PTbbdkJFH79ik8jsjtpfoJPQd3AeP11NthALmoVlv41prnREsDLD9gcNbSzyLpU4GbzmQYFGw1lwaf76P/U4b1aQyY5E4i+curbXd5Xax/YHD2fOkkex48Ihqk2YtdWNNXp2Ebfbdki49inB71v7k4hK8fi+7H7dzDiWzWOrH5XJxw6uXkd8lSKDQT54vD1/Qyw4Hj2CP4xvffsMVES7e7fq0EV7iEtYfNoBdDt/OKpNGYkconQS3x83YT2/mP2c8wPT3ZoIqm26/ERc/ei7BQhu9ZWn7bDxiMOMXPMTn//ua1cvWsMXIIWyw5aAmlXXvPx7h9+//TLsvEUtQtto64puCVSidiO7rFDPmDRPVpcmkdbJb2h2BfD97HL9L/QfWQTQS48PxU0nEk2n3+/P97HhwTqZxtHusQumEeH3NSzZYiaoSrojgC3ibPLnMYmltYuEoyWR6ZQIwfO8t2mzuuraO7QUsTeLTV77kxIHncljxyRza9WQeu+Y5EolE/SdaLDkmWBSk13o90u7beJvBXPfixYikz/xsqRurUFoBVa0187w9M+PDH7j1xHtYOn85iXiSUFmYl8e+ycPNXAjKYmkNRIQLHzoLX9BXtWSAJ89Dfpcglz91vh1tNwN751qQaCTGfRc+zkGFJ7Gf71jO3vpSZn0xO9diNZunbniRSI1ldyMVUd546L02vQKgxVLJ1qM2597Px7DHCbuw0YjBHHj2Xoz77s60IfSWhmN9KC3IrSfdw5dvflO15vlvM/7g8r1u5IHpt7frdbEXzVmcdru4hFVLVrerddwtnZdBmw/g8ifPz7UYHQo7Qmkhli1YwRdvTK9SJpVEwzFeuPO1HEmVHTKlVBcRuvft1rrCWCyWNoNVKC3Eojl/pY2mSiaSzP0uffx7e+HkG4/BF/RW2+bP93H81YdnLYLMYrG0P6xCaSH6bdyXWEoW30rceW423nZwDiTKHhsNH8ztk69ns502wRf0sc76vTj37lM55rJDcy2axZJzFsxexNizx/HPna/h/oseZ+m8ZbkWqdWwC2y1ILef8l8+mfh5tSVMA4V+Hppxp/UzWCwdkFlfzObyvW4kFomRiCfx5Hnw+vO45/MxDBiyXqvIkMsFtuwIpQW5+JFzOOayQ+nSoxCP18Ow3TZj7JSbrTKxWDood58zjnB5pGoWfjwWJ1QW4oGLnsitYK2EHaFYLBZLFohFYxwQPAFN1u5Tvf483qx4rlXksCMUi8Viaee4PW7yMgSldJbls61CsVgslizgcrnY55Td8fqrKxVf0Msh/9gvR1K1LnZio8VisdRDJBRh6itfsXzhSjbdfiOG7rxJ2nxfZ915EssXrmD6pO/I8+URDcfY5YjtOe6Kw3IgdetjFYrFYrHUwZ+z5jN65PXEIjGi4Rh5Pg+bbrchY966ijxvjdFIwMeNr17OX38sZdFvS+i/SV96rNs9R5K3PtbkZbFYLHVw09H/oXRlKaGyMIl4gnB5hFmfz+aVe97OeE6fgb3YetTmnUqZgFUoFovFkpGl85axeO4SagbDRkJR3nnsg9wI1YaxCsVisVgykExqxrVRkonMi3R1VrKiUERkXxH5RUTmiMgVafafIiLLRGSG83d6Nuq1WCyWlqT3gJ706Fc74ak3kMdef9s1BxK1bZqtUETEDdwH7AcMAY4TkSFpDn1eVbd0/h5pbr0Wi8XS0ogI10wYTbAogC/oAyBQ4Gf9zQdw5OiDcixd2yMbUV7bAnNUdS6AiEwADgFmZaFsi8ViySkbbDWIZ/94gA8nTGX5ghUM2XFjRuwzDLfbnWvR2hzZUCjrAvNTvi8Atktz3BEisiswG7hIVefXPEBEzgTOBOjfv38WRLNYLJbmU9A1n4PO3jvXYrR5suFDSeexqpnM5nVgoKpuAUwGnkxXkKqOU9URqjqiZ8+eWRDNYml7aLIMDb2Klj+FxufkWhxLE4lFY3z22te89fBk/vxpQa7FaRNkY4SyAEjNy9wPWJR6gKquSPn6MHBbFuq1WNodGp2GrjoDE4cah1IXGjgcKbo+YzSRpe3x508LuGT364mGYiQSSVSVkUftwCWPnYvL1XmDZ7Nx5V8DG4rIIBHxAscC1da4FZF1Ur4eDPyUhXotlnaFagxddS5oOVABRIEwhF+FyIc5ls7SUFSV6w+9nZJla6goDRGpiBANRZny0he8/8yUXIuXU5qtUFQ1DvwDeBejKF5Q1R9F5EYROdg57AIR+VFEvgMuAE5pbr0WS7sj9i0Qr71dK9DQxFYXx9I05v+yiOULV9aa7Bguj/DGQ5NyI1QbISu5vFT1LeCtGtuuS/l8JXBlNuqyWHKNRqehFc9AYgX4RyGBoxFXsAEn1l4SukH7LG2KWCSGy5XePBkJRdNu7yzY5JAWSyNIlj8LpbcDYUAh9h1a8Tx0n4i48us+2TuC2vEqgASRwCEtIK2lJRg4dD18AS+hsnC17b6Alz2O3yVHUrUNOq/3yGJpJJosg9LbgBBrFUMYEgvR0Iv1ni/iQ7rcCfgBr7M1CN7twN851svoCLjdbq589p/4833k+cw7ub/AT/8h/Tj43H1yLF1usSMUi6WhxL4H8aQZZIQhPAnyT6m3CPGPgp7voqHXILka8e0K3u1thFc7Y+s9t+Cxn+7mncc/YPmClWy95xbsdOg2ePI6d5faua/eYmkMri6gGRICuhqeplzc6yAFZ2VJKEuu6NmvOydde1SuxWhTWJOXxdJQPJuCuxe1H5sAkn9SLiSyWNoUVqFYLA1ERJDiR8G9HkgQpADwQeFFiHfbXItnseQca/KyWBqBeNaDHpMg/iMkSyBvC8RVmGuxLJY2gVUolk6BagItfxDKnwBdA55NkKJrEO82jScnVCoAACAASURBVC5LRCBvaPaFtFjaOdbkZekU6JqboWwcaAmgEP8JXXkaGrNZgCyWbGEViqXDo8k1EJqImT+SSgQtuz8XIlksHRKrUCwdn8RCkLw0OxTiv7S6OBZLR8X6UCwdH3e/DLmyBDwbt7o4TUUTyyA6BfCBbyTiKsi1SBZLNTqcQlmzopSHr3iGKRO/QFzCHsftzN/HHEd+l3ryLFk6LOIqRINHQ8WLmBxclfiQgnNzJVajSJY/AaV3gbgxa9opdP0v4ts54zmqilY8B+X3Q3I5uAcihVcg/t1bS2xLJ0O0Zg7mNsKIESN02rRpjTonFo1xxtDRLPlzGfFYAgCP10P/TdblgW9u79QL33R21kZ5PWkc855NnSivEbkWrV409jO64miqK0NAAkjPqRlHKsnyR6H0Hqr7jvxI8QOIb6eWEteSY0RkuqrmpGF3qB72s/9NY+WS1VXKBCAejbN47hKmT/ouh5JZco2IG1fBebh6f4Wrzy+4erzaLpQJgIZewSzGVRMXWv4UyRXHkfxrC5JLR5Isf8aMTDQBZfdTOxAhjJb+p+WFtnRKOpRCmfvdH4RKw7W2R8MxfvvuzxxI1Hw+feVLLtr1Wk7b7EIeufIZ1qwozbVIltZGw0CaHGKagPL7IDYdCENyMZTegZbdbebaaCR9eYk/WlBYS2emQ/lQ1t1wHfwFPsJl1R8kbyCPdTdcJ8NZbZenbniBF+98jXC5uZ7Fc5fw/rNTGPfdXRQWW4dsZ0H8+6DhV0Frjjai1FY0ISh/DM0/DcQHmmZk4x5Ya5NqAqJfQnIleEcg7j5Zkt7SmehQI5Rdj9oBf9BfbTU1l9tFQdd8djhoeA4lazxrVpby/G2vVikTgFgkzprlpbz+wLs5lMyiyZVo+D00+jWaKftwNvHuAL69QALOBhdmTZUMKV/EjSSXQP65QKDGTj9SOLraFo3/ji7bDV19HlpyLbpsT5Jr/k1b9a9a2i4dSqH4gz7u+XwMW4wcgsvtwu1xs/Wem3P31DHtbp2COd/8Tp6v9tyJaDjGtHc7pz9Ik6vQioloxXg0sajZ5SVjv5Jc8x+Sa+5EYz807Jyy/6JLR6Ill6OrzkSX7YHG/2i2LHUhIkiXO5CuD0DgOAiejHR/HrxbpD9B4+Dqg+T/HQovBVdPQMA9COl6dzWHvKqiq86C5FLQcqAciEJoAkTea9HrsnQ82lcv2wDWGdSbO97/F9GImXfgTdMptweK+3QlkRJcUIm4hJ7r9ciBRC2HqqKhl6HiCUiuAd9uSMF5iLtX1THJ0CQouQTE5axJcgtacCGugtOaVGdy1UUQeXOtDBWPoYG/4epyRWY5I1Og7GEgstY/oRXoqtOhx3stukiWiIBvR8S349qNBf9AV35N9egvP3i3NYuBebfBlX8i5J+YueD4r5BcQq1VwzSEVjyL+PfO4lVYOjodaoSSiteX126VCcCgof3pt3Ff3B53te1efx6HX3hAjqRqGbT0Flhzo5m1nlwMoRfQ5QeQTCw3+5MlRpkQBq0w/4lA2d1o7OdG15csf7qaMjHEIfQUGvsxs5zlz1A7akohsciYimKzGy1LQ1GNo6H/kVx5GslV56LhDyFvS6T4XpNOHxeQB0QhOh1dfS66dGc0NquegkOAO/2+ZHl2L8LS4emwCqUjcMtbV7HJthvg9ecRKPST3zXI6IfPZuMRg3MtWtbQxHKoGE/1jjph5oosH4VGv4HIB6RvqlE0/HrjKy27L8OOOBp6pw5h12Q8j/BEdMWRJMseaLw89aCaRFedhZZcZ2bKRyajqy9ES8cgvpG4er4P3cZj7lESKDfmK12JrjwVTZslwCFvU8xEyZr4wb9/1q/F0rHpcCavjkRx766M/fRmls5bRumqcgYM6dfufEH1Ev8pczSShtBVp0HBxaRZyN1sq6uzzISW1LGzjncs/z4Q+5FaEwwB05GHoex+1L8/4hnQeLkyEZ3ihAanKt0QVDyPBk9EPAMh9CqQ7l5EIfoF+HZJW7SIFy26BUoudc5PGOe/uz+Sf1z2rsHSKbAjlHZAr/49GTxsYMdTJgCuPsaJnAlVR9nU9ieBH/Hv2/g63Zk7ewkeVMe+Y8AzgNqRU6koRCY3Xqa6Sgx/4pj6akkE0c+dg0pIO1cFIFn33CVXYB+kxysQPBF8+yFF1yPdX0Kkruu0WGpjFYolp0jehuDZkPRmF4AoQgIKLwN8mEG1AAEIHAZ5WzW+zqIrMP6GGvgORDwbZD5PAkj3F5Giq8G9PukfH6kqW5MVaORzNDazeSG4ri7p5RU3SJH56NvTLEtcE42Bd7t6qxDPYFxFV+MqvhsJHI6It+nyWjotHfCV19LekG4PoytPh/j3afbmgW97XHmbo74d0dAboFHEvzfi3bJp9fl2g+L70DW3QuJPcHWFgvNxBY+v/1zxQ/Bo8O6ALt8fSDMb3b83yYqXTKCBuIEkSDF0ewTxNN7/JYHD0fJHqG3SEvDv4dS5j/FFxX7AmMYE8EHBPxB390bXabE0hQ6VHLKt8M3kmTx48ZPM+2kBXXt35YSrD+fAs/Zu0bDS9o6ZD3EaRKdR5aOQAHh3x1U8NqeyZSJZ/hyU/hvTeQuQhKJ/IXmboiuOpbqvRcDVC+n5ESIZoqrqqis0CdZchhkVKeBFisch3mFVx6jGIPw2Gn4LpBAJHot429eEXkvzyWVySDtCyTIzP5nFdYfcRiRknMwrFq7koUuepmJNiGMuOzTH0rVNNPQGWv4AJJaDZ33AA64gEjgS/AfmWryMuPKPR/2jnCg0Ad8oxN2TZMm11E7mqKBlRmH66jdB1aorsDfq3xWi00G8kLcVItUfX5E8CByMBA5u8jVZLM3BKpQs8/i1E6qUSSWRigjPjXmZIy46sGM61ptBsmycE8brRDDFV4MEka4vI55BOZWtIYi7NwRrREMlV5DeQS6gq9OWo8kys3ZJ5CNw90KCJyPe6v4hET9Um+Ueg/CbaPhtkCIkeEy7yaDc2YlFY7zx4CTefeIjRIR9T9uDA87Ys933D+1b+jbIvFkL0m5PxBOULC+l+zrFrSxR20U1bLLlVguHVRMuXHY/0vWOXInWLMQ3Co1OrZ3MUaOQV7vD1+QadMWhkFgGRCAmaPgDtOg6XMEj09ahGkNX/g1is6j0mWj4XbTgfFwFZ6QcFzFzUqTYmlzbCMlkkiv3HcPPX/1KpMK8fD582dN8+cZ0xrx5Vbv+nWyUV5bpt1H6rMbidlHUvXNlCFaNo+G3Sa76J8mSq9HojOoHxOeTvgkmIfZNa4jYMgQOAvcgqocXB6DgnLQOcq14aq0yMVuAMJTebJRuOsLvQuwn1ipj55yyu03ySo2QLLkKXTIcXborumwXkiGbm6stMOODH/hl2m9VygQgUhHl+yk/8eNnv+RQsuZjFUqWOfnGY/EFq4dc+oI+jr7kYPK87TcVTGNRTaCrzkBLroTI2xCaiK78G8myR9ce5O6ZeWKie73WEbQFEPEi3SdA4eWQty349kSK78dVcF76E8LvkzZaDBdkSC2j4feANHNTJA+iX5n7Hnod48uJmuSPJRebzAOWnPLD1J8Jl9V+UYhFYvww5accSJQ9rELJMluP2pxrJoym7wZmPYmi7gWcfMPRnHhtetNFhyUyGWLfpkzIq3yDHosmVgAgrq4m3BVfjZP9SP45rShs9hHx48o/Hlf3Z3AV31/3kruubum3a9yZg5LunC5kenxVFcKTqK2kwmgLpIaxNI7i3l1rvXQCeP1eivt0zYFE2SMrCkVE9hWRX0RkjojUStcqIj4Red7Z/6WIDMxGvW2V7Q8czpOz7+Xd+PO8tOxxjrr44HZtF20KGp6Ufna3eNbO7gakyy3GRITP/Lm6Q5d/I02IhGqvSP7J1J597wbP+hkDEyR4NJBu8qEP3P1MJFg6EvMA0MjnJFccT3LpziRXnt7g9P2W5rPbMTvidtcOHXe5Xex65PY5kCh7NFuhiAmqvw/YDxgCHCciQ2ocdhqwSlU3AP4PuK259bYHXK5OPACUIjLOJJf8td/Eh6vLLUjvr5Ce7yM9p+IKdKxsyvUhvl2h4DzAB1Jo5t94BiPFD2Y+J2+oManhAykw99TVHen2GOJZP0M6Gxd4tyQZmmTWQIlNM6aw6BR0xfG1fVyWFqGwuIDb3ruWXv174M/34Q/66DOwF3d+8C8CBe073U2zJzaKyA7Av1R1H+f7lQCq+u+UY951jvlcTPD8X0BPraPy9jyx0QIam5Vmch8gXZBeU1s9tYdq3ITYhl4D8SPBo8A7sk2NHDW5xsx0d3UDz8YNkk2TpRD92igU74iqSZPJsv86a7dUOu3FpGbp9jKsOsUsE1CTvOG4uo/P2vVY6kZVmffTAhCh/ybrZq0ttveJjesC81O+LwBq2iuqjlHVuIiUAN2B5akHiciZwJkA/fv3z4JollwheUPQwsug9DZj5gLAgxQ/kgNlkkRXnQHRb6jsYDXyKQSPRYqubFVZ6kJcRZC6gFaDzilcm34ldXv+eairL5Q/5KwTPxwpvARcvdDk0vSFxetZO8WSVUSEAUPab/BJOrKhUNKp1Zojj4Ycg6qOA8aBGaE0X7TGsXzhCp6//X/M/HgWvQf25OhLD2HoTpu0thgdBlf+iWjgQIh+6aRR2T43SQejn5gAgVrp358h6R6E+M0M9+ai4bfR0rGQWGT8H4WX1XLGq8bQimchNBE0AYFDkfyTzaTFLCIiSPBwCB5eo/6Es1xAGv+Wq2OtBGppfbKhUBYAqWq2H1Bzwe/KYxY4Jq8uwMos1J01lvy5jHOGX0ZFaYhELMHcmX/yzeSZjH7kHPY4dudci9duWRvJlTs0/GGG9O8xKB2Dlt5s1hUpvLzJZgeTDPIGqkx88Z/QVedA8f2Iz7Qfk6/sTJM+pfK4svvQyPvQbQIidURtRb8wy/q6+4J/L0RqRsY1DBE3GjgJKp6kujkyAO08ss6Se7LhNf4a2FBEBol5/TwWeK3GMa8BJzufjwQ+qMt/kguevvFFyksqqq3jHqmIct/5j5KIp1uLw9JucBWT+d0pAkQhNB7CbzepeFWFsruovfBWGC1Nme0f+8aZsBmudgzx2WYUlbbsMLryOHT1OWjZWHTNtejSkWj8j6bJmiwDV08TCYYb8Bv/S8F5SOCIJpVpsVTSbIWiqnHgH8C7wE/AC6r6o4jcKCKVWeoeBbqLyBxgNFArtDjXfDN5JslE7fxLkYooX739LbFoE1YGtLQJJHA49Q7GNWRmrDeJMCRXpd8V/33t59iM9BM5tSLjhEMte9isEqkVQNxZ2ncVunp0o6XUxCJ0+V5G+SXmAHlmImS3F3AVnNmmAhQs7ZOsxLWq6luqupGqDlbVMc6261T1NedzWFWPUtUNVHVbVZ2bjXqzSXHv9BPIIqEo/z7hbo7sdRrvPvFhK0tlyQbi6Q9dbncWoEqzCFUlyUxrxteHv1oodDXcfdd+dvUy/os054u7T/rzQy9Te4KiQvwXNNk4q7GuGeMovkpfUthkQC69sVHlWCyZ6MQTJapz9CWH4M9Pb5cOlYWpWBPi3n88yg+ftu/UCJ0VV2BfpNcX0PUeIF3n7wX/3k0qW0Qc/0PNOQR+pPCfKV/3JO1ISdzgzzT3JpNlWEAzLPmbicgn1M6CrBD92jjrG4lGvya56lySy48gWXo3mmmUZuk0WIXiMPLoHTn2isPwBbz4C9IrlmgowsT/vN7KklmyhYgfl39XM1rBj/EhYD67eyH5pza97Py/Q+EFIF1Mua6eZrEt/34p9QeQbs85iSP9Tr39kG5PIZlSrAQOovaMeAHPIMTdyKgsyWT2c9PYriBZ8SK68jSTYif+PZQ/jC4/qNGjJkvHwq7YWIOK0hAfTZjKg5c8Sai0dgK3jYavz31fd4qJ/m0SVTUdWHI15A3L3BHXV05sNlrxtBPiu4np2D2DwLttxmirBstHGPBX80loshwSC8Hdx8yGTywAkuDuX6fvQpPl6MrjzVLFWgEEQPKQbuORvA0bJVuy5AYIvUj1xb/ywL8vrq53NeIaI+jS7Y0/pxpeyD8ZV+GljZLLkl3a+8TGDkWwMMBux+7E/Rc+Xmtfns/D8L2HpTnL0hpofD666u+QXAa4QGNowT9xFZze6LIkbyMouh5dfRFUPA3iQhGTS6zbM5l9GvWVK0Kq6UtV0dI7oeIpM0LQGAQOQ4qur7XiYtryXPnQ/WWIfIzGZiLuvuDfH3E1fikEKbwEjf/opL2PA2rWui9opIM/Pof0U8uiEPkYsqBQQmUhnh3zEu8/MwUE9jxxV46/+ggC+dmdr2PJLtbklYZgYYCTrj+qmk/F4/VQUFzA4Rd2rjxTbQUzh+MMSMw3b+paBkSg/F408lnTyix/ynSAhJ0yzShCV1+cPbkrnoKKZ4ysWo4JUf4fWvp/DS5DxI3498BVeCESPLpJygQc5VSlPBRIgq6BlcdVZYBuEK6umZcdcNVe76WxJBIJRo+8npfHvsXyhStZvmAlL499k4t3u55kspF+I0urYhVKBo657FCuHn8Rw3bbjP6brssh/9iXh2bcSdeeTTOxWJpJ/Gcn/1SNDkVDxnTVFELjqT13JAGxGXU6mDWxzIyWGmIuLn+E6jP0MXWGnkUb61RvBKoxY9ZL/JWyTaHkCkzUWGXdYUiuQMvua3DZ4l4X8oZS28ARML6kWrJE0fA7aPljaPTreu/btHdmsPDXxcQia5VWNBxjwS+LmP7ezAbLaWl9rMmrDrY/cDjbHzg812JYALSUtU70GjQ1ukjTLWoFxpxWe58mFqOr/+ksu+syEya73ol4t8lcRzL9GvJoGGN2yn4qmmToTVhzPZAAjaN5myNd7wUiznr3NYkZ5zrXNbgO6fpfdPXZEPvFMeXFoeACxDey2nEan4euPM4ZAUbNvBfPEOj2eMbZ/rOnzyVUXtt/GamIMHvab2yzz5YNltPSutgRiqV94Blqcl/Vwg++poX74t8LSLOKprs3uHpX26SaRFeeALGZGKd2GJKL0VWno4k0mXsryRuafrt7vRbJa6axmVBypTFlaTlmjfoZJuWLBKgdNuyQaR5NBsTdHVf3F5EeryHF45Ben+EqOK22PCUXGyWm5UDMKJbY92jZuIxl9x7QE38aX4kv6KP3gObnXLO0HFahWNoF4gpC0VWYcNtKh7Af3H2R4DFNK7PgH0Z5VDnRvSBBpMsdtSOvol86I6GaJrcEWvF85jqKrnTKr3zUBPAjRdc3Seb60PInqD0RMg7xX03H7h1ObcOEH4InNqk+8QxEvCPS+nU0udIZzdVUYhFnwmZ6djlye7z+vGq/gYjgDXjZ5YjOs/Bae8QqFEu7wRU8Bun2NPgPBO8OUHgJ0v1l42xuAuIqRnq8iRRdDf5DIP8spMc7iHer2gcnl5B+kmG0ahXEtHXkbYF0f9GMotz9wbsb0u3pqoSRWSexKL2c4oHEMqTLf8AzyGQNkALAB4H9kOBx2ZdFlfTRYACZJ1IG8v2MnXITGw5fH4/Xg8frYaNtBjP205vxBZqWFNPSOlgfiqVdId5hiLfhcybqLU8CEDzaWVK3DvK2yGByCyDeHequI28jpPiepgvZGLw7mdxfNUcpGoO8TU325+5vGNNdcjF4NkM8LbMmh7i7o55BEP+lppDOhM3MrLfxutz31a2sWVEKAkXdCltERkt2sSMUi6UBiGd9JzVLanoVL7h7QuDArNenqiaZY2PCeQHJPwlcXajuGwpA/hlGmeCsleIdhvj3raVMND4PLXuYZOn9aGx2M68CpMudZjloce6bBMEzEGlgqvyi7oVWmbQj7Ex5i6WBqCbQigkQeg6SIQjsi+Sf1eTZ+hnriU5DV18KyeWAQt5QpOvYBk+21ORK4/SOfAiuYiT/VKQBa9IkyydA6RiMzyMJ5EHgWCRwALjXbXyqlyp5SiH8Bhqfj3iHgW9UgyZ1WppGLmfKW4VisbQhNPEXunwf0NS5K27TofeY1Ky0MHXXuxRdNoraDn0wo7Ik+EchXW7PzaqblgaTS4ViTV4WSxtCK55P46tJmAit6FctV3HkQzJ3ByEgAuH30TW3tJwMlnaPHXdaLA1AVSHyERp6ATSCBA4C/0HVTDeqIQi9YeaCeAYjgUOr/BYNJjGP6skbq0p3MgW0FGICsuo0WEQg9BJadI01WVnSYluFxdIAtPRmqJhIZRoVjU6H0GtQ/CgiLjSxHF1xhDMzPgT40bL/QmOzAudtB+HJ1ErXokkTadZS+PaANTc14MC4ySJgFYolDdbkZbHUg8b/gIoXqN7JhyD2LUSnmGPK7nSyIKeuhliKrrmyUXVJ8CBw96D6u54ffHsgnsFNvoZ663X3gKIbAR/VJ4/WwN2vyfN+LB0f+5phsdRH9AvSdrBagYY/MvmrwpMxubmqHQCxH9FkhZnpXw+aLENLb4HEUszEP59ZNjj/b0gTZ7I3BlfwMNS3I4QnoYlFUPEsEHNkcQFepOhfLS6Hpf1iFYrFkoJq1KwXIn7wbGTSf0iRWaa3ln8hzySIBDI/SmLOrbdeRVeeCvFZmE4c819LjS+mAWVkA3H3hvyTjDsleCxaPg5i34F7MFJwFpI3pFXksLRPrEKxWBySofdgzRVUzcNw9YLih8C/O6xJZwJyI4HDzMfgEVD+FNXDbj3g3TljVt1qxGZCYjZrlQlGBg2jFRORNIkXWxrxDEC6jGn1ei3tF+tDsVgAjf8OJRebNPlabuaBJOahK08CvEjx4yDdTFZeKTD/u9xZNdNcCs53nOYBwG/2u9dDujQwzDb+G+n9FmGI/5Sdi7RYWhg7QrFYcOZ/pPOBaAVEv0B8O0GvqcYRrzHwbl1t5CHih27PmJFG/CcnEeT2DZ+I6BnsJFOsid+sH2KxtAOsQrFYwMkmXFOhVO5bCZilePFmnoAsIuAdZv4aS94WkLexk+69ch6KC8SPBI9ofHkWSw6wJi+LBUyklgRq79C4s4ZIC9cvYsxqgcMcORz/S/eJWc8VZrG0FHaEYrEA+PeH8sch/jtV68xLAAJHI+6+rSKCuPKRLjdBl+oTDFUjaNkDEJpozG3+vZDC0YirW6vIZbE0FKtQLBYwCQ+7T0DLx0P4LXDlI8Hjm768cBbRVWdC9BuqIshCL6ORT6Hn22Y9F4uljWAVSo75c9Z8fvj0Z4r7dGWbfbckz5tmjXNLqyASQAr+DgV/z7UoVWjse4jNoHo4ctykeAm9AcGjciWaxVILq1ByRDKZ5I5T7+OTiV8gAm63G2/Ay10f3UD/TdbNtXiWtkJsVoaEjRVobAaCVSiWtoN1yueI9576mE9f/pJoKEqkIkpFaYiSZSVcf+jttNU1aiw5wN0P0oYe+8E9qNXFsVjqwiqUHPHGQ+8RLq++mJEqLFuwggWzF+VIKkubw7sDuHoCNVKvSB4SPDwnIlksmbAmrxZCVflx6s98/vo0/AV+Rh2/C30Hr13CNRpOt+YFuFxCLJJhPoSl0yHigm7PoiWXO0kqAc+GSJdbbZSXpc1hFUoLoKrccep9THnpC8IVETweNxNufZUL7j+dfU7eHYBRJ+zCwtmLiYSqKxZf0MeAzfrlQmxLG0XcPZFuj6HJCiCBuArrPF6j36BlYyE+B9zrI4X/RLzbtI6wlk5Ns0xeItJNRN4TkV+d/8UZjkuIyAzn77Xm1NkemP7eTKNMyiOgEI8liIai3HPOI5StLgfg4HP3pf+QfgQK/ADk+fLwBX1c+ew/cbtbJ7OspX0hrmD9yiTyGbryFDOaSS6H2FfoytPQyCetI2Q7ZOYns7hy/zGcuuk/+c+ZD/LXH0tzLVK7pbkjlCuA91X1VhG5wvl+eZrjQqq6ZTPrajd8NOHTWv4RAHeei+mTvmPk0TviD/q457MxTH31a759fyY9+nVnn1N2p2e/7jmQ2NJR0NJ/UzUxs4owumYM0nPXXIjUpvlg/BT+c8aDRCqMpWDxb3/xyYufc/+026qZqC0No7kK5RBgN+fzk8BHpFconQqP14OI1IrWEgR33trRhyfPw8ijdmDkUTu0toiWjkp8Tvrtid9RVZNvzAJAIpHgvgser1ImAIl4klBpiCeum8BVz16YQ+naJ82N8uqtqosBnP+9MhznF5FpIvKFiBzazDrbPHueNBJvwFtrezKZZPjeTUgcaLE0lEyOeulqlUkNli9YWcuHCZBMKjM/npUDido/9SoUEZksIj+k+TukEfX0V9URwPHAWBFJuzi2iJzpKJ5py5Yta0TxbYuhO23CERcdgNefh9efhz/fhy/o47oXLyaQ78+1eJaOTP6ZmDVZUglA/hm5kKZNU1CcTzKRTLuvuHfXVpamY1CvyUtV98y0T0SWiMg6qrpYRNYB0nqzVHWR83+uiHwEbAX8lua4ccA4gBEjRrTr2X2n3nQc+5yyO1+/MwN/vo+dDt2Wgq75uRbL0sGR4N9QLYXyR8zEJgGCpyL5rb/iY1snvyjITodty2evfkU0vHalTF++j2Mvb11DSiKR4Ku3vmXGB9/TrW839jxxV7qvkzbGqU0jzZmVLSJ3ACtSnPLdVPWyGscUAxWqGhGRHsDnwCGqWueYcsSIETpt2rQmy9ZWmfPt74y77Gl+/vJXuvYq4pjLD2X/0/e05ghLVlGNmigvV/eGLUHcSQmVh7ntpHv5+p1v8Xg9JOJJTrz2CI69/LBWkyEaiXHZqBuYO/NPQmVhvP48XC4XN79xJcN226zR5YnIdMci1Oo0V6F0B14A+gPzgKNUdaWIjADOVtXTRWRH4CHMQt0uYKyqPlpf2R1Rofzx43zO3/7KahFgvqCPoy45iJP/dUwOJbNYOjerlqxmxeJVrLvhOq1uln757jd57KrnavlzuvbuwvMLx+FyNc7VnUuF0qwoL1VdAYxKs30acLrz+TNg8+bU01F45qaJtRpNpCLCi3e8xtGXHmL9KxZLjiju3bXKbzLvFCwy3gAAEN9JREFU54XM/e4P+m7Qhw23Xr/FrQeTn/kkbXBApDzC3Jl/ssGW7Sdnm50p34rMnvYbmqw9InR73Cz9cxkDhqyXA6ksFgtALBrjpqP/wzfvzcTtcZNMJum/aT9uffcaCosLWqxeT176blhVM+5rq9jkkK1Iv43WSbs9Fo3Tva/Ny2Sx5JLx/36F6e/NJBIy2b/Dzgjh/856qEXrPeDMPfHn1/Zzde3VhQFD2lcaJqtQWpHjrz4CX435Kb6Alz1P3MVGgFksOebNh94jWsP0FI/G+fy1aUQjsQxnNZ+9/jaS7Q8cji/oxevPI1Dop7BbATe8clm7C9ZpX+Opds7QnTbhmudHc+/5j7Bi4So8XjcHnLkXZ9x2Yq5Fs1g6Pen8GACaTJKIxcHXMqupulwurh5/Eb999wffT/mJ4t5d2eGg4Xj9tSdHt3WsQmlltj9wONsdsDUVpSH8QR9uj00EabG0BbbZd0s+mfhFrcmOA4f2J1BQc7Jo9hk8bCCDhw1s8XpaEmvyygEiQn5R0CoTi6UNccbtJ1HUraDKLJ3n8xAo8DP64bNrHRuuiPD7D/NYs7K0tcVs09gRisVisQC91uvBYz/fzduPvM+sz2czYEg/Djx771oZwCfc9grP3PQSLreLeDTOLkdsx8WPnNMuTVTZxioUi8VicSgsLuDoSzOnKfxg/Kc8e9NLRCrWTk7+9JUv8fq9XPzIOa0hYpvGmrwsFoulgYz/98uEK6qvdRQNxfjguSm1tndG7AjFYrFYUli1tIRfvppDUfcC5sz4g7fGvUcslmDU8TuzcvHqjOeVl1TgD3buvGlWoVgsFgtmZvoT101g4l2v4/F6CJdH0GSSynSHz455OePiefldghT37pIDqdsWVqFYLBYL8Nn/vublsW8SDceqpbOvJBqKImKiv+KxRFV4sS/o5dyxpzY6iWNHxCoUi8ViAV69961qmcDTEamIsvvxO+NyuZj12S+sM7g3x191OMNGNj7NfEfEKhSLpROhqhD/AZIlkDcMcRX+f3v3Hh5Vfedx/P2dZC6ZkAQhQctdhIWyFigExCrURyHeKDeholZptaI+dN1HFhDXrYhtdcVuL+5Kl1ZxrQVURBELyiIg4ApoEDVARAGRBCiEa4BcJpP57R8z0MBMmIFM5syc+b6eZ57MnDnPOZ/8njPznd+5/ayOlDSOHzkZdR6Xx0mXXp0YN9X2I5lfEC0oSqUJ49+NOXJ3cOAtHGDqMDmTcGT/xOpoSeGqkQMo+2JPxN1dpzgyHBTd9f0EpkotutNPqTRgjMEcuQfqy8FUgTkB1MLx32F8H1kdLymM/uebyG/XGrc3eIGiSPCR6crEk+0mv30rnnrnUVpdknpD8yaK9lCUSgf+rRCoIDhwakPVmJMvI64BVqRKKtl52fz3ppm8O2clG5ZuoqB9a0b87Aay87z4fX7a/0PblLv7b6JpQbG5QCBARdkhWrT0kp2nt8hPW4FjQCP3jgscTmiUZJbVIotRD97MqAdvtjpKStKCYmNrF67n2YnPU328mkAgwMBh/Zg8ZyLenOa/c6pKMs5eYCIdG/CApyjhcZQ96TEUmyrd8BVPj/9Pjh44Rm21j7paP+v/upFf3vobq6MpC4ijBeQ8DHiAU7ttPJDRDskaa2EyZSfaQ7GpV2cuCht9rq7Wz2fvb+FA2UHadMi3KJmyiiP7Dozz25iql6H+EHiGIFljEIe3WdYXCARYNf//WPKn5dTV+hnyo0Hc+NMhuJppoCplPS0oNrVv537OujsEAE63k4N7DmtBSVPi6ou4+iZkXb++exZrF64/fbHg1yXfsOqVD/mP9x8nI0PHArIj3eWVgo7sP8ryl1ezesE6qk/WRJyn1+CeZDrDP7R1tX469Wzf3BFVmtu1pYw1C9adceV5bZWPnZ/tYsOSTyxMBr4aH3+dvZwpQ2YwY8yv2bSyxNI8dqI9lBTzxrNLeGHa3OBojyKYgGHGoqn0ve47Z8z3wykjeO8va6iqrD59zyFPtptbJg0jO7d5dnEodcrnq7dGnF59ooZP3vuc7w3vn+BEQb7aOh4a/Bi7t5afvt188bJPGTdtFHc8eoslmexEeygpZMdnu5jzyDx8NXVUn6ih+ng1NSdrmD5yZlhPpaB9a/6wcSbX3n41rdteRJfenXho9n2Mf/xWi9KrdNKyTS4ZmeFfL053Jq3bWndh4Kr5H7C7tPyMsUtqTtYy71cLOVpxzLJcdqE9lBTyvy+9T11t+KmfDofw0dJNfH/slWdMv6RzGx5+6Z8SFU+p0wYO60eGM/zrxZGRwdA7rbt1yYeLP454A8hMVyYla79g0OgrLEhlH9pDSSG1VT4CgfAj7QFjws7oSlb19fVUHj5OfX291VFUM3J5XDyzYjptOuXjaeEmK8dDbuscZrw5hfx2raMvoJm0LMjD4Qi/2t0YyG3VwoJE9qI9lBRy9egrWDF3TdgvrIC/nn5FvSxKFRtjDIv+6x3+/Phr1JysxeVxctsjo7h16ki9nYVNXda7M3/ZOYuvS3ZT5/PTtU/n4LE/Cw27bygr5q6hturvP8BEwJvj4fJBPSxMZg/aQ0kh/Yb24oqb++HJDg4z6nAI7iwXdz95e9LfsO7dF1fywiPzOHHkJH6fn6rKaub+YiFv/H6J1dFUMxIRuvTqRPfCyywvJgDd+nZh4u/vxu11483NIivHQ377fJ5e/pieyhwHcvZQlsmisLDQFBcXWx0j6Rhj2Lj8c9YuXIfb66Zo/DV07XOp1bGiuqPzAxzYfTBsem5+DgsPzLEgkUpn1Seq2bruS7y5Xrr3v8xWoy2KyEZjTKEV69ZdXilGRCgs6k1hUW+ro5yXQ/uORJx+/FDweIr+OlSJlNUii35DU+szlArsU5ZVUuvYo13E6Rd3KtBiopRNaEFRCXHvzDtxZ7nOmObOcjHhmbsueJnbincwdegTjGw1nnt7TWL1gnVNjamUaoImFRQRGSsiW0QkICKN7rMTkRtEZJuIbBeRaU1Zp0pN/a/vwy/enkb3AV3x5mbR9buX8tjrkxl0y8ALWt5Xn+zkX66ZzqYVJZw8WsWuzWU885PnWDzr3TgnV0rFqkkH5UXk2wSHgJsNTDbGhB1FF5EM4EtgKFAOfAzcZoyJfG+GED0or87lkRt/RfGyT8OmZ+d5WVgxJynOKFLKClYelG9SD8UYU2qM2RZltgHAdmPMTmOMD3gFGNGU9Sq1fdPOiNP9Pj9H9h9NcBqlFCTmGEo7oKzB6/LQNKUu2MWd2jT6Xo5e8ayUJaIWFBF5T0Q2R3jE2suIdBl0xP1sIjJBRIpFpLiioiLGxat0dOf0sbi9Zx3k97q5ecIQ3Flui1Ipld6iFhRjzBBjzOURHm/FuI5yoEOD1+2BvY2s64/GmEJjTGFBQUGMi1fp6Iqb+vLgrHtp2SYPp9uJ2+vmBw8UNemsMaVU0yTiwsaPgW4icimwBxgH3J6A9SqbK7rrGob8aDCVh46TnefF6dKhZZWyUlNPGx4lIuXAlcASEVkWmt5WRJYCGGP8wM+AZUAp8JoxZkvTYisV5HA4aFmQp8VEqSTQpB6KMeZN4M0I0/cCNzV4vRRY2pR1KaXOnzGGt557l/lPvsHRiko69mjH/b8Zr7cdUc1Cr5RXysbmPfkGL0yby+G/HSVQH2DXljKmj5xJydpSq6MpG9KCopRN1fnqePXpRWcMdwtQW+3jxZ+/YlEqZWdaUJSyqWMVlQTqAxHf211anuA0Kh1oQVHKpvIKcpGMyB/xDt3bJjiNSgdaUJSyKafLyQ8nD8fjPfNCT7fXxY+fGGdRKmVnWlCUsiF/nZ/Vr31IRdkhel7VPXg7Ggn2TB5bMJne1/yj1RGVDemIjUrZTE1VLZMG/5yyL/dRc6IGpyuTjMwMnnrn31JupE+VWrSHopTNLJ61jG9K91BzogaAOp+fmqpa/v3OZ6mvr7c4nbIzLShK2cyq+WvxVfvCpvuqfXxdstuCRCpd6C4vpRqoKD/Ewt++TcnaL+jQvS1jJw/nst6drY51XpxuV8TpgYDB5Yn8nlLxoAVFqZC9O/7GxP7TqKmqwe+rZ/snO/ngzQ1MXziF/tf3sTpezH7wQBFfl3xDzcm/X9AoAvntWunpwqpZ6S4vpUJe+Nd5VFVW4fcFjzMEAobaKh+/u282TRkqO9Guu2MQg8dciSvLhdvrJisni7yCPGYsmopIpOGJlIoP7aEoFfLpys0EAuGF48j+oxw7WEnLgjwLUp0/h8PBlBcncuvDI9n8wRe0uqQlhdf3JtOpH3fVvHQLUyqkxUXZVB46HvE9T7YnwWmarmOPdnTsoaNtq8TRXV5KhYyZNAz3WVeVO91Orh49MOxqc6VUOC0oSoUMu6+IYROG4PI4yc7z4vI46XPt5Tw0e4LV0ZRKCZKsBxsLCwtNcXGx1TFUGjp2sJLdpXto0zGfizsVWB1HqfMiIhuNMYVWrFuPoSh1lrz8XL4zKNfqGEqlHN3lpZRSKi60oCillIoLLShKKaXiQguKUkqpuNCCopRSKi60oCillIqLpL0ORUQqgG+szhGSDxy0OkSS0zaKTtsoOm2j6KK1USdjjCUXUCVtQUkmIlJs1YVCqULbKDpto+i0jaJL5jbSXV5KKaXiQguKUkqpuNCCEps/Wh0gBWgbRadtFJ22UXRJ20Z6DEUppVRcaA9FKaVUXGhBUUopFRdaUCIQkbEiskVEAiLS6Ol5InKDiGwTke0iMi2RGa0mIq1EZLmIfBX6e1Ej89WLyKehx+JE57RCtO1CRNwi8mro/Q0i0jnxKa0VQxv9WEQqGmw7P7Uip1VEZI6IHBCRzY28LyLybKj9PheRvonOGIkWlMg2A6OBNY3NICIZwHPAjUBP4DYR6ZmYeElhGrDCGNMNWBF6HUm1MaZP6DE8cfGsEeN2cQ9wxBjTFfgt8HRiU1rrPD47rzbYdp5PaEjr/Q9wwznevxHoFnpMAP6QgExRaUGJwBhTaozZFmW2AcB2Y8xOY4wPeAUY0fzpksYI4KXQ85eAkRZmSSaxbBcN2+514DoRkQRmtFq6f3aiMsasAQ6fY5YRwJ9N0HqgpYh8KzHpGqcF5cK1A8oavC4PTUsXFxtj9gGE/rZpZD6PiBSLyHoRSYeiE8t2cXoeY4wfOAa0Tki65BDrZ+eW0O6c10WkQ2KipYyk/P5J2yGAReQ94JIIbz1qjHkrlkVEmGarc7DP1UbnsZiOxpi9ItIFWCkiJcaYHfFJmJRi2S5sv+1EEcv//zYw3xhTKyL3E+zRXdvsyVJHUm5DaVtQjDFDmriIcqDhr6b2wN4mLjOpnKuNRGS/iHzLGLMv1NU+0Mgy9ob+7hSR94HvAnYuKLFsF6fmKReRTCCPc+/esJuobWSMOdTg5Z9Is+NMMUjK7x/d5XXhPga6icilIuICxgFpcRZTyGJgfOj5eCCsVyciF4mIO/Q8H7gK2JqwhNaIZbto2HZjgJUmva4wjtpGZx0PGA6UJjBfKlgM3BU622sgcOzULmhLGWP0cdYDGEXwF0AtsB9YFpreFljaYL6bgC8J/uJ+1OrcCW6j1gTP7voq9LdVaHoh8Hzo+feAEuCz0N97rM6doLYJ2y6AJ4DhoeceYAGwHfgI6GJ15iRso6eALaFtZxXQw+rMCW6f+cA+oC70XXQPcD9wf+h9IXim3I7QZ6vQ6szGGL31ilJKqfjQXV5KKaXiQguKUkqpuNCCopRSKi60oCillIoLLShKKaXiQguKUkqpuNCCopRSKi7+H2dVanrLzvHVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Part 3.8: Testing Shallow Network on circles600 Data-Set.\n",
    "np.random.seed(200)\n",
    "\n",
    "file = 'circles600.csv'\n",
    "# Training shallow neural network for circles600 data-set, using the train data calculated in Part 2\n",
    "# with best parameters found.\n",
    "w_600, b_600 = Stochastic_NN(X_train[file], y_train[file], 0.5, 5000)\n",
    "\n",
    "# Evaluating the model on training data\n",
    "print(\"\\nEVALUATION ON TRAINING DATA:\")\n",
    "training_predictions_600 = predictNN(X_train[file], w_600, b_600)\n",
    "print(f\"Training Accuracy: {round(accuracy_score(y_train[file], training_predictions_600), 4) * 100}%\")\n",
    "\n",
    "# Using the model to predict classes of test data, using the test data calculated in Part 2\n",
    "print(\"\\nPREDICTIONS FOR TESTING DATA:\")\n",
    "predictions_600 = predictNN(X_test[file], w_600, b_600)\n",
    "print(f\"Accuracy: {round(accuracy_score(y_test[file], predictions_600), 4) * 100}%\")\n",
    "\n",
    "plt.figure()\n",
    "plt.scatter(X_test[file][:, 0], X_test[file][:, 1], c=predictions_600) \n",
    "plt.title(f\"Testing Result for blobs300 dataset for Shallow Neural Network\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 4 Solution: Challenging Task\n",
    "\n",
    "Following section contains solutions for loading CIFAR-10 data set, converting it into grayscale 1-dimensional vector consisting of **dog** and **deer** images and classification between the classes of **dog** and **deer** using shallow neural network with **one hidden layer**. The code used for loading and visualising the CIFAR-10 dataset is based on [4], [5]."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "# unpickle is taken from [5]\n",
    "def unpickle(file):\n",
    "    '''\n",
    "        Method to load CIFAR-10 data-set\n",
    "    '''\n",
    "    import pickle\n",
    "    with open(file, 'rb') as fo:\n",
    "        dict = pickle.load(fo, encoding='bytes')\n",
    "    return dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loadbatch is taken from [4]\n",
    "def loadbatch(batchname):\n",
    "    '''\n",
    "        Method to load batch from data-set\n",
    "    '''\n",
    "    folder = 'cifar-10-batches-py'\n",
    "    batch = unpickle(folder+\"/\"+batchname)\n",
    "    return batch\n",
    "\n",
    "# loadlabelnames is taken from [4]\n",
    "def loadlabelnames():\n",
    "    '''\n",
    "        Method to load names of available labels for CIFAR-10 data-set\n",
    "    '''\n",
    "    folder = 'cifar-10-batches-py'\n",
    "    meta = unpickle(folder+\"/\"+'batches.meta')\n",
    "    return meta[b'label_names']\n",
    "\n",
    "# visualise is taken from [4]\n",
    "def visualise(data):    \n",
    "    # Plot.imshow requires the RGB to be the third dimension, not the first, so need to rearrange\n",
    "    picture = data.transpose([1, 2, 0])\n",
    "    plt.imshow(picture)\n",
    "    plt.show()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Following code is based on [4]\n",
    "\n",
    "# Loading data from all five batches\n",
    "batch1 = loadbatch('data_batch_1')\n",
    "batch2 = loadbatch('data_batch_2')\n",
    "batch3 = loadbatch('data_batch_3')\n",
    "batch4 = loadbatch('data_batch_4')\n",
    "batch5 = loadbatch('data_batch_5')\n",
    "# Loading test data from test_batch\n",
    "test_batch = loadbatch('test_batch')\n",
    "\n",
    "# Getting data (RGB features) from batches\n",
    "train_data = np.concatenate((batch3[b'data'], batch4[b'data'], batch5[b'data']))\n",
    "# Getting labels (RGB features) from batches\n",
    "train_labels = np.concatenate((batch3[b'labels'], batch4[b'labels'], batch5[b'labels']))\n",
    "# Getting test data from test batch\n",
    "test_data = test_batch[b'data']\n",
    "# Getting test labels from test batch\n",
    "test_labels = test_batch[b'labels']\n",
    "\n",
    "# Loading label names\n",
    "names = loadlabelnames()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Following code is based on [4]\n",
    "def extract_images(data, labels):\n",
    "    # initialising data set and labels for storing dog and deer images\n",
    "    X = []\n",
    "    y = []\n",
    "    dogs_deer = []\n",
    "    # Extracting dog (label: 5) and deer (label: 4) images from the data-set of all images\n",
    "    for i in range(len(labels)):\n",
    "        if labels[i] == 4 or labels[i] == 5:\n",
    "            picture = data[i]\n",
    "            picture.shape = (3,32,32)\n",
    "            dogs_deer.append((picture, labels[i]))\n",
    "    for picture, label in dogs_deer:\n",
    "            # Converting the images into 2D grayscale by taking mean over each color channel\n",
    "            gray_picture = np.mean(picture, axis=0)\n",
    "            # Converting the image into 1D vector using flatten() and saving in the data-set\n",
    "            X.append(gray_picture.flatten())\n",
    "            # Saving corresponding label of image in y (class 0 for deer and class 1 for dog)\n",
    "            if label == 4:\n",
    "                y.append(0)\n",
    "            elif label == 5:\n",
    "                y.append(1)\n",
    "    # Visualing few images from dogs_deer\n",
    "    visualise(dogs_deer[1][0])\n",
    "    visualise(dogs_deer[7][0])\n",
    "    \n",
    "    return np.array(X), np.array(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAfFElEQVR4nO2da4yc53Xf/+ed+8zeeVkul6RISpQlWa5olxCE2jUcJw0Uw4BsoAnsD4Y+GGFQxEANpB8EF6hdoB+corbhTy7oWohSOL4ktmMhVZsYagrDQCCLYiRKIiXexDu15HLvu3Of0w87DCjl+T+74u7O0nr/P4DY5XP2mffM875n3pnnP+ccc3cIId7/JJvtgBCiNyjYhUgJCnYhUoKCXYiUoGAXIiUo2IVICdm1TDazxwF8G0AGwP9w969HD5YveL5YDtsyRucVc7mwIaIazi0uUpvHJkb8SDKZ8DgicxK+xLlCntrcOtSWzXL/O63wPE/463omQ9YXQL3WoLZGtU5t3g77mM8V6Jx2p01tHYuds/B5AYBsPmwzfsrQqvHn1a5zH5OEn8+ELzGy2bAzjaUaneOd8HluNRtot1rBB7Q71dnNLAPgFIB/A+AygBcBfN7dT7A55YFhf+DR3wraRvr5Qj0wNhYcb7f5GXv+H/6B2hrZJrVZxI++oYHgeNGKdE4pP0xt4/vvobZGqUptw8Pc/8Zk+EWuVq7QOQNDo9R27sQFart84jy1tabDLxLjY3vpnIX6LLclPMgw0k9NW3YPBccL/PUBU6fPU9vs2WlqK5d2UVtpB3+xHRkJvwBeevUNOqc1Hz7PV946hXp1KRgYa3kb/yiAM+5+zt0bAH4I4Ik1PJ4QYgNZS7CPA7h02/8vd8eEEHcha/nMHnqr8M8+E5jZYQCHASBXLK3hcEKItbCWO/tlALtv+/8uAFff/UfufsTdD7n7oWxkc0YIsbGsJdhfBHDAzPaZWR7A5wA8uz5uCSHWmzt+G+/uLTP7EoC/xbL09rS7vx6bkyQJSn1h6c28Rec1GuGd3Qz4nGaV75pOzM9T2+DoFmpLsuHlyg3zne5KiX90SVr8tXZxhu+4dxr8uSXz4V18i0h509UlamsvcVviXA3ZsTOsoOQqfD12DPIt8ouT/+xN4z/hGa4oNTvhdZydmuFzFrkSUq9xOWxomIfTXI0rDbXJ8GMu1fj65lvkWJFzsiad3d2fA/DcWh5DCNEb9A06IVKCgl2IlKBgFyIlKNiFSAkKdiFSwpp2498rSSZBsRKWqTILc3Rek0hviwmXGbY/eJDa2ks8k6svIqPlSmGpb3AgLCcCQL7K5UHUeXZVqxmRk5ynUPV3wok8SauPzpmfW6C2DFeaUM7xpKFde3YHxy9Nctmwk+XnpX+EJxstReTBmWOXguPZiO+FLD9Wq8Svj5ZHMuL44VAshyXHVomfs4KT85xw+VJ3diFSgoJdiJSgYBciJSjYhUgJCnYhUkJPd+MBwMnLSzZSR6xOduM7Zb4LPjS+h9pqTb5r2urwBIkiSdQY2T5I55Qm+E5xLrJ721/mCSN7dvPj4Wb4eDdrXBVoz3JbbYbv1GcjtfearfBjLkQUiOIwf7yHHv4Atb30j29SWycJqxrZWN29Ra4KLM3z9ajwCmTYtYeX/rpnNLzr/toNnpBTrUXKdBF0ZxciJSjYhUgJCnYhUoKCXYiUoGAXIiUo2IVICT2V3ixJkC+EK8zmYkkhrbAU0p/nr1VX529QW9X5sbZt5xLg9p3hziNJpFuJZSJSU4vXmRuqROS8Pp4k05gLr1UyyyXAPLj/1QUuNQ0Ob6O2NmlPFKsXV2BtvgDUI3XyqgX+mMP/8kBwPFvlMl/jxGVqS7KR+2OkRdXQEE+geeTh+4Ljs2f52p+8cS44HuvvpDu7EClBwS5ESlCwC5ESFOxCpAQFuxApQcEuREpYk/RmZucBzANoA2i5+6EVZiCTDcsrSYa/7hQyYTeTiNDQF1bJAABbyvxYgyNc/mktLQbHp2tcQtud4/XMtoEXJps13gTT8rxFVT0bzjYbiMhriEiY23fxbK3hAW5rtMMSZi7L21BNzXKpae4mr11HLqmuLfzcahFJsbCDZxx2GrwuXI4cCwCaS/wauTIR9mUxUr6w2Q4/njuPifXQ2X/L3SfX4XGEEBuI3sYLkRLWGuwO4O/M7CUzO7weDgkhNoa1vo3/qLtfNbPtAH5hZm+4+y9v/4Pui8BhACjGPkgLITaUNd3Z3f1q9+d1AD8D8Gjgb464+yF3P5Qv8jJSQoiN5Y6D3cwqZtZ/63cAvwvgtfVyTAixvqzlbfwogJ+Z2a3H+Qt3/z+xCe4d1Eh2W67NpaFyf1gKqUWyxraNb6W2fJkXFMwUuDTUJq2c+vNc+0kihR5LEbmx3uJ9l7zJ51X6w1UPCzxpDB0LS4oAYP1calqc5etfrYXPc8G471cmuagzOsg/AhopKgkACbnCh4eG6Jz5Je5HLeH+15p8PZYa/HwOk4Klozv5NXz+tdPBcePJfHce7O5+DsAjdzpfCNFbJL0JkRIU7EKkBAW7EClBwS5ESlCwC5ESelpwstPpoFYN968qNLhs4Z1wBphFsp1aERmkQyQ0AGhE/CgVwzJUIYlkSdXnqc15DULsyvKMuEuRjC3WwqwdyXorRqRDd16Ac6nD13FpMXyeq3Uu8+Uj956BSGrbliGeWVglkm7SjDyvKX7Oaku8gOjW8QFqsyIPtbn5ieD4YpVn+hUK4cezhGtvurMLkRIU7EKkBAW7EClBwS5ESlCwC5ESerobDyzvyIdoNHhyytR0eFdyJJIoUEz47m02y+u7LURaQ1XnwjvMSTs8DgB9g7yNU6vMfdxFEloAYCqSgDJTDe92J5HEmkxkrXLOL5FmpKxdkg+v8UiFt4zaGbE9sofb9jw4Tm2/OvNGcPzS9CydM7/As4YaVX59lAs8hbvV5Ot/7Vp4N356iu/GJ0lYTTBoN16I1KNgFyIlKNiFSAkKdiFSgoJdiJSgYBciJfRUenMHOh1eL4zR1xdOQBmq8EyS6WmezDBb59LKIs+PQGUknPCypcQTMdqR2no32lzGieQzYGxkhNomroRlnP4Cf2LtasTHqSlqm49ob61CeE2ykXpxW8a4lFoZ4Wu8WJ+htjrCkm4tcpsrROTSUpPLa7V5LsEm/XytqgvhpCcDP5YZv76pD+95hhDiNxIFuxApQcEuREpQsAuREhTsQqQEBbsQKWFF6c3MngbwaQDX3f3h7tgIgB8B2AvgPIA/cHeeotMlk0lQqYTlq8E8l4ZGt28JjmcjvW5uvn2T2m7M8Yyn0g6ebdYqhGWjhYicWCtyH2vO5Zj5Pl7fLee8jtswaa9UIFIYAJx66y1qm1ni9wMrcYlqoRaWhpIMX6vGNG+9dW2GP+eW8/N5k7TYmpvn56XT5mufz0SyyiK3ziuXzvPjJeFWVCXj12JiJF4iMbGaO/ufAXj8XWNPAXje3Q8AeL77fyHEXcyKwd7tt/7ub1Y8AeCZ7u/PAPjMOvslhFhn7vQz+6i7XwOA7s/t6+eSEGIj2PCvy5rZYQCHAaBQ4e1/hRAby53e2SfMbAwAuj+vsz909yPufsjdD+WLvJmCEGJjudNgfxbAk93fnwTw8/VxRwixUaxGevsBgE8A2GpmlwF8FcDXAfzYzL4I4CKA31/d4RxIwnLTjvEx7gMp6Dg7y+WYcpnLQvcM8eyqdjkihxE/ZsBbNbUSbsvU+LF4gydgb4UXzGzMhz8qHTt9ks5ZjBRRzBd4ocfFJZ7lVSRKn5f4JbdQ40VH5yLZg0mHy7ZJLlxM0yO+F3L8Hpir8GNlK/xcFztc+ty5P7zGtQtcRpu/g4KTKwa7u3+emH57pblCiLsHfYNOiJSgYBciJSjYhUgJCnYhUoKCXYiU0NOCk0nGUBkIy0ZLdS42LU1fC45nnGdQFXID1Na3jUtv0w3uR5YkZXme90qzNs/kSuq8Z9vk2/R7SljqTFLb0eOvB8fbeV68cGzfg9R27TqXqDItfq9YIkU9ywO8SOhgrL/d9avUVoikm709QbIfjZ+X3ftHqW1xgsuD9Sq/HrfuuIfa+ith2Xm+Hi4eCgDZTPiaszVmvQkh3gco2IVICQp2IVKCgl2IlKBgFyIlKNiFSAk9ld4y2QyGtoQlsckzvOhhuRHObquQzB8AyBS4tDIZ6V9WGuCS3ba+cObSlNfonHLk9XS0xGWoxRnev2zBeXZVaThcNKg0soPOmVrkEmDDuZTjEZlngkiH23N8zsj2cWp7+/oNaqtEinq2iI+VgX46h9TsBABUl/i5LiS8B1+rwWW5wb6wLHeuGpacASAhhTRj6M4uREpQsAuREhTsQqQEBbsQKUHBLkRK6OlufLvdwdxiOEGi0eK75wP58C740FC4bQ4AnJ/gLYGGx8PtpABgbCjS/qkVTpLxKk8WqVXDzxcA9u7fS218NYDRvfdR2xhx5exV3p1rZm6O2pbq/Ll1jLdJKpfDiRrVRa4ynDjxCrU1I4rH6M7d1Da4LVzfbXKGXx/X3+ZJN7UqPzOVwUhClPEaehcuXA6OLy7y81IhbagiAonu7EKkBQW7EClBwS5ESlCwC5ESFOxCpAQFuxApYTXtn54G8GkA19394e7Y1wD8IYBb2QlfcffnVnqsdruDqelwUotVuYyzaOG6X5kx3uKp6ryWXH2C1/baP8gTYaYuXQmONxf4sebmuPQ2neUJLfv38DporQaXwy68Ffbx5lxEzIvU8svk+LwSkdcAYCwfljCTSCJMXz+XUpMWTzKp8M5KyLbC105S4okw9Tz3o53l57PZ5G20inl+X51fCNcUzOS4XNehrbL4uVzNnf3PADweGP+Wux/s/lsx0IUQm8uKwe7uvwTAc0KFEL8RrOUz+5fM7LiZPW1m/GtnQoi7gjsN9u8AuBfAQQDXAHyD/aGZHTazo2Z2tFnjX3kUQmwsdxTs7j7h7m137wD4LoBHI397xN0PufuhHGvaLYTYcO4o2M3s9hYWnwXw2vq4I4TYKFYjvf0AwCcAbDWzywC+CuATZnYQy/v85wH80WoO5h1HsxqWJyIl45AhGT4N8lgAUBjhmW2dSJGxgUgmXXu2Lzj+5ukzdM7cEvfxRKQF0YEHeUumyQUuyVwm2X5JpN7dQD+XAI2X+UMuG2m/Rdp8VUrhcQAokJZGAJB0+HNGh7dkcnKu+8g1BQCVreE6fgAwUZuntsUWf8xGm0vLNSKjNVq8NmCLfCTudPg1tWKwu/vnA8PfW2meEOLuQt+gEyIlKNiFSAkKdiFSgoJdiJSgYBciJfS04GSSGCqFsMyTJOFsOAAYLIe/jON1/o28xTZ/HWtnuPwzvcQzyhLiR2UoLMkBQN+2rdRWqPCsvReOn6W263Nc4mlbOTheyXNZa2goPAcAcuCyXD4ih1VyYc2uHKmIWG9xmXIxUvjSOxEJsBg+1znjc/ryXG8cGd9JbWeneDHNiwu8fVXftvB1lWT5dbpYDUus7cha6M4uREpQsAuREhTsQqQEBbsQKUHBLkRKULALkRJ6LL0l6O8PF/qbfZtLE9VqOPunlC/xOZFss22RXm+FHF+Sejvsxyc+/jE655OPf5raJue4nPQXf/ks96POM68GB8PZbUMVvlaVPH/OhYTLULlIR7o8k7baXK6zhMtydodNzAysJxr33cCz6EpFbvvQQ3uoza/xDLZmjmTEDXBJ14vhPnBJwhdDd3YhUoKCXYiUoGAXIiUo2IVICQp2IVJCb3fjLUE+H06sqNb4znSNbOx2+nhdr2Jkh9mcz2vUeHuffbvGg+P777uXztm5c4za+rfx19qxXXxn99rN16ktY+HnlmnzXeQsXw7ksnwXPJ/l/lNbZBe8WYvsWDe5LUYmE1YTYrv7SSRJptPhCVv3732I2nY9wBNojr75cnB8Zn6azlnIhRObktjzohYhxPsKBbsQKUHBLkRKULALkRIU7EKkBAW7EClhNe2fdgP4cwA7AHQAHHH3b5vZCIAfAdiL5RZQf+DuXCsA0PEOakRiY5IcAJTLYTezkS/9V3L88ZqR2nUzU1yHmkvCtdpuXucJC//7fz1Hbb8+cY7aUBigpv5+ntRSSMKyUSlypgsdLmtZg8tQFrl88sWwj7mII7OLPMFnfn6B2pa7kIVh0lsuy5uMtiMyn4FfH4VIYtD47t3UdvLc8eD4gvOkoXYzLKW6r60GXQvAn7j7gwAeA/DHZvYQgKcAPO/uBwA83/2/EOIuZcVgd/dr7n6s+/s8gJMAxgE8AeCZ7p89A+AzG+WkEGLtvKfP7Ga2F8CHAbwAYNTdrwHLLwgAeOtLIcSms+pgN7M+AD8B8GV3D2fOh+cdNrOjZna0UeVfiRVCbCyrCnYzy2E50L/v7j/tDk+Y2VjXPgbgemiuux9x90Pufihf4htLQoiNZcVgt+WMge8BOOnu37zN9CyAJ7u/Pwng5+vvnhBivVhN1ttHAXwBwKtmdis95ysAvg7gx2b2RQAXAfz+6g4ZzspJjL/usIynQoG3NOKV04BOTNJocfmkkAvXz3NSmw4Azrx1idpOnT5Pbbvu5RlU0Uw0sowl0o4JALzBpchORMppR6SmVjPsSDYiiZYj7/zqEbm0XObtq7LZ8POOXAKwTiRjMlKfrhCTgvNc6mPdpjKRW3HH2drz87VisLv7r8AiFPjtleYLIe4O9A06IVKCgl2IlKBgFyIlKNiFSAkKdiFSQk8LThbyBdy390DQdvz8BTrPWYFIktEEAM0az07yCpfKCiUukRx88EPB8f4tPOvtnvs/SG17PxD8HhIA4FcvvkJtzUhrq2IpLG0lCT/VjYi81orIitaJFY8k2lak4KRT0QdAh/tYLPBzxiTdVuT6yBi/rpKIrZzjftSoVAZYrhAcb+f4erRJQc/YGurOLkRKULALkRIU7EKkBAW7EClBwS5ESlCwC5ESet7rrZQPywydiMRTb4UzjdotPqeS5RlUxW28mGOuzgtszE6Fa3aUh8LZcABwY/IGtV04c4Yf6zqflytyqS8h0ksukm3WqPPX/FYrlh7GTQVynuuRx1tc4GtfKoYfDwCSSKZXuxWW2GKyoSXcx1KZy2uVIr8OXjrFi4u+eXEiON6s8+u7Qwtfrq3gpBDifYCCXYiUoGAXIiUo2IVICQp2IVJCT3fjG406Lp4/G7QlFvkCP0l4KUfqqhX5RiYKCd/ZnV+4SW0nyY7qsdPh5wQA127MUNvsUqSGW4PbKn2RCnukuFrGIrXJIrYkopIMVirUVsyH6wNOT/M2TosLvEJ5zH90+O45e26lAa7WVFs80Sh2ezx6LNzGCQD+79Ur1HZi4nJwfCAihFQXZ4Pjncha6M4uREpQsAuREhTsQqQEBbsQKUHBLkRKULALkRJWlN7MbDeAPwewA0AHwBF3/7aZfQ3AHwK4lbHxFXd/LnqwTIJtJGlkLpJgMJQLJ34UIm2QMnUuQczOLlJbpcRbCTVIe5+zF8LSCQC0Ovz1tBNrMxRRmtDh9dPaRLJbmOMSoEekpsEKT6AZ7uNrNbcQltiqVd7GKZJXE5UHy0XuYx+5rkpFfr15RLZtRK6rv/nrv6a22bGd1FYeHgqOl0gSDwBk+8KyZxLpGbUanb0F4E/c/ZiZ9QN4ycx+0bV9y93/2yoeQwixyaym19s1ANe6v8+b2UkA4xvtmBBifXlPn9nNbC+ADwN4oTv0JTM7bmZPm9nwOvsmhFhHVh3sZtYH4CcAvuzucwC+A+BeAAexfOf/Bpl32MyOmtnR6iL/rCyE2FhWFexmlsNyoH/f3X8KAO4+4e5td+8A+C6AR0Nz3f2Iux9y90OlyHephRAby4rBbmYG4HsATrr7N28bH7vtzz4L4LX1d08IsV6sZjf+owC+AOBVM3u5O/YVAJ83s4NYLnp1HsAfrfRAiQH5JCyh7Nyxg84rk4y4JOEZVB7RcZYi2UT50S3UduVCuFbY1DzP1hocGqG21hKXoep1bltaiOhyHq7Xl7SX6JRCOEENADA6zNcja3whC7nwg7IMRgDIEGlzJQrZSBZgO+zj3Bz/SFmP+FGt8jp528e2Ult+lG9pzZIai//qscfonMrBsFx67fSbdM5qduN/hbAEGtXUhRB3F/oGnRApQcEuREpQsAuREhTsQqQEBbsQKaGnBSe900GrEZYu9uziWUFLU+Hien1Frq8VPJIxVORaUyvHH3O6EZZrTpzhcscH7v8Atc1O8Uy0bMJPTV8fz9iqE2moWeCv6yODg9TWiWTEzc9y28AWIqVOcpmyWOFFIEsVXiS03QxLVwCQyYdluVIff84Ti1xe84TLfAceuI/ajpMCkQBQIvLgvt276Jw2ydzMEskT0J1diNSgYBciJSjYhUgJCnYhUoKCXYiUoGAXIiX0VHrLZBIM9YcllJzz3mb5TDjLa88uLk1Mdc5TW67Ms7W2DPElKe7fHhz3F7kcc+riRWrLRvpyjW0JFyEEgGKBr9X2gbCPhUhfvOGhAWprNHjvu0ady5SdfFi+ahvPvhsc3UNtu4Z51li+xmWt2kL4eNXaPJ0zMswLWE42wgVTAaBT52s8bryKZWH3NvKA/Po4eeJ0cLxW5XKo7uxCpAQFuxApQcEuREpQsAuREhTsQqQEBbsQKaGn0ls2Y9gyGJY1SogUIuwPSxP77uGNaa4vTVPbthFeBDJb5H4Ut4bn7d3HJaM3TvCMuF3bR6ltocolqlqd2+7fHc4e7CvzbKhiX0RuzPBCoN7mj3lzKSwBeYEf6+0ql9AGt3B58KF7xqitTGpznjt1hs4plnmG3cIsz7DLZfm905zLeeP79gXH52cikmItLG12IhK27uxCpAQFuxApQcEuREpQsAuREhTsQqSEFXfjzawI4JcACt2//yt3/6qZ7QPwQwAjAI4B+II76T3UJZdJsGOoHLQVM3xnt0FqgtWXeD2zLYN893Yuz3dba3X+FLYOh9v7/OvHgj0tl20HP0ht7TZPJDl9/m1qKxhv/7R3d1gx8HakRdUoX6u92/ZS2+R1/phZ0rH34iWegDLf4nUDj51+g9ryHZ4Q9bGHHwyO723yOW9eukJt/Qm/PoaI0gQAGZ4Hg858+Po+d+oynXP16qXgeLPB/VvNnb0O4JPu/giW2zM/bmaPAfhTAN9y9wMApgF8cRWPJYTYJFYMdl/mVgfFXPefA/gkgL/qjj8D4DMb4qEQYl1YbX/2TLeD63UAvwBwFsCM+z/Va74MgH/DRQix6awq2N297e4HAewC8CiA0Aeh4AdJMztsZkfN7OjcHG+xLITYWN7Tbry7zwD4fwAeAzBkZrc2+HYBuErmHHH3Q+5+aGCgby2+CiHWwIrBbmbbzGyo+3sJwO8AOAng7wH82+6fPQng5xvlpBBi7awmEWYMwDNmlsHyi8OP3f1vzOwEgB+a2X8B8I8AvrfSAyUGVIhs1KyFpRoAaNXCiR8zDS7j9Jf5u4jCEE+EKVV4jbGshaWVzBLXVR4ev5/azl6ZoraLk7yW2Af38VZZH3ogvHUyMx2WagCgb8duatuxlUtU2eQGtXk1LOfdX+QtnqqR1kXzs3ytFto1ajt1Kfy89w1wuXGwwJOhSiUu22bAa8bVEz6v0hf25dAjj9A5MzfD65HN8Pv3isHu7scBfDgwfg7Ln9+FEL8B6Bt0QqQEBbsQKUHBLkRKULALkRIU7EKkBHPnGVTrfjCzGwAudP+7FcBkzw7OkR/vRH68k980P+5x92DRxp4G+zsObHbU3Q9tysHlh/xIoR96Gy9ESlCwC5ESNjPYj2zisW9HfrwT+fFO3jd+bNpndiFEb9HbeCFSwqYEu5k9bmZvmtkZM3tqM3zo+nHezF41s5fN7GgPj/u0mV03s9duGxsxs1+Y2enuz+FN8uNrZnaluyYvm9mneuDHbjP7ezM7aWavm9m/7473dE0ifvR0TcysaGa/NrNXun785+74PjN7obsePzIjaZgMd+/pPwAZLJe12g8gD+AVAA/12o+uL+cBbN2E434cwEcAvHbb2H8F8FT396cA/Okm+fE1AP+hx+sxBuAj3d/7AZwC8FCv1yTiR0/XBIAB6Ov+ngPwApYLxvwYwOe64/8dwL97L4+7GXf2RwGccfdzvlx6+ocAntgEPzYNd/8lgHcnJD+B5cKdQI8KeBI/eo67X3P3Y93f57FcHGUcPV6TiB89xZdZ9yKvmxHs4wBuryiwmcUqHcDfmdlLZnZ4k3y4xai7XwOWLzoA2zfRly+Z2fHu2/wN/zhxO2a2F8v1E17AJq7Ju/wAerwmG1HkdTOCPdQZYbMkgY+6+0cA/B6APzazj2+SH3cT3wFwL5Z7BFwD8I1eHdjM+gD8BMCX3Z13oOi9Hz1fE19DkVfGZgT7ZQC310GixSo3Gne/2v15HcDPsLmVdybMbAwAuj+vb4YT7j7RvdA6AL6LHq2JmeWwHGDfd/efdod7viYhPzZrTbrHfs9FXhmbEewvAjjQ3VnMA/gcgGd77YSZVcys/9bvAH4XwGvxWRvKs1gu3AlsYgHPW8HV5bPowZqYmWG5huFJd//mbaaergnzo9drsmFFXnu1w/iu3cZPYXmn8yyA/7hJPuzHshLwCoDXe+kHgB9g+e1gE8vvdL4IYAuA5wGc7v4c2SQ//ieAVwEcx3KwjfXAj49h+S3pcQAvd/99qtdrEvGjp2sC4F9guYjrcSy/sPyn267ZXwM4A+AvARTey+PqG3RCpAR9g06IlKBgFyIlKNiFSAkKdiFSgoJdiJSgYBciJSjYhUgJCnYhUsL/BxfiAZ+yY+qqAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAYEUlEQVR4nO3de5CddXkH8O9zLnvJ7pJkk82FJJIYM0hEDbgSHBylWpGiHWCmOvIHwx/U2I6M0tqZUtqpdGw72Kk6/mUnChU7SkhFBkZtBSkVr2hECCEh5MLmTjaQ22av5/L0j/MyXeLvefbsu+85Z5ff9zOTye77nN95f+c95znv2fc5v99PVBVE9MaXa3UHiKg5mOxEkWCyE0WCyU4UCSY7USSY7ESRKMyksYhcC+CrAPIAvqGqd09xe9b56iQiZozlUvKoavDFI2lfOCKSB/AigA8BOAzgNwBuUtWdThv1XsSxyefzZsw7TuVyOdN+8I2ltaxjnCZXVNVM9pl8jL8CwF5V3a+qEwC2ALh+BvdHRA00k2RfAeDQpN8PJ9uIaBaayd/soY8Kv/d5REQ2Adg0g/0QUQZmkuyHAaya9PtKAEfPv5GqbgawGeAFOqJWmsnH+N8AWCcia0SkDcAnADySTbeIKGupz+yqWhaR2wD8CLXS272q+nxmPWuRtFdA08S8q+relfqsefuqVCpmjFfqs1EohNMw62OfuvSWxlwovTUz2T1eAlar1VT3abFebACTvRms5zrtsW9E6Y2I5hAmO1EkmOxEkWCyE0WCyU4UiRmNepurvCvuuZz9/lfV8FXwzo5Os82SpUvNWGliwoyNj4+bsZMnT5qxNIMqvKu+1HjWay7r54VndqJIMNmJIsFkJ4oEk50oEkx2oki8Ya/Ge99w92LVqn0FdP36twW39/X1mW1yOfs7zAMvHTRjZ86eNWP6+9MGTInfY5+9SqXStNtY1RXveeaZnSgSTHaiSDDZiSLBZCeKBJOdKBJMdqJIzOnSm7+Sid3Om9Zp5cqVZuymm24Obj83PGK2eezRh8zY/v37zZg6j02cch7CMxJRRrzSVrFYNGNpVt3x9pVmkAzP7ESRYLITRYLJThQJJjtRJJjsRJFgshNFYkalNxEZADAEoAKgrKr9WXSqXuq8VxXaO8zYxiuvNGPXfPjD9g7zbcHNK9deaDb58HUfMWM7nttjxsYmxux+sLzWMt4chT09PWbMm1NwdHQ0uD3rkYpZ1Nn/QFVfyeB+iKiB+DGeKBIzTXYF8KiI/FZENmXRISJqjJl+jL9KVY+KyBIAj4nIC6r65OQbJG8CfCMgarEZndlV9Wjy/yCAhwBcEbjNZlXtb/bFOyJ6vdTJLiJdItLz2s8ArgGwI6uOEVG2ZvIxfimAh5LRPAUA31HV/86kV+exRgxVq2WzjTU5JADc/hd/aca88smJEyeC2xcvtJd/Ojdov58Wc3YJbcyrurDy1jJeOWxkxB79mPY+s5Q62VV1P4B3ZtgXImoglt6IIsFkJ4oEk50oEkx2okgw2YkiMScmnExTmjhzctCMPfzdLWasVLbX3epsC08o+N7LVpttXhl4wYzlnNKh/z5sT5hJjZVm4kgg3XpuWeOZnSgSTHaiSDDZiSLBZCeKBJOdKBJz4mq8JedcGT0w8JIdOzBgxrwrqhevWhrc3r/0g2ab0qvHzRicZagEdj9E8/Z9Gv0X2MsFeVPaqTvqxulH5tJVIKyj6D0qVXtffYsXmbHe3l4ztnffPjNWrYT3l3N6aT0ub1EontmJIsFkJ4oEk50oEkx2okgw2YkiwWQnisScLr1ZZaYau2whYpeMRO3BKV0d7cHt+fKE2aZQtfvoFZO8cpg4Lc124r2v2zvLuSXA5g3Iqaacd6+QDz/XVafs6YTcgTBnz56x77NiF8WsEnLeK70ZoWrFfr54ZieKBJOdKBJMdqJIMNmJIsFkJ4oEk50oElOW3kTkXgAfBTCoqpcm23oBPABgNYABAB9X1VON66bVuXTN1Cknue00XD4pT4ybbbo6wvPW1WJtZmxoyF5KKJ+3H3jVKONUynN83jrnKfNeBm1GubTszAlXduqep0/bL/OqU2b1qsT5Qrg82NVhLys2OjYa3F5y6ob1nNm/CeDa87bdAeBxVV0H4PHkdyKaxaZM9mS99ZPnbb4ewH3Jz/cBuCHjfhFRxtL+zb5UVY8BQPL/kuy6RESN0PCvy4rIJgCbGr0fIvKlPbMfF5HlAJD8b67IoKqbVbVfVftT7ouIMpA22R8BcEvy8y0AHs6mO0TUKPWU3u4HcDWAxSJyGMDnAdwNYKuI3ArgIICPNbKTaXjlGKdC4qoYEwNOOGWcC5csMGNXv/utZuzo4Gkzdsmll5ixBStWB7fv3GNPwHlw7x4zNnTKLjWNOSOsqkatyRv95WlrD5fQAGB0OFyGAoCzQ0PB7fm8fZ7zXjvlst1/9z6d0XJ5Y2Red1eX2WZk1HjMzmt7ymRX1ZuMkD2lKhHNOvwGHVEkmOxEkWCyE0WCyU4UCSY7USTm9oSTDr+6lq72dmboXHD70PCY2aarzZ7c8h1vDq8dBwDvv/xiM3bROrtk17v27cHt111rl35ePviiGRs+ccyMuTNm5sLnEa9M2TGvx4wtXhN+XAAwXrEf26M/+lFw+9YHtpptTg0Nm7G88bgAoFCwRzhWq/akpGNj4VGTY+P2aMo0r2Ce2YkiwWQnigSTnSgSTHaiSDDZiSLBZCeKxJwuvam3FpYzw58zAAnqBF85Ey7J7Dtir/HVMW++GcsV7H1V2uaZsWF7OTrI8SPh+3MmIuxp7zBjfWvW2ftyjtWpofCorFMjdply2UVrzdiiFXasc759jN+64fLg9ve892qzzT994QtmbN9L+82YOOvpuasSGqPezo3Zx6psPJ/efnhmJ4oEk50oEkx2okgw2YkiwWQnisScvhrvXQ32+Ffq7fscKYWvgP7qeXt+NzhX1S9ebw9oueBCeyBMW99yM1Yp2oMxLPmivQxVW7fd/04nljOuxg/utOe7O3rcHoBSKrxixjqMgSQAsKi3N7j9ssveZbbZuPFKM3ZuxO7jyZPnr6Xy/7zln4rFcBqOT9iDZ6w5/jw8sxNFgslOFAkmO1EkmOxEkWCyE0WCyU4UiXqWf7oXwEcBDKrqpcm2uwB8EsCJ5GZ3quoPG9XJ2cIqy73kLNV0wWG7ZPSWdy02YwtW2Us8FeZdYMaOHj8e3D509qzZptcoTwFAX7e9BNG8C+w59FavXBjcXhG77/t222W5vDO/m1eAHT4XLpXt+t2zZptf/OIXZqzLWZLJK71VnYFIJWNePq9EnEY9Z/ZvArg2sP0rqroh+feGT3SiuW7KZFfVJwHYb1lENCfM5G/220Rku4jcKyLhz2xENGukTfavAVgLYAOAYwC+ZN1QRDaJyDYR2ZZyX0SUgVTJrqrHVbWiqlUAXwdwhXPbzarar6r9aTtJRDOXKtlFZPJIjBsB7MimO0TUKPWU3u4HcDWAxSJyGMDnAVwtIhtQm/JqAMCnGthHU9aliSkZpbeS048zo/bIpcHTI2Zs557DZmzJsmVmrFyqBLfv33/AbHPwgL2vNWvtud/yOXvuujOvhh/bkZcHzTZnR8Mj5QBgsTMCrDiv3YyNGvf5gx/aBaT9A/Y8c4VCuoGi3mhKqyyXdlSnZcqeq+pNgc33ZNoLImo4foOOKBJMdqJIMNmJIsFkJ4oEk50oEnN6wsm0GjFRpeXEq6+ased3Pm/Gxpwlnt60yp5wEpVwGac8ao96K1XD5ToAGNhtl8NGzthDJorGxJcvvLDLbNM5zy7lzV9kjzZbuGSRGdu3e3dw+09/+hOzjffq8EavpZV1ic3CMztRJJjsRJFgshNFgslOFAkmO1EkmOxEkYiy9NZMQ8aEhwDwy1/ZExsePGSPUtu40V6n7MSJ8Kiy//rBw2ab5UvsiS97e+2y1p4Xd5oxaxLFvXvsSSUXOBNfrrholRm7WOxz1qGD4eN40imJ5vL2/Wm1ySMtM8QzO1EkmOxEkWCyE0WCyU4UCSY7USR4Nb6FJpx51Q4dOmTG9u7da8bGxsIDVwYGBsw2Xqy9aL9E1LkKXi6HR/J4A0kKB+wKRM98e2mCdWvWmLF3vP3twe3LLrSXrnr5aHgJLQDI5Zwr9c2eE3GaeGYnigSTnSgSTHaiSDDZiSLBZCeKBJOdKBL1LP+0CsC3ACwDUAWwWVW/KiK9AB4AsBq1JaA+rqqnGtfV2cubQ8wrx3gxr0Q1apTXAGB0xF5SypJ3pkArlezJ8LzZ2Kxj4pWuKhV7LryfPP6EGesstpmxv/m7vw5u/8xnP2O2ufuf7zZj54bsgU3eY2vE3HXTVc+ZvQzgc6p6CYArAXxaRNYDuAPA46q6DsDjye9ENEtNmeyqekxVn05+HgKwC8AKANcDuC+52X0AbmhUJ4lo5qb1N7uIrAZwGYCnACxV1WNA7Q0BwJKsO0dE2an767Ii0g3gQQC3q+rZeue6FpFNADal6x4RZaWuM7uIFFFL9G+r6veSzcdFZHkSXw4gOEWKqm5W1X5V7c+iw0SUzpTJLrVT+D0AdqnqlyeFHgFwS/LzLQDseY+IqOXq+Rh/FYCbATwnIs8k2+4EcDeArSJyK4CDAD7WmC42m/fnSbhU1ojRTmrsCwB6urvNWMkZSWezH7P311qaRYu8YyXOPeacdv/z4x+bsSXLwvPrffLP/tRsc/TIMTP27/d804yNjNhluTSvq6xNmeyq+jPYPf1gtt0hokbhN+iIIsFkJ4oEk50oEkx2okgw2Yki0YIJJ2f5+4v7zcBwiWReZzHV3Y2Nj097XwDQ2dlhxvr6rG8t2x2pes+JOGUht+SYbTlJxB41Nl4aM2NbtmwJbr94/VvNNhs3vse+v+9sNWPDw/aIw5wzOadXZp0+e+TgLM88IsoKk50oEkx2okgw2YkiwWQnigSTnSgSTS+9qdqlgVkhRRVEy/Z7ZsFZK61N7FipXLJjo/boqvFxa9SbN9rMLmulHdBntXMn53Tur6J2H73RcmOj4bLcz3/+S7PNu99lH/uhoTNmzJuC0z+MzRn1xjM7USSY7ESRYLITRYLJThQJJjtRJJp6Nb6rqwsbLr00GCuX7WWGJibCA0YqFfvqp3c1u5C3H3Z7u72U0OnT4SuxWvWWNHL6WLLni/OOx8g5e8DFvK7O4PaLVq4w2xTyeTNWcvo/PjH9JY3GjecSAKrOvjrb7OelVLGP1ZgxJ9+qlReZbY4cOWLGys7rKpdLN5dfmtn8rLn8vOoJz+xEkWCyE0WCyU4UCSY7USSY7ESRYLITRWLK0puIrALwLQDLUPum/2ZV/aqI3AXgkwBOJDe9U1V/6N3XgvkL8JE/vjEYq1TsATKlUrjc4Q6ccEpXeafUJDn7/W9sLFw2ausIl7uAqUqKdult3Jmfrm/ZSjPW090V3P7+qz9ktvGOYz5nHytvDj2r/9ZzCfjPS7VkH8fhEbsUOTIeHgjzpjetNtu8uPsFM7buLZeYsWLRnoswzeugXLZzwioBDp542WxTT529DOBzqvq0iPQA+K2IPJbEvqKq/1rHfRBRi9Wz1tsxAMeSn4dEZBcA+xsaRDQrTetvdhFZDeAyAE8lm24Tke0icq+ILMy4b0SUobqTXUS6ATwI4HZVPQvgawDWAtiA2pn/S0a7TSKyTUS2DQ97S9oSUSPVlewiUkQt0b+tqt8DAFU9rqoVVa0C+DqAK0JtVXWzqvaran9XV/jiERE13pTJLrV5hO4BsEtVvzxp+/JJN7sRwI7su0dEWannavxVAG4G8JyIPJNsuxPATSKyAbXKzQCAT011R5LPo6NnQTjmjPwRYzRRteoVjZy5zpyhQVq1R151GTFxS3nOskvOvrzSYVfPBWZswcLwpZO3bXi32cYrrzndR2nCXnapapVSveFfzvNScUpvlapdoho1Sm8XLrevMVfLdj8WL1xqxtyFw7zXnBEbHR0124wY5cZHvv+g2aaeq/E/Q/hxuDV1Ippd+A06okgw2YkiwWQnigSTnSgSTHaiSDR1wkmBIGeUqbzyiVkOU/u9yit1uOO8xHn/M2orE+aSS/5yR+4or4pThnJKjsX2DqMj9uMqOyVAdwZDdUp2xmPLOWU+73kpttkxb0RZ9wXzw9vndZttcs6xWrSo127nlTC9GqZhfNwuvVmvq472drsP0+4BEc1JTHaiSDDZiSLBZCeKBJOdKBJMdqJINLX0VqlWMTwUHoWUcyZ6tNZ6q3rlOtixdqc8UXJGV1mj1LzymldyqVbsUo03eWHfYrv8c+zo4eD2c0OnzTYdzoSZxYL9EskV7OfMOlaq9vPiliKd8qBXZp0/P1x6s0aNAf6IQ3/0mj0BpzjlPPv1Y+/LPL5OG57ZiSLBZCeKBJOdKBJMdqJIMNmJIsFkJ4pEU0tv1UoF586GS0De5HoTE9b6YM76cGW7DNJWbDNjFafEMzoaLtcUvPKUMxLKKzW1tdmlt94F9oSThw68FNx+5swZs023sT4c4Jcp260RdrBHKuby9vmlUvHKa2YIZadcapXeTp06Zd+hwy8PpptctFQKj5psb7dfp1Y/3DKwGSGiNxQmO1EkmOxEkWCyE0WCyU4UiSmvxotIB4AnAbQnt/+uqn5eRNYA2AKgF8DTAG5WVXsyNtQGhXR3ha8yl0v2Cq/WWJKODnseMZEeJ2ZfsfSurJdK4apAxVrqaIpYsWBfcffmVduz+wUzZvV/WV+f2cZbkank9N8b1GLN1+cNJMnnvaWhnPOSc6X+5MmTwe3eayDtvIFtbfbV86wHelmvK+/41nNmHwfwAVV9J2rLM18rIlcC+CKAr6jqOgCnANxax30RUYtMmexacy75tZj8UwAfAPDdZPt9AG5oSA+JKBP1rs+eT1ZwHQTwGIB9AE6r6mufNQ8DsJfFJKKWqyvZVbWiqhsArARwBYBLQjcLtRWRTSKyTUS2DQ/bf5cTUWNN62q8qp4G8L8ArgSwQEReuxq0EsBRo81mVe1X1f6uLvtrmUTUWFMmu4j0iciC5OdOAH8IYBeAJwD8SXKzWwA83KhOEtHM1TMQZjmA+0Qkj9qbw1ZV/b6I7ASwRUT+EcDvANwz1R3lRNBWDA+sWLjQnlfNKicUnNJV3imheaUar3RhDU4xl6ea4v7cJaqcKlTBmZ/OWuap6gwy8QaneINMqs4yVGb1Spx+OOUpb19ezHq9lcteP5zBJM7z4pXsvOe6aA7Msu/PKr15c91Nmeyquh3AZYHt+1H7+52I5gB+g44oEkx2okgw2YkiwWQnigSTnSgS4pZ/st6ZyAkAB5JfFwN4pWk7t7Efr8d+vN5c68dFqhoc4tjUZH/djkW2qWp/S3bOfrAfEfaDH+OJIsFkJ4pEK5N9cwv3PRn78Xrsx+u9YfrRsr/Ziai5+DGeKBItSXYRuVZEdovIXhG5oxV9SPoxICLPicgzIrKtifu9V0QGRWTHpG29IvKYiOxJ/l/Yon7cJSJHkmPyjIhc14R+rBKRJ0Rkl4g8LyKfTbY39Zg4/WjqMRGRDhH5tYg8m/TjH5Lta0TkqeR4PCAi9gyXIara1H8A8qhNa/VmAG0AngWwvtn9SPoyAGBxC/b7PgCXA9gxadu/ALgj+fkOAF9sUT/uAvBXTT4eywFcnvzcA+BFAOubfUycfjT1mKA2trU7+bkI4CnUJozZCuATyfZ/A/Dn07nfVpzZrwCwV1X3a23q6S0Arm9BP1pGVZ8EcP4cx9ejNnEn0KQJPI1+NJ2qHlPVp5Ofh1CbHGUFmnxMnH40ldZkPslrK5J9BYBDk35v5WSVCuBREfmtiGxqUR9es1RVjwG1Fx2AJS3sy20isj35mN/wPycmE5HVqM2f8BRaeEzO6wfQ5GPSiEleW5Hsoek3WlUSuEpVLwfwRwA+LSLva1E/ZpOvAViL2hoBxwB8qVk7FpFuAA8CuF1VzzZrv3X0o+nHRGcwyaulFcl+GMCqSb+bk1U2mqoeTf4fBPAQWjvzznERWQ4Ayf+DreiEqh5PXmhVAF9Hk46JiBRRS7Bvq+r3ks1NPyahfrTqmCT7nvYkr5ZWJPtvAKxLriy2AfgEgEea3QkR6ZJkjSgR6QJwDYAdfquGegS1iTuBFk7g+VpyJW5EE46J1CZvuwfALlX98qRQU4+J1Y9mH5OGTfLarCuM511tvA61K537APxti/rwZtQqAc8CeL6Z/QBwP2ofB0uofdK5FcAiAI8D2JP839uifvwHgOcAbEct2ZY3oR/vRe0j6XYAzyT/rmv2MXH60dRjAuAdqE3iuh21N5a/n/Sa/TWAvQD+E0D7dO6X36AjigS/QUcUCSY7USSY7ESRYLITRYLJThQJJjtRJJjsRJFgshNF4v8ADFB4JI4AG8EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAfM0lEQVR4nO2da2xd15Xf/+vey8u3KFKUKFqSJcuSH/JDss1o0iad8cTjGScx4qSdvD4ERmGMB4Mx2gDTD0EKNCnQD5likjQF2rRKY4ynSPNoHohn6k5ieGbgOjPjWHZsPWzrYUdviqTEN3kv72v1w70KZGf/N2lRvFSy/z+A4OVe3Ofss+9Z59y7/2etZe4OIcSvP5nVHoAQojnI2YVIBDm7EIkgZxciEeTsQiSCnF2IRMgtp7OZPQDgKwCyAP6Hu38h9v9t+bx3dXQEbVciAMb6WKxjxGjGr392JdfGSJe2tlZqa4/YUKtyG5FSLZunXUpVvr2F+Tm+q1otMozwONz45Mdk4NnZeWpbWChTm5H9tWT4GxN7l6uRY65Gxl+LnK25lpZg+5ZtW2mfDuJHZ06fwvjFi8GDvmJnN7MsgP8C4H4AZwC8YGZPuvurrE9XRwce/Ge/FbRVIw5YJRMVm/hsLkttkfcZ2YhT5HOdYUPkqpON+OxNu26ktttv3cE7zk9yWzHsuLnezbTL2ekpaju+/wVqKxciDlgJO2AtMvnlGp/IZ//fS9T2xvGz1Naeawu2D3SFnQUA2rxCbVPz/JjnqrzfTMS2buC6YPuf/ef/Tvvcfc9dwfYH77+X9lnOx/i9AI67+5vuXgLwLQAPLWN7QogVZDnOvgnA6cv+PtNoE0JcgyzH2UMfvH/pc5iZPWpm+81sf7FUWsbuhBDLYTnOfgbAlsv+3gzg3Nv/yd33ufuQuw+15fn3YSHEyrIcZ38BwE4zu8HM8gA+AeDJqzMsIcTV5opX4929YmaPAfgR6tLb4+5+eJE+KJfDq7QV4yuxTNKoRaSOamRlN5flh+3OZSivTQfbtwz00z5De26jtpt28hXy1jwf/+unjlHb9Gh4jDv+SXjFFwDuf//7qa23LbyaDQDDrz5Hbe2Z8OpzR+ca2qcI/slv7MxpapseH6e2qanwOCbmIkpORNks18IyGQCUIrJtLWI7dWY42P7FL/wn2uePHvtXwfbZGa4WLEtnd/enADy1nG0IIZqDnqATIhHk7EIkgpxdiESQswuRCHJ2IRJhWavx7xR3oEaCV2oR6e1KLLHrmDs/7GKxSG037RgMtt83tJ32yRcuUNuRp1+ktnKZj2Pddr6/NbcNBdu9o4fvK8ODhm65jUuHhbMHqG3mzOthw2RYZgKAWoYHp+zdwse/pe0Oanvx9ZFg+7FTE7RPNSKTwSIu41zOy0Rk4hZyHh8/dpL2OXrifLC9WOIRgLqzC5EIcnYhEkHOLkQiyNmFSAQ5uxCJ0NTVeFgkLZHx6INsleRV44ufqDrPczUbSTt0wyAP1Pi9oW3B9s4SD8SYGB6jtmwhHLQCAJXCLN/maC+17dpzf7C9dYCv4BfmeJ6B9s5uautYz9NqvXE4vBpfHufzMXxhhtpawANQuJYA7NocXuFf28WDbg4fG6W2qdmINhRZqa9ZJE8eCb6aJinGAOBkIWwrRQLAdGcXIhHk7EIkgpxdiESQswuRCHJ2IRJBzi5EIjRVerNMBvn2cE6zhUhJo5yHr0n5yKWqZw3PnbZpAw+q2L2N55NrL4aDJ3yBl0jqIKV9AGC+yuWYSiyHXkSGqpHrd0uel6aZK3LprRTRtdZv20ltvQPhEgIXJrhMWSVyEgBcmONz1dfbRW2l+YvhPm2kug+Ae27luQFfev2XEij/gvMzXC6NBXqxikfleT75z/wkLG1Oz/IAKt3ZhUgEObsQiSBnFyIR5OxCJIKcXYhEkLMLkQjLkt7M7ASAGQBVABV3DydAa5DNZrG2d23QNl/isouR3F43bllH+9yzg9vW+xS1zY2fpTYm/7S18mvm7BiPbJue4VFe3hmeJwAY2HYrtfUOhMs8eYZHAYLkBQSAWN3dTBePiOteH5YwZ09xSXT9Wh5xODoTltAAYHqBnzv5WvgUn73AI9vWrttIbbu2c2l2/hiX3kYjsmKNuKGXCrTP+YMvBdsrhRUq/9Tgt92dZ1UUQlwT6GO8EImwXGd3AD82sxfN7NGrMSAhxMqw3I/x73H3c2a2AcDTZva6uz97+T80LgKPAkBXJ39EUQixsizrzu7u5xq/RwH8AMDewP/sc/chdx9qj9T6FkKsLFfs7GbWaWbdl14D+F0Ah67WwIQQV5flfIwfAPADM7u0nf/l7n8d62BmaGkJ77I1UrZmoC8csTV0A0+82DnHBYLzI1xeKy3wcRQq4Sikk5Nc7ihN8Yi4nvXt1JbfMEBtG2+5ndra1oQlx2LkuGJli2YLkYi4Mu+Xbwsfm2e5BNi5lpd/6p3n4xgbm6S29d1heTDXwj9lzs1xaXZtO5d0NxJZGQBG5/n56BYeS875cVVHwqXDvLoC0pu7vwlg95X2F0I0F0lvQiSCnF2IRJCzC5EIcnYhEkHOLkQiNDXhZCZj6OgM73JtK6+9tffmcBRSy8wI7XP+7HlquzjF5Ym5eR5pxCL23jjJkyi2OJe8OgZ44suNW26itt7rtlJbhSTurJT4MRfnefTd7PwCtdXKfK5mJsLy1ZnT/D2bmObyWi1SR61aiSTu9HAyR57+EShVItYCn6v2PB9jLsOThJZBkoE6j6LLePh9qYHLobqzC5EIcnYhEkHOLkQiyNmFSAQ5uxCJ0NTV+GzW0NMT3uW2Pp5/LF8Jr4C+eewN2mdmhq/QZrORUFvj178qyeM2XuIr1utJEA8ArBncQG3bb+LBLsU5vgp+8dTpYHulGFmNL/PV56LzEkS1Gj/uyfHwe3ZumOfkG7nAx3jdZp4XbsF5cA0rK9aS5e/z/HyF2moVPsZKNZLnD3yu6ikcfxl3rk7kSPASKyUF6M4uRDLI2YVIBDm7EIkgZxciEeTsQiSCnF2IRGiq9JbPZbC5ryto29DFA2HOHj4ebB8e5iWBWF4vAOjs5HJYJVIKqYUE68TkjvZIiaSurvBcAMDxwz+jtpnKMWp7/cibwfbuLp7fbedtd1BbS3cftbXneWrw/o03ku3xvIF9EQmtr4/n6zvL45AwPhuWvDZ18/MjJlNWs1yKnJ4v8m3W+DaNHHYmEtRCO5HAn/r2hBBJIGcXIhHk7EIkgpxdiESQswuRCHJ2IRJhUenNzB4H8CCAUXe/vdHWB+DbALYBOAHgY+4+sdi28rkMtvaH5Saf4jJacTosWywscKkm08oPrVDiUU2ZyIzk8mHZJVcJRy0BwOxFXv7pzCleEmi6wqfz4NFhahu9GC4ZNLT3HtqnI1Jdt6ufS2UteS6Xdg79RrB9eHSU9jn49z+O7CsSUdbC87sx6W2gg0t5VuWS15peXv6pPM/zHlYi8mwLwudVLIauRjcX8YnI9i7x5wAeeFvbZwA84+47ATzT+FsIcQ2zqLM36q2//bGFhwA80Xj9BIAPX+VxCSGuMlf6nX3A3YcBoPGbZ2EQQlwTrPgCnZk9amb7zWz/5Az//iqEWFmu1NlHzGwQABq/6aqLu+9z9yF3H1rbzReChBAry5U6+5MAHm68fhjAD6/OcIQQK8VSpLdvArgXQL+ZnQHwOQBfAPAdM3sEwCkAH13KzsxryLKEfZVIQj6iM2SzPHqtVuPRScUFLr215/j1r1wh/bjyg+kM317/7VwOe+/d/5TabiBRgABwcTgsy/VGJLTufi4n9azntlwLl3l8XViWu/+jH6d92tq5PDV++ii1DZb4efDqgXC/Cteu0NHOZTmPRL3NFniSUwOPsssQucxIIsq6LdynHOmzqLO7+yeJ6b7F+gohrh30BJ0QiSBnFyIR5OxCJIKcXYhEkLMLkQhNTThZq1Qwe2EsaOuM1BTL5cKyhWV4Ej+vcgkiksYPZa6eoERqovWu47JW1/U3U9vgrr3UZl38CeStu7iMc/3N4f11dvEHmvrW9VNbniTZBBALvcL8QrhOWa2XJ+C8877fo7YTr11Hbafm/pHaMplwcs6YrIVIxOSRs+eobWaenzwZliASQNnC5zGLhgOALMJyo/G3RHd2IVJBzi5EIsjZhUgEObsQiSBnFyIR5OxCJEJTpTcHl8Sm57iMlmsNS02ZHB++VWIyCL/G1Spcu6iUwmPffsM22qdj81ZqKxZ5bTBEEhSuWbOG2nIkaq89EskVqzmXiUTt1SI6ZWUhHMVYiPTp6OQS5obNO6ithJe4jShslYg0OzE7Q23nx8MJPQHAnIc/ZiOaWJUmieTSm1HXXV7CSSHErwFydiESQc4uRCLI2YVIBDm7EInQ1NX4TC6Ptv7rg7aR8SPRfiHaI0EaTgIxAKBS46EwVuWrmQvz4W2u6eF52izD9zUzy1d2NzhfLe5s7+D7IwEX1cjqcwyPBFZEphEt1fBcZaZ5mS+f4erE2lauJlw3uJnaXiJBVAXjK+cXijxHYaHGz49YuaaWyDyCjiWiNl3BfVp3diESQc4uRCLI2YVIBDm7EIkgZxciEeTsQiTCUso/PQ7gQQCj7n57o+3zAP4AwKWEcp9196cW21ampQ0dm3cFbdlzXJKpVcKBCZ0dvOxPuVCgNie55ACgFlGo5mbCwTo913HRpb2TS0YLJS41FQqz1NYaCQByopV1dPAcdJksv+ab8WCMCiIS1UQ41+DYkZdpn9GRcWqbWOA63+ZIaavb7twdbD92+EU+jiJ/P2vG5d5sRC7NxoKvSP0wj9yL2ftivrxAmD8H8ECg/cvuvqfxs6ijCyFWl0Wd3d2fBcAvuUKIXwmW8539MTM7YGaPmxn/HCWEuCa4Umf/KoAbAewBMAzgi+wfzexRM9tvZvsnpqavcHdCiOVyRc7u7iPuXnX3GoCvAaDVDtx9n7sPuftQbw/PsCKEWFmuyNnNbPCyPz8C4NDVGY4QYqVYivT2TQD3Aug3szMAPgfgXjPbg3qitBMA/nBpe2uBrd0YNA3ccjvtNnb0eLC9q8ZloVokBml6nMty1Ujut0JpLtiebeWy1tYdO6ltNs+j17I5fmyx+Kr2zvBY2jv5vhCR16oRLXJ+apTajh7aH2yfPvkm7TN8isuvx0+dp7ae7TdS2+533xFsPzlyivZZGOVfN2PBa9WIvGaR94ynp4tJb7EYuzCLOru7fzLQ/PV3vCchxKqiJ+iESAQ5uxCJIGcXIhHk7EIkgpxdiERobvknB4pEyTkD/sBN5y3vCrb3nufyCUo8uqo8HS5NBADIcHHFauHopPUDYTkRAAYHB6ltvMyjxjryXA5rj9jy+fAYLZI5slgIS4oAUIok7ixESnbNzoTn+JUDx2ifo6//nO+L1XECcNNAP7Vtv+PWcPvNN9E+I2cvUNvY2Ai1VRFJZBqT0YgsVwUvlZWths+BmDSoO7sQiSBnFyIR5OxCJIKcXYhEkLMLkQhydiESoanSm2UyyLeHo7LeKPLIsZnxcE203954He2TaQ9HygFAW/sUtblzOczaeoLtm27ikW2tkUSPlXNcxhm7eIbaNkRqm41PTgTb29t54suODh4RVyzypJgLJS4N5fLdwfaJOT6/UxGZr6MnvD0AaOvqorZaJSzZbbvhBtrn5DY+9xcneYa2O27nkZt33r2H2rItYbn00KHXaJ8jL78abM/M83Nbd3YhEkHOLkQiyNmFSAQ5uxCJIGcXIhGauhqfyWTQTlanCwivdANAPht+vP/66zfQPoWxc9Q2P88DOEpFXnZp8Jabg+09W3fw7WXaqG1tHw+gWSjylWmL5DorzoWVi4kLPF9cb28ftXkkuGN8jG/z9LmzwfZcO5+PzTuvp7Zsnpddmi/ynIIZEmRy9z330D6FAlcM1vfxufrUv3yY2nbcFi57BgAtHeE5uTjJc+E9/l+/Fmz/Pz/8Bu2jO7sQiSBnFyIR5OxCJIKcXYhEkLMLkQhydiESYSnln7YA+AsAGwHUAOxz96+YWR+AbwPYhnoJqI+5ezgKo0GlBkwWwjLa+CQPuPj47nB5n+0380CYgW08WOTNm8N5yQBgeoofwrabw9JbW/c62qdc4VnBunp4AEpnpAZmNZJProfIcoVzPLjj9VcPU1ulxCXAqQmeq230YjjIp7WPB620OJfljASLAED/4FZq61sflmc7Oltpn9+5773UNvMbQ3xfA+uprVzjQUNeDsuDuXaea/BDH//nwfbnnn2K9lnKnb0C4E/c/VYA7wbwx2a2C8BnADzj7jsBPNP4WwhxjbKos7v7sLu/1Hg9A+A1AJsAPATgica/PQHgwys1SCHE8nlH39nNbBuAuwA8D2DA3YeB+gUBAH+cTQix6izZ2c2sC8D3AHza3flzfL/c71Ez229m+ycnol/phRAryJKc3cxaUHf0b7j79xvNI2Y22LAPAgg+KO3u+9x9yN2H1vb2Xo0xCyGugEWd3epV378O4DV3/9JlpicBXHry/2EAP7z6wxNCXC2WEvX2HgCfAnDQzC7VVPosgC8A+I6ZPQLgFICPLrahUqWGU2PhqLL+dh7VtGt7eDkgG8md1tPDJZ7d976P2splLpGUFsK2cplHhmUy3FarcdvCAi9RVYlIb1US5dW7jpdIyoW7AABGzvLowVonj1S0TaSk0QKPUJub4d8O8508l9+m7Tyf3LqB8LmTy/L7XCkSRVcsc5fJt3M5j5XlAgDLhOcqU+XnRzuJlMtk+HEt6uzu/hxAziDgvsX6CyGuDfQEnRCJIGcXIhHk7EIkgpxdiESQswuRCE1NOLlQquD46YtB244eLr11doVlC8/yqCBE5KmI4gWnwgNgmfD+LMenMRMZRyUiryHDt1kp8X7lcjhKrRTZVylSdilDjhkA8nkuNXWQMl+VyPY8IjV1ruFhgJs2baK2tSTqrVYNl4UCgMlxbkN1jppac/zemc/y86ri4f1lI+dOjpynERVVd3YhUkHOLkQiyNmFSAQ5uxCJIGcXIhHk7EIkQlOlt2rNMFEMSy8fuoVHNbVYuPZWzSIyWeQ6Vq5xaaUSSRBZqoSloVhSyVKZ1w2LyWGViDRUIvIaAMzOhaWhC+eHaZ/xkfPUVitFogAL3DY1GU5UUirwOnuFuRlq272R18UbjNjyXeHIvLk5XtPPIudVaz4is0bq4mUQiVQsh8+DaomfO625sFQdG7vu7EIkgpxdiESQswuRCHJ2IRJBzi5EIjR1NR4G5EiOrC09PECiRhYyI3ECqFb4SmaFrKoDwALvhgWSa64SWbFeiAStVGL57iL9xi+MUdvI2XCZp2myOg7w1WAAmJ/hq9anfn6a2s6cPhtsz0ZWi++4k5flunHnTmrLkJJXAA/ymY0c19TEJLUV5nh+utidM2bLkeAgd666XAm6swuRCHJ2IRJBzi5EIsjZhUgEObsQiSBnFyIRFpXezGwLgL8AsBFADcA+d/+KmX0ewB8AuKQDfdbdn4ptK58BtnSHry8dzmWoqoflmlw0v1uR2oqkjBMAlKMBNOH9VSOBNbUq1/IKc7zc0cULwTqZddt5HrhSmgsHmtSKfD7Gxvi+jh49Qm2nT3LpbfOmrcH2j/3+v6B9tm/nAS0zkdJQw6f5OKwtXAZsenqK72uKS2/I8bx7uRaeRzGT49JyhuSgiyiKAC0rFstbtzgVAH/i7i+ZWTeAF83s6Ybty+7+Z0vYhhBilVlKrbdhAMON1zNm9hoAns5TCHFN8o6+s5vZNgB3AXi+0fSYmR0ws8fNTMXXhbiGWbKzm1kXgO8B+LS7TwP4KoAbAexB/c7/RdLvUTPbb2b7Z2f49yQhxMqyJGc3sxbUHf0b7v59AHD3EXevunsNwNcA7A31dfd97j7k7kNd3byetxBiZVnU2a2e5+brAF5z9y9d1j542b99BMChqz88IcTVYimr8e8B8CkAB83s5UbbZwF80sz2oL7WfwLAHy62oXzGcX17WPYqlbg00bIQjjTyfAftU4mUEqpWeDRRLJKO5RGbK/DcaVMT4XJXADA+OkJt0xPj1FYpRGS08+FtHj/+Bu1z5kw4Qg0AZiLRYd3da6ntQx/8QLB9z64dtM+x1/dT23xELi1leP7CyQJ5ryNvdDGSJ6+fSIoAL+MEAJWIPJtvbw+3l/h5WiASa+y4lrIa/xzCJaSimroQ4tpCT9AJkQhydiESQc4uRCLI2YVIBDm7EInQ1ISTeVSxycJP0U1P8+iwzs5wFFJnRzffWSSxYTmSzHEuUoJociqctHFinMtkc5M8gmr2IpflxiKRXEeOvUltJ8+FpbeFAp/faiWSsJEk2QSAWpVLgAcOHA22j5LxAUBrjstrXd1cZr0wzd+zsamwRFWY5/LaXCQa8YM38qSYWa6uwSOlnJhMXCClvACgOB+e+1otVoJKCJEEcnYhEkHOLkQiyNmFSAQ5uxCJIGcXIhGaKr1lzdGTC0fyzMxy3WL25Ilg+4YOHu20kA1HEgHA5HhE8jrPI8CmLl4Ib2+YJ4A8f+oUtY1E5LW5SL2x81O83tjUbFjimZnhMtkUkacAoLjAZcqqcznp4MHjwXYjtf4AYE0Pf8+2buqntnXruG14LCyXTkbmN5vhkWN9vXxflb18jnfccjO1IRd2w9HzvKbfT557Ptg+E5EhdWcXIhHk7EIkgpxdiESQswuRCHJ2IRJBzi5EIjRVequ6Y7Icjmzq8EgU0kQ4CqnMc1Siaz2vG2aTYQkNACpnuFR2/tDhYPvYiRO0T3ueD3LXxnXU1noTT8z494f5/g7/5GfB9slpPr+VWiQiDjwSzSOnT6ZG7iM8KAujFyNy40UeLWfBFImXdheW0bLg70s2cgv8u7/5G2obete7qK21ncvEbxB59vl//Cnt85fffTLYHpMUdWcXIhHk7EIkgpxdiESQswuRCHJ2IRLBPFbvCICZtQF4FkAr6qv333X3z5nZDQC+BaAPwEsAPuXuvF4NgMH+df7IBx8I2j7+4H2RMYRXTmfneVXYNV1ruK2T5zObmYwEyYyMBtunxnmJpLa2NmrrWMtX4//h4BFq2/fNH1Db8EQ4b5kbXzmvRc6BGmJvKe9nZKXeIyvnFlmqt0wLtWXAbXyDkRX8KlcnOtr4PO6+aze3DXFbtjW8zb/+0f+lfX7+6olge6E6i6pXgge3lDv7AoD3uftu1MszP2Bm7wbwpwC+7O47AUwAeGQJ2xJCrBKLOrvXuXTramn8OID3Afhuo/0JAB9ekREKIa4KS63Pnm1UcB0F8DSANwBMuv8ioPkMgE0rM0QhxNVgSc7u7lV33wNgM4C9AELJs4Nf4MzsUTPbb2b754s8uF8IsbK8o9V4d58E8HcA3g1grdkvVn02AzhH+uxz9yF3H+qILFYJIVaWRZ3dzNab2drG63YAvwPgNQB/C+D3G//2MIAfrtQghRDLZymBMIMAnrC6/pUB8B13/yszexXAt8zsPwD4GYCvL7ahSs0xVghLObXuPtqvNRuWSazCJa+Z+XDuMQDItvB8d/ke/uljsCu8LNE/wLfnzqf43DgPTvnRs/upbWSSlwVyIinFJK+axaQ3jnnkuMHkq9j9pZVaMh6R1yKyIoisGJOcM5k8tZW5Koef/MML1PYKCaICgL7+3mD7hQs8B11nW7j02UKBn1OLOru7HwBwV6D9TdS/vwshfgXQE3RCJIKcXYhEkLMLkQhydiESQc4uRCIsGvV2VXdmNgbgZOPPfgA8GVzz0DjeisbxVn7VxrHV3deHDE119rfs2Gy/uw+tys41Do0jwXHoY7wQiSBnFyIRVtPZ963ivi9H43grGsdb+bUZx6p9ZxdCNBd9jBciEVbF2c3sATM7YmbHzewzqzGGxjhOmNlBM3vZzHiY2dXf7+NmNmpmhy5r6zOzp83sWON3OBRq5cfxeTM725iTl83sA00YxxYz+1sze83MDpvZv260N3VOIuNo6pyYWZuZ/dTMXmmM49832m8ws+cb8/FtM+PheSHcvak/ALKop7XaDiAP4BUAu5o9jsZYTgDoX4X9/iaAuwEcuqztPwL4TOP1ZwD86SqN4/MA/k2T52MQwN2N190AjgLY1ew5iYyjqXMCwAB0NV63AHge9YQx3wHwiUb7fwPwR+9ku6txZ98L4Li7v+n11NPfAvDQKoxj1XD3ZwGMv635IdQTdwJNSuBJxtF03H3Y3V9qvJ5BPTnKJjR5TiLjaCpe56oneV0NZ98E4PRlf69mskoH8GMze9HMHl2lMVxiwN2HgfpJB2DDKo7lMTM70PiYv+JfJy7HzLahnj/heazinLxtHECT52QlkryuhrOHUqasliTwHne/G8D7Afyxmf3mKo3jWuKrAG5EvUbAMIAvNmvHZtYF4HsAPu3u4TrdqzOOps+JLyPJK2M1nP0MgC2X/U2TVa407n6u8XsUwA+wupl3RsxsEAAav8PlZ1YYdx9pnGg1AF9Dk+bEzFpQd7BvuPv3G81Nn5PQOFZrThr7fsdJXhmr4ewvANjZWFnMA/gEgHBl+RXEzDrNrPvSawC/C+BQvNeK8iTqiTuBVUzgecm5GnwETZgTMzPUcxi+5u5fuszU1Dlh42j2nKxYktdmrTC+bbXxA6ivdL4B4N+u0hi2o64EvALgcDPHAeCbqH8cLKP+SecRAOsAPAPgWON33yqN438COAjgAOrONtiEcbwX9Y+kBwC83Pj5QLPnJDKOps4JgDtRT+J6APULy7+77Jz9KYDjAP43gNZ3sl09QSdEIugJOiESQc4uRCLI2YVIBDm7EIkgZxciEeTsQiSCnF2IRJCzC5EI/x9L2LoCXmBhVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAd+0lEQVR4nO2da4ik55Xf/6feundXT99nWjMjjS7ji2KysjMIg8Pi9SaLYhZkQ3axCUYfzM4S1hDD5oNwIHZCAt4Q2/hDcBjHymqD40vWNhbBJGvELo4XInmsyLqs1rZkj+be0zN977pXnXyoEoy0z//pnunu6rGf/w+arn5PPe976nnfU2/1869zjrk7hBC//uQO2gEhxGhQsAuRCAp2IRJBwS5EIijYhUgEBbsQiZDfzWAzewTAFwFkAP6Lu3829vxqpeQTE1WyLz7O+/3g9l4vvB0Asjx/afkso7accSmy024Htzdb3Ygf/P00F3mvjUmifTIfANAhruQyPsGFPLflbvN20CV+tNp8rkpFfs5ifnTZwQDkcuF95iPHKpVL3FbitnzkmoudM+/3gtt7kTHs2r9xYw0bG/XgCb3tYDezDMB/AvCPAVwE8CMze8rd/4aNmZio4rF/9ltBWz4SFO2NzeD2jc06HTM2PUNtc9OT1Dae71DbxfMXg9t/+toiHTM9VaO2cr5MbZ0296PeaFHb4o3whV8d4xfp/Bx/86tV+Xnp9fmbxPJa+AJ+7dwKHXPi+FTED2rCtSW+z/Hx8D7njh/mfjx4L7U9cN8D1DY1ya+rdrNBbY3GVnD7ZuT6Xl8Lx8S//ff/lY7Zzcf4hwG86u6/cPc2gK8DeHQX+xNC7CO7CfajAC7c9PfF4TYhxB3IboI99Bnu7/yjaWanzeysmZ1tRD5+CiH2l90E+0UAx2/6+xiAy299krufcfdT7n6qUuH/Nwoh9pfdBPuPAJw0s3vNrAjgIwCe2hu3hBB7zW2vxrt718w+AeB/YyC9PeHuL0cPls8wOXsovL/Iym6/GZYZtuprdEwb4dVKAMiMr4JvIbyKDACvvx5e9bWIbhj7NGPOx7U2ueyyvsV93GqGV+N7kWNVS/wyqBaK1FasFKitNhF+3bUa/1fu/GW+qv7AA3dTW7vLV63XVsOr4IUyv3aWry5RW/7k26htZn6e2mIycacVXo2v15t0zNZm+HVFpUFq2QHu/j0A39vNPoQQo0HfoBMiERTsQiSCgl2IRFCwC5EICnYhEmFXq/G3fLBiAfN33RW0NbZ44kebyFCdLpdIWhF5amKSS0bL17mMs7QcznqbPcKzNHIF/n7aafDXvLLKpcPlTf7aGp2wxNZq80SMYu72EmFi0lupFPajWuWy59Ub/DVfvRGeewDIsglqW10Jy3ntPpf5yhPcx0hyJsrVCrW1O1x6Y9Pfj4Sn5cKSaC5yLnVnFyIRFOxCJIKCXYhEULALkQgKdiESYaSr8ZXyGB78e6eCto11vhJ7/fIPg9vrdV6nrTzOEwLaXb6mevU696PRCb831ibCyT0AYBmvj9bq8hXmdiRxot6N1CYDWY2NdPnqcDewRsofAUBpjI+zYtjHXMZ9L0Z2eO36BrXNTvFyUK1++Dpob3IlZGMzUtMuUr/QEKlFGCmi5yTZKCvyk5Zj05iL1BOkFiHErxUKdiESQcEuRCIo2IVIBAW7EImgYBciEUYqvZUrVbzjnf8gaLu2yLuq/B//6+D2RoMnhJQiHVBaHT7u2soytRUy1kqIJ4RkBS41dXtcqjHStggAOn0+rkvkn3yeS0awSNeXiMyXZXxcn9Tyc0T2F7n1bJKaawDQrHHpzSrhjjydNq+F1+ryunura+vUdmPxErWVKzxZxyrjYUPknPVj/dIIurMLkQgKdiESQcEuRCIo2IVIBAW7EImgYBciEXYlvZnZOQAbAHoAuu4eTmkbkuUKmBo/ErS1SIsnAFjfCsskW5Fm9ffcxzPR1ja4fLJZ5/ucqJIMqg6XwubniKwCoFnlmVetCb5PX+MyVKcbzpQqOM+gKpW5dDg3z2Wt8RrPUmsTCbB2KOLHCm/J1Hc+V1tNPh9ZPlwfsN6KtIxa57bl69ep7cgcr12XGQ+1XD48rt+LpCoak+W4JLcXOvtvuTufASHEHYE+xguRCLsNdgfwF2b2YzM7vRcOCSH2h91+jH+fu182s3kA3zezv3X3H9z8hOGbwGkAuPtu3nZXCLG/7OrO7u6Xh7+vAfgOgIcDzznj7qfc/dTc3OxuDieE2AW3HexmNmZmtTceA/gdAC/tlWNCiL1lNx/jDwP4jg2yb/IA/ru7/6/4EANIQcSxMpd4YGH5qlLl2Um1cW57/SovotiLyB2WhW1d5xlUxRyX3ip5Pv21CZ6ZVy7x9+itNin0SNoxAcD8ES4ZjU3Esu8i0lApfJ7vOjFDh6xt8P0trqxSWzeSSVcpEVmxz8d4N5KN2OUSYDFSVLJc4nNcrzeD2/tUXgMy0ubJ9kN6c/dfAPiN2x0vhBgtkt6ESAQFuxCJoGAXIhEU7EIkgoJdiEQYacHJAWHJoFIKFwYEgLsW7gtuvzj7Mzqm2eWZS6vrN6jNjcsuE5NTwe1TMzz7q9vnjdTWt3iWFyI9xYpEdgGAPJGhinn+vj5W5VlvnX5YFgKAXo/7cWRuLrg9V+RS5MxseH4BoPDLK9TmfS5TFrLwaysWIrJnjRcrhfPrY3OTn8+Z+aPU1tkKz3EvkilXrJIClpE6lLqzC5EICnYhEkHBLkQiKNiFSAQFuxCJMNLV+G63i+Ub14K2apWv0r7tgRPB7c+drdAxS8s8cWJriyfCWKStzvh42MejR4/RMd0GP9bV6z+ntslJXkPP+OIzWPpP5nxQLJ9lfYuPmxnnyUt33/OO4PbVDT4fhTy3ZZGkkHaD16DrVsJqSL7Az/ORu6a5H3k+WevrK9TmkRp6fdLOqxVJyKGr8RF0ZxciERTsQiSCgl2IRFCwC5EICnYhEkHBLkQijFR6a7W38NovfxS0jVV4IszcfFh2KZZ5nbmf/fQStUVyXZBz/v7XJi2qlm/wZJFrizw5otnh8lqvz6WVYp7XTyuwFxeprdfu8kSYjQaXvG68usHHtV4Obi9W+CV3/uJVassZf805Il0BwAaRw6pj/Nq5/4F7qK1c5tdHq8VrEW5u8ZZj3W54jjc3eRJVpRau1Bzp8qU7uxCpoGAXIhEU7EIkgoJdiERQsAuRCAp2IRJhW+nNzJ4A8LsArrn7u4bbpgF8A8AJAOcA/L6785SfIf1eG+vrF4K2zc1Iva1iOLttcopnveWMZxkdmalSW73BZbTFy2HfG41FOqbR4tIVujzT78YK98Mib9Hj42EJs93h+7twZYvaWn1ej63ZjNRja4azG8tVLqF1ulw3OnaMNwW9dJHLrFuNsOR1eIHLa4fnw/XzACCf45l5zQafx9VVLsF6Fr4eO51IGypSbC5Sgm5Hd/Y/BfDIW7Y9DuBpdz8J4Onh30KIO5htg33Yb335LZsfBfDk8PGTAD60x34JIfaY2/2f/bC7XwGA4e/5vXNJCLEf7PsCnZmdNrOzZnZ2dZX/vyOE2F9uN9gXzWwBAIa/w6sxANz9jLufcvdTk5N8QUoIsb/cbrA/BeCx4ePHAHx3b9wRQuwXO5Hevgbg/QBmzewigE8D+CyAb5rZxwGcB/B7Oz4iyV6KtfBhnZDuf4C31Ok1uRKY73KJZG2dZ3KdvxL+AHP0GPfj2gqXTxYvcTks1oKokOe2ahb+9LSyyl/zRpNLXlbg94Me+LiTb39bcPvCUd4q68IvuYQ2Xoxkm9X5uW4uhjPRSpHsuyzWeisrU9tmJJtyI1JoM18OC2at1q0XCY0kvW0f7O7+UWL67e3GCiHuHPQNOiESQcEuRCIo2IVIBAW7EImgYBciEUZacBJmyOXD0kUh4xlsExPhb+O+/wP8W7qVPNdBls6/SG2HarzQY8fD+7zv7XfRMbVl7oc7L0I4wetvItICDNYPz2Mtx+U6y3GpKSvxPKosx+8VJ99+PLi9WuHi0NLVG9Q2zi8P3H2U92art8PZZpVqJBsRPDMvX+LFOT1y71xb5pJurhg+3kab769LVDkVnBRCKNiFSAUFuxCJoGAXIhEU7EIkgoJdiEQYqfRWLJZx7PjJoK1S4pLXoUPhAoCVCtdjzv30b6mtsfIatRWyBrflw7JLL9JrbHaa93O7Os7HVSYipQP7/LR1iFxTqPAim6USt/V6vHBnqcR1nkMT4X02IhlqvR6fj1zGj7VwZIrarq6EZa0scuW3WvwaKEekN4tIkRuRrMM1MifVmWPcj0o4uzEXkVF1ZxciERTsQiSCgl2IRFCwC5EICnYhEmGkq/GlYgX33fOuoM2dr3JmVgxuN+Mr1idOvJPaLp07S23dbrhm2YBw9sHGapuO2Krz5A7v8RXm2hhfYe60IyvkhfBqbD7Ha6e588tgfZ2vkF88f4XaXpsOZ/JMT4fP5cARvgrOaq4BA5WHUSqHVYFcpFHS2jpPUCqXeIVki/TlakdaOS1df2sPlgHHpmKr8eFafrmIIqA7uxCJoGAXIhEU7EIkgoJdiERQsAuRCAp2IRJhJ+2fngDwuwCuufu7hts+A+APACwNn/Ypd//e9ofLkNlk0NJ3XveLvSfFWt3ce/+D1PbySwvUVt+8Tm2Th8Kyy+Yal94uXlqltpnZcJ02AMhFkl2qZS5T9nusOBmXmupbXMqj+wPQ2OIy5UsvhBOR7j0xw/dXD9eLA4BmiUt2TuoaAkCXvLSY7Nlu8fnotPl8AJEklEiCSpdc+uO1cKwAQLkcvhZjyTg7ubP/KYBHAtu/4O4PDX92EOhCiINk22B39x8ACKv+QohfGXbzP/snzOwFM3vCzPjXvYQQdwS3G+xfAnA/gIcAXAHwOfZEMzttZmfN7OzS0hJ7mhBin7mtYHf3RXfvuXsfwJcBPBx57hl3P+Xup+bmwhVnhBD7z20Fu5ndvJz9YQAv7Y07Qoj9YifS29cAvB/ArJldBPBpAO83s4cwUL/OAfjDnR8yLAHFMoaMjPFIr5uxcb6MMDXL2zWde/Un1FYqhiWvq1d4fbEeV+VAdgcAaNQjtfCKXMbp98NyWKPOs66a9cjcRzILi6QmHwAsXg5n+/XadTpmfJzLr36I18lbXmlS28ZG+HhTs7zmYUwF3tzk/udzfGAvIvWZhefx6LF76JjK2K3XoNs22N39o4HNX9lunBDizkLfoBMiERTsQiSCgl2IRFCwC5EICnYhEmGkBSfjRHLYjNgiGkks+yeLtDvaqnOtzPthGSqWJTVWDRcGBIBSmb/m1RVe9LCQcXmlUAxnZdU3uTy1fJ1nr01O8wKLxQK/fHqkDdXWGs8amxznc9Xr8qy3y5d5ttz6WlgWzUduc12WhgagXufSW7kYa8vFX3d1PNwibP4Il4hhtx66urMLkQgKdiESQcEuRCIo2IVIBAW7EImgYBciEe4g6S1WcDIsUXlUeuPZWrPzR7kXziWedicsu5QjxRALRS7zLRzhmXmb64vUVh3jEtVYLXxKmy3ec86MS2+HD3Mf11f4uMWL4UKb/S6/v/S7PIvuyiUur129yot6Zll4Pno9ngUY68tWyPPrqtPhEmwrYps/fHdwey1ScLLVCvsYSQTVnV2IVFCwC5EICnYhEkHBLkQiKNiFSIQ7aDWer3I6aV3EatNtZztxN28Nddfd76C25579YXB7znj7oULGEyAKxld9a3zBHfPHj1FbrhJOqmjaNTpmZZPXC+32eAJNocjnuDQWTtYhuUQAgC1+KGxuROr89flK98Lx+bAhX6JjGpGklcmJyIkBT6JqOd/nwlR41b3X5ZPVqoeVkH6fK1S6swuRCAp2IRJBwS5EIijYhUgEBbsQiaBgFyIRdtL+6TiAPwNwBINslTPu/kUzmwbwDQAnMGgB9fvuvnL7ruzt+04sIWCiNkNtd594G7U9+3//Ori93eUSWpnVzwOwubVJbTFZ6/yFC9TWy8K169bXeCJJIVJLrl7nelg+x2Wo2qFacPvGBn/N9Qav77YVmavJad7KqUbaRnV7XAprNCN1CMFr8uVJ0g0AFCLJUpYPy5SrkXNWLIavuX5vd9JbF8Afu/s7AbwXwB+Z2YMAHgfwtLufBPD08G8hxB3KtsHu7lfc/bnh4w0ArwA4CuBRAE8On/YkgA/tl5NCiN1zS5+dzewEgHcDeAbAYXe/AgzeEACQryoJIe4EdhzsZjYO4FsAPunuvKj53x132szOmtnZpaWl2/FRCLEH7CjYbdBA+lsAvuru3x5uXjSzhaF9AUDwy9fufsbdT7n7qbm5ub3wWQhxG2wb7GZmGPRjf8XdP3+T6SkAjw0fPwbgu3vvnhBir9hJ1tv7AHwMwItm9vxw26cAfBbAN83s4wDOA/i9/XGRE0mgQj8ivRUynvG0cITXp6uMheWktUhGVi4i42w2eQ23sUj9sWf/6kVqqzfD799ZpGXUux68l9q6He5/o81lni6zRU7M1hb/77BU5md7bJyfzywLH6/T45lyvUjmWIxcpOVYqcwzI/tkTlaWed3A6nhYOuz1uaS4bbC7+w/B4+q3txsvhLgz0DfohEgEBbsQiaBgFyIRFOxCJIKCXYhEuIMKTnKcpbCRQpQAMPh6wK0zFmmtNHEoLIddusIzslbWNqgtK0Xkn6lYEctIi6qtcCukKVLUEAC6kQKLaxsNaut3uZy3uRF+3TnjxypxBQ3TM9z/6Vlu60QKPVI/inx+WTspACjw7lWoZNyYJ+ez0+XSbLsdzkZ0FZwUQijYhUgEBbsQiaBgFyIRFOxCJIKCXYhEuGOkNyqvxUdFLFyCMET6rxW4ZFcktnKRS1A31ngRxeVrXHpbvc79L5XDRRQB4FAtvM9Snp/q189fpLZOJAGs1+bz326EMwHLBT73Y1UuT5WK3NaNFPzcaoQlqsp4OIMRADJSABIA8hFbtcq1w0qBy3nlsXARy0KBH4vZYpKz7uxCJIKCXYhEULALkQgKdiESQcEuRCL8SqzGsxVGy/Gl4r5HkkzAEwz6zmuuZbnwqm8ucqxcpOba5jJvrdRs8RXmFl+kRZG0m2qu8YQW7gVgOb4K3o8k0BQQ9r+Q5/ORz/MV61YrolxsXKe2cjW86l6t8DZO5RJPQjKL1JmLtHgqjvNknUI1nHxVqXI/xsbDikwu4/7pzi5EIijYhUgEBbsQiaBgFyIRFOxCJIKCXYhE2FZ6M7PjAP4MwBEAfQBn3P2LZvYZAH8A4I3WrJ9y9+/F9uUAOqRGVqx2Fks+6EcSWrrgEpr3IjJUc43aDk2E5Y5uJyKhRdo/dTuRRJIutzW6/HV3nMxjJM+oF5Fr+h0ueVWKfNz8VFjaqlV4ski9wc/L6jpvDZXLcz+mKtPB7YcmuKxVKXNtM9YOC+AJSjHJsVwOz0l1jO8vTxKDYokwO9HZuwD+2N2fM7MagB+b2feHti+4+3/cwT6EEAfMTnq9XQFwZfh4w8xeAcC7Hwoh7khu6X92MzsB4N0Anhlu+oSZvWBmT5jZ1B77JoTYQ3Yc7GY2DuBbAD7p7usAvgTgfgAPYXDn/xwZd9rMzprZ2etLS6GnCCFGwI6C3cwKGAT6V9392wDg7ovu3nP3PoAvA3g4NNbdz7j7KXc/NTs3t1d+CyFukW2D3QbLe18B8Iq7f/6m7Qs3Pe3DAF7ae/eEEHvFTlbj3wfgYwBeNLPnh9s+BeCjZvYQBqLOOQB/uJMD9nPh9xfvx9r0hGWLTmRM1yLF0/pcTuq0ufwzMxVeljg0MUHHXLxwidq6fT79nnH/sz5/j3bSEitW4c8imXmljNvm57k0NDcVtvXb3Pd6MyLzjfG5Gq/xbLPpmbBteipS767AMw67vcg11+OyXK/LaxGWiuHXFsuwu/WmVjtbjf8hgNAVFNXUhRB3FvoGnRCJoGAXIhEU7EIkgoJdiERQsAuRCCMvOElzcniyDkCKKMZb3fCX1o+8x3Ui8k9GjrewwL8sdOH8MrUtr0SOleP+52Jtr9iURAp6ekR6K5S4RDUeycrK5cKOrG5EsteMH2vh8Ay1TU7xDLZDk2FZNJaxF5Nfs4zLfM1WpFhpJPuxR+Y/kihHz1lMYtWdXYhEULALkQgKdiESQcEuRCIo2IVIBAW7EIlwANJbWBzoOc/jaXfDGWAZyaADgMx40cB2pEdZo86LR26srQa3lwp8f0fmeY+vVoMXt2xGssMy45Jdn8xvP9pLj5pQKHA5rFioUJuT/neRU4ZCJMOumEX6ykVseTJX9S0uATbbvBdgbYIXZOr2+UTW+S7RaIWv73KNhyc7L3E5WgiRBAp2IRJBwS5EIijYhUgEBbsQiaBgFyIRRiq9GXjGVrfH5SRHWJooOH+vMpIpBwCb61zyitla9c3g9nYzLMkBwES45RkAYCaSrXUjkhEHj2T0kZ553R4voljIc3mtWuHyWpbxcUZ6zk1O1uiYYhaTjSJZYx0uvW1thDWvXqRoZ1bk5yUX8bHR5nNcqvELoVQO22IZdjlyn44lj+rOLkQiKNiFSAQFuxCJoGAXIhEU7EIkwrar8WZWBvADAKXh8//c3T9tZvcC+DqAaQDPAfiYu/Ml0yHs3SWXRWqC9cOrnJ0uP1wxz/fXqG9Qm0daQ41Xw6ujrSZfNe1Gkm5mZ/j0tyL1zPotnuTTJYfLRRKDKlW+4p7PR47V4S2q2PwXinx/WeTy8YiaUC7yle7x8VJwey+iaKDAa+vlSasmAOhF7p21yXlqq4yF6+T1Yh3M2uG5itUT3MmdvQXgA+7+Gxi0Z37EzN4L4E8AfMHdTwJYAfDxHexLCHFAbBvsPuANgbkw/HEAHwDw58PtTwL40L54KITYE3banz0bdnC9BuD7AF4DsOrub3y2ugjg6P64KITYC3YU7O7ec/eHABwD8DCAd4aeFhprZqfN7KyZnV1aWrp9T4UQu+KWVuPdfRXAXwF4L4BJM3tjteIYgMtkzBl3P+Xup+bmeDMFIcT+sm2wm9mcmU0OH1cA/CMArwD4SwD/dPi0xwB8d7+cFELsnp0kwiwAeNLMMgzeHL7p7v/TzP4GwNfN7N8B+H8AvrKTA7IadCDJLsNB4RHO5Zgu06AAdNpb1FbkKhqq1bCM0+lw6SrHlSYAvN7d2jpPaVhvRuaKkGX8VI9FpLd2L1IbsMPnn+WL9Lq8tRJRyQAAhya4HDY5MUZtEyQTqQd+sI0mf82Wi7XK4gk0+SL3v9MldQP7kcJ1JNGIJUIBOwh2d38BwLsD23+Bwf/vQohfAfQNOiESQcEuRCIo2IVIBAW7EImgYBciEcwjbYH2/GBmSwBeH/45C+D6yA7OkR9vRn68mV81P+5x9+C310Ya7G86sNlZdz91IAeXH/IjQT/0MV6IRFCwC5EIBxnsZw7w2DcjP96M/HgzvzZ+HNj/7EKI0aKP8UIkwoEEu5k9YmY/NbNXzezxg/Bh6Mc5M3vRzJ43s7MjPO4TZnbNzF66adu0mX3fzH4+/D11QH58xswuDefkeTP74Aj8OG5mf2lmr5jZy2b2L4bbRzonET9GOidmVjazZ83sJ0M//s1w+71m9sxwPr5hZpEczQDuPtIfABkGZa3uA1AE8BMAD47aj6Ev5wDMHsBxfxPAewC8dNO2/wDg8eHjxwH8yQH58RkA/3LE87EA4D3DxzUAPwPw4KjnJOLHSOcEg6Tu8eHjAoBnMCgY800AHxlu/88A/vmt7Pcg7uwPA3jV3X/hg9LTXwfw6AH4cWC4+w8ALL9l86MYFO4ERlTAk/gxctz9irs/N3y8gUFxlKMY8ZxE/BgpPmDPi7weRLAfBXDhpr8PslilA/gLM/uxmZ0+IB/e4LC7XwEGFx0AXmh8//mEmb0w/Ji/7/9O3IyZncCgfsIzOMA5eYsfwIjnZD+KvB5EsIdqmByUJPA+d38PgH8C4I/M7DcPyI87iS8BuB+DHgFXAHxuVAc2s3EA3wLwSXdfH9Vxd+DHyOfEd1HklXEQwX4RwPGb/qbFKvcbd788/H0NwHdwsJV3Fs1sAQCGv68dhBPuvji80PoAvowRzYmZFTAIsK+6+7eHm0c+JyE/DmpOhse+5SKvjIMI9h8BODlcWSwC+AiAp0bthJmNmVntjccAfgfAS/FR+8pTGBTuBA6wgOcbwTXkwxjBnJiZYVDD8BV3//xNppHOCfNj1HOyb0VeR7XC+JbVxg9isNL5GoB/dUA+3IeBEvATAC+P0g8AX8Pg42AHg086HwcwA+BpAD8f/p4+ID/+G4AXAbyAQbAtjMCPf4jBR9IXADw//PngqOck4sdI5wTA38egiOsLGLyx/OubrtlnAbwK4H8AKN3KfvUNOiESQd+gEyIRFOxCJIKCXYhEULALkQgKdiESQcEuRCIo2IVIBAW7EInw/wEsCOLTGLmSPQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "size of data: 6066 , size of labels: 6066\n",
      "size of test data: 2000 , size of labels: 2000\n"
     ]
    }
   ],
   "source": [
    "# extracting dog and deer images for train and test data-sets\n",
    "dd_X, dd_y = extract_images(train_data, train_labels)\n",
    "dd_X_test, dd_y_test = extract_images(test_data, test_labels)\n",
    "\n",
    "assert len(dd_X) == len(dd_y)\n",
    "assert len(dd_X_test) == len(dd_y_test)\n",
    "\n",
    "print(\"size of data:\", len(dd_X), \", size of labels:\", len(dd_y))\n",
    "print(\"size of test data:\", len(dd_X_test), \", size of labels:\", len(dd_y_test))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0 ==> Loss: 0.10176522676557169, Error: 0.09675841627374149\n",
      "Epoch 200 ==> Loss: 0.7590794019820655, Error: -0.5319028419696895\n",
      "Epoch 400 ==> Loss: 0.9297101479484018, Error: -0.605331910850321\n",
      "Epoch 600 ==> Loss: 0.523586848374442, Error: 0.4076080872627931\n",
      "Epoch 800 ==> Loss: 1.007723418088607, Error: 0.6349509015294933\n",
      "Epoch 1000 ==> Loss: 0.6765311692001913, Error: -0.49162258747297755\n",
      "Epoch 1200 ==> Loss: 0.749614066521602, Error: -0.5274511100094458\n",
      "Epoch 1400 ==> Loss: 0.5640561273221085, Error: -0.43110314037427294\n",
      "Epoch 1600 ==> Loss: 1.1366972063297625, Error: -0.6791229356565387\n",
      "Epoch 1800 ==> Loss: 0.8384665076047759, Error: -0.567626943901733\n",
      "\n",
      "EVALUATION ON TRAINING DATA:\n",
      "Training Accuracy: 100.0%\n",
      "\n",
      "PREDICTIONS FOR TESTING DATA:\n",
      "Accuracy: 50.0%\n"
     ]
    }
   ],
   "source": [
    "from sklearn import preprocessing\n",
    "\n",
    "# Normalising image data using sklearn standard scalar function\n",
    "scalar = preprocessing.StandardScaler().fit(dd_X)\n",
    "dd_X_train = scalar.transform(dd_X)\n",
    "dd_y_train = dd_y\n",
    "dd_X_test = scalar.transform(dd_X_test)\n",
    "dd_y_test = dd_y_test\n",
    "\n",
    "# Training neural network for image classification with single hidden layer\n",
    "ddweights, ddbiases = Stochastic_NN(dd_X_train, dd_y_train, 0.05, 2000)\n",
    "\n",
    "# Evaluating the model on training data\n",
    "print(\"\\nEVALUATION ON TRAINING DATA:\")\n",
    "training_predictions = predictNN(dd_X_train, ddweights, ddbiases)\n",
    "print(f\"Training Accuracy: {round(accuracy_score(dd_y_train, training_predictions)) * 100}%\")\n",
    "\n",
    "# Using the model to predict classes of test data\n",
    "print(\"\\nPREDICTIONS FOR TESTING DATA:\")\n",
    "ddpredictions = predictNN(dd_X_test, ddweights, ddbiases)\n",
    "print(f\"Accuracy: {accuracy_score(dd_y_test, ddpredictions) * 100}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- From above results it can be seen that running the shallow neural network with one hidden layer consisting of 3 neurons, provides 100% accuracy on training data, however, very low accuracy on the testing data, which could mean the neural network is overfitting the model.\n",
    "- However, the changes in the loss could be an indication that the data-set consists of saddle points.\n",
    "- Further, it was found that increasing the number of neurons does not affect the accuracy.\n",
    "- Reducing the learning rate and epochs from the set 0.05 alpha and 2000 epochs reduces the performance on training data as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 5 Solution: Deep Learning Enhancements\n",
    "\n",
    "The following section contains enhancement to Part 3 neural network architecture. \n",
    "\n",
    "> #### Part 5.1: Implementing Enhacements to Part 3 Architecture\n",
    "> - The Enhanced_NN() provides support for mutliple hidden layers, multiple neurons in the hidden layers.\n",
    "> - Further the Part 3 architecture has been updated to perform back propagation with momentum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Part 5.1: Implementing Enhacements to Part 3 Architecture\n",
    "def Enhanced_NN(X, y, alpha, epochs, layers=2, hidden_layer_neurons=[3], beta=0.9):\n",
    "    '''\n",
    "       Method to run deep neural network using Backprop with Momentum\n",
    "       \n",
    "       X: training data-set of fixed samples\n",
    "       y: labels for each sample of X\n",
    "       epochs: total number of epochs\n",
    "       layers: total number of layers of neural network, including output layer\n",
    "       hidden_layer_neurons: array of number of neurons for all hidden layers\n",
    "       beta: momentum set to default value of 0.9\n",
    "    '''\n",
    "    # Getting number of samples and number of features in training data-set X\n",
    "    (nsamples, nattributes) = np.shape(X)\n",
    "    # Initialising threshold to small value to check convergence\n",
    "    threshold = 1e-6\n",
    "    \n",
    "    # Setting the number of neurons in hidden and output layers    \n",
    "    layer_neurons = []\n",
    "    for layer in range(layers-1):\n",
    "        layer_neurons.append(hidden_layer_neurons[layer])\n",
    "    layer_neurons.append(1)\n",
    "    # Setting the number of neurons for input layer\n",
    "    layer_neurons.insert(0, nattributes)\n",
    "    \n",
    "    # Initialising weights and biases for each layer to small random values\n",
    "    weights = []\n",
    "    biases = []\n",
    "    for i in range(1, layers + 1):\n",
    "        weights.append(np.random.rand(layer_neurons[i], layer_neurons[i - 1]))\n",
    "        biases.append(np.array(np.random.rand(layer_neurons[i])))\n",
    "    \n",
    "    # Initialising cost value to 0\n",
    "    J_prev = 0\n",
    "    V_w = weights\n",
    "    V_b = biases\n",
    "    for epoch in range(epochs):\n",
    "        # Randomly selecting a sample from X\n",
    "        idx = np.random.choice(nsamples, 1, replace=False)\n",
    "        random_X = X[idx[0]]\n",
    "        random_y = y[idx[0]]\n",
    "        \n",
    "        # Performing the forward pass, implemented in Part 3.2\n",
    "        sigmas, activations = ForwardPropagation(random_X, weights, biases)\n",
    "        \n",
    "        # Getting the activation value of output neuron\n",
    "        y_hat = activations[len(activations)-1][0]\n",
    "        \n",
    "        # Calculating the cost\n",
    "        J_curr = -1 * ((random_y * np.log(y_hat)) + ((1 - random_y) * np.log(1 - y_hat)))        \n",
    "        # Checking if convergence has reached\n",
    "        if np.absolute(J_curr - J_prev) < threshold:\n",
    "            break\n",
    "        else:\n",
    "            J_prev = J_curr\n",
    "        \n",
    "        # Performing the backward pass, implemented in Part 3.3\n",
    "        delta_weights, delta_biases = BackPropagation(random_y, activations, sigmas, weights)\n",
    "        \n",
    "        # Setting the exponentially weighted moving average of gradients of weights and biases (V_w and V_b)\n",
    "        if epoch == 1:\n",
    "            # Initialising V_w and V_b with gradients for first iteration\n",
    "            V_w = delta_weights\n",
    "            V_b = delta_biases\n",
    "        else:\n",
    "            # Calculating V_w and V_b using momentum beta and gradients of weights and biases\n",
    "            for layer in range(len(delta_weights)):\n",
    "                for i in range(delta_weights[layer].shape[0]):\n",
    "                    for j in range(delta_weights[layer].shape[1]):\n",
    "                        V_w[layer][i][j] = ((1-beta) * delta_weights[layer][i][j]) + (beta * V_w[layer][i][j])\n",
    "            \n",
    "            for layer in range(len(delta_biases)):\n",
    "                for i in range(delta_biases[layer].shape[0]):\n",
    "                    V_b[layer][i] = ((1-beta) * delta_biases[layer][i]) + (beta * V_b[layer][i])\n",
    "        \n",
    "        # Updating the weights and biases using V_w and V_b, implemented in Part 3.4\n",
    "        weights, biases = SGD_Parameter_Update(alpha, weights, biases, V_w, V_b)\n",
    "        # Printing loss in intervals of epochs\n",
    "        if epoch % (epochs/10) == 0:\n",
    "            print(f\"Epoch {epoch} ==> Loss: {J_curr}, Error: {random_y - y_hat}\")\n",
    "    return weights, biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> #### Part 5.2: Testing Enhanced_NN on CIFAR-10 Data\n",
    "> Using the same data-set loaded and converted into gray-scale 1D vector, the Enhanced_NN is tested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0 ==> Loss: 0.003472512824039907, Error: 0.003466490624122187\n",
      "Epoch 100 ==> Loss: 0.7067223724048205, Error: -0.506741732234717\n",
      "Epoch 200 ==> Loss: 0.9252069864453678, Error: -0.6035506490601293\n",
      "Epoch 300 ==> Loss: 0.5239163001206281, Error: -0.40780321966775007\n",
      "Epoch 400 ==> Loss: 0.4456808560035204, Error: 0.3596118845251034\n",
      "Epoch 500 ==> Loss: 0.6000236114229988, Error: -0.4512013219766802\n",
      "Epoch 600 ==> Loss: 0.8180753288764381, Error: -0.5587198431245959\n",
      "Epoch 700 ==> Loss: 0.4863975531645456, Error: 0.3851626718054796\n",
      "Epoch 800 ==> Loss: 1.1471829241990217, Error: -0.6824699832581921\n",
      "Epoch 900 ==> Loss: 0.7413368098717807, Error: -0.5235234689453543\n",
      "\n",
      "EVALUATION ON TRAINING DATA:\n",
      "Training Accuracy: 100.0%\n",
      "\n",
      "PREDICTIONS FOR TESTING DATA:\n",
      "Accuracy: 50.0%\n"
     ]
    }
   ],
   "source": [
    "# Part 5.2: Testing Enhanced_NN on CIFAR-10 Data\n",
    "# Training neural network for image classification with 2 hidden layers, with 4 and 8 neurons respectively\n",
    "ddweights_enhanced, ddbiases_enhanced = \\\n",
    "    Enhanced_NN(dd_X_train, dd_y_train, 0.05, 1000, layers=3, hidden_layer_neurons=[4, 8])\n",
    "\n",
    "# Evaluating the model on training data\n",
    "print(\"\\nEVALUATION ON TRAINING DATA:\")\n",
    "training_predictions_enhanced = predictNN(dd_X_train, ddweights_enhanced, ddbiases_enhanced)\n",
    "print(f\"Training Accuracy: {round(accuracy_score(dd_y_train, training_predictions_enhanced)) * 100}%\")\n",
    "\n",
    "# Using the model to predict classes of test data\n",
    "print(\"\\nPREDICTIONS FOR TESTING DATA:\")\n",
    "ddpredictions_enhanced = predictNN(dd_X_test, ddweights_enhanced, ddbiases_enhanced)\n",
    "print(f\"Accuracy: {accuracy_score(dd_y_test, ddpredictions_enhanced) * 100}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- From the above results it can be seen that increasing the number of hidden layers and neurons, along with adding momentum is still giving the same results as Part 4. \n",
    "- Thus, for the given enhancement there has been no improvement in the accuracy of the data-set.\n",
    "- Different configurations of hidden layers, neurons and epochs were tested, however, every configuration gave similar results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References:\n",
    "\n",
    "    [1] Madden, M. (2022). Topic 02: Fundamentals of Neural Networks, Part 1.\n",
    "    [2] Madden, M. (2022). Simple Python routine to read and plot a small dataset. [Source: LoadDataset.ipynb]\n",
    "    [3] Agrawal, S. (2021). How to split data into three sets (train, validation, and test) And why?. Available at: https://towardsdatascience.com/how-to-split-data-into-three-sets-train-validation-and-test-and-why-e50d22d3e54c#:~:text=Train%2DValid%2DTest%20split%20is,of%20these%20datasets%20is%20below. (Accessed on: 25/01/2022)\n",
    "    [4] Madden, M. (2021). Load and View CIFAR-10 Data. [Source: LoadCIFAR10.ipynb]\n",
    "    [5] Learning Multiple Layers of Features from Tiny Images, Alex Krizhevsky, 2009.\n",
    "\n",
    "### Acknowledgments:\n",
    "\n",
    "Following resources were also referred to complete the above tasks,\n",
    "\n",
    "1. sklearn.model_selection.train_test_split. Available at: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html (Accessed on: 21/01/22)\n",
    "2. Madden, M. (2022). Topic 03: Fundamentals of Neural Networks, Part 2.\n",
    "3. Madden, M. (2022). Topic 04: Deep Neural Networks, Part 1.\n",
    "4. Madden, M. (2022). Topic 05: Deep Neural Networks, Part 2.\n",
    "5. numpy.ndarray.flatten. Available at: https://numpy.org/doc/stable/reference/generated/numpy.ndarray.flatten.html (Accessed on: 25/02/22)\n",
    "6. numpy.mean. Available at: https://numpy.org/doc/stable/reference/generated/numpy.mean.html#numpy-mean (Accessed on: 25/02/22)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}