(old) KTAI_Crawler_with_simple_Q-Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning to Crawl with Q-Learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: gymnasium in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (0.29.1)\n",
      "Requirement already satisfied: numpy>=1.21.0 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from gymnasium) (1.24.3)\n",
      "Requirement already satisfied: cloudpickle>=1.2.0 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from gymnasium) (2.2.1)\n",
      "Requirement already satisfied: typing-extensions>=4.3.0 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from gymnasium) (4.7.1)\n",
      "Requirement already satisfied: farama-notifications>=0.0.1 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from gymnasium) (0.0.4)\n",
      "Requirement already satisfied: importlib-metadata>=4.8.0 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from gymnasium) (6.0.0)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\tobia\\anaconda3\\envs\\ktai\\lib\\site-packages (from importlib-metadata>=4.8.0->gymnasium) (3.11.0)\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install gymnasium"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np, numpy.random as nr\n",
    "import gymnasium as gym\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from crawler_env import KTAICrawlerEnv\n",
    "\n",
    "\n",
    "np.set_printoptions(precision=3)\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's run the crawler environment with a random policy. You will see the random controller can sometimes make progress but it won't get very far. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = KTAICrawlerEnv(\n",
    "    render=True, # turn render mode on to visualize random motion\n",
    ")\n",
    "\n",
    "# standard procedure for interfacing with a Gym environment\n",
    "cur_state = env.reset() # reset environment and get initial state\n",
    "ret = 0.\n",
    "done = False\n",
    "i = 0\n",
    "while not done:\n",
    "    action = env.action_space.sample() # sample an action randomly\n",
    "    next_state, reward, done, info = env.step(action)\n",
    "    ret += reward\n",
    "    cur_state = next_state\n",
    "    time.sleep(0.01)\n",
    "    i += 1\n",
    "    if i == 1500:\n",
    "        break # for the purpose of this visualization, let's only run for 1500 steps\n",
    "        # also note the GUI won't close automatically"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can close the visualization GUI with the following method \n",
    "env.close_gui()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We can inspect the observation space and action space of this Gymnasium Environment\n",
      "-----------------------------------------------------------------------------\n",
      "Action space: Discrete(4)\n",
      "It's a discrete space with 4 actions to take\n",
      "Each action corresponds to increasing/decreasing the angle of one of the joints\n",
      "We can also sample from this action space: 3\n",
      "Another action sample: 1\n",
      "Another action sample: 0\n",
      "Observation space: Tuple(Discrete(9), Discrete(13)) , which means it's a 9x13 grid.\n",
      "It's the discretized version of the robot's two joint angles\n"
     ]
    }
   ],
   "source": [
    "env = KTAICrawlerEnv()\n",
    "\n",
    "print(\"We can inspect the observation space and action space of this Gymnasium Environment\")\n",
    "print(\"-----------------------------------------------------------------------------\")\n",
    "print(\"Action space:\", env.action_space)\n",
    "print(\"It's a discrete space with %i actions to take\" % env.action_space.n)\n",
    "print(\"Each action corresponds to increasing/decreasing the angle of one of the joints\")\n",
    "print(\"We can also sample from this action space:\", env.action_space.sample())\n",
    "print(\"Another action sample:\", env.action_space.sample())\n",
    "print(\"Another action sample:\", env.action_space.sample())\n",
    "print(\"Observation space:\", env.observation_space, \", which means it's a 9x13 grid.\")\n",
    "print(\"It's the discretized version of the robot's two joint angles\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's implement Tabular Q-Learning with $\\epsilon$-greedy exploration to find a better policy piece by piece.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Q-values for state (0, 0): [0. 0. 0. 0.] which is a list of Q values for each action\n",
      "As such, the Q value of taking action 3 in state (1,2), i.e. Q((1,2), 3), can be accessed by q_vals[(1,2)][3]: 0.0\n"
     ]
    }
   ],
   "source": [
    "from collections import defaultdict\n",
    "import random\n",
    "\n",
    "# dictionary that maps from state, s, to a numpy array of Q values [Q(s, a_1), Q(s, a_2) ... Q(s, a_n)]\n",
    "#   and everything is initialized to 0.\n",
    "q_vals = defaultdict(lambda: np.array([0. for _ in range(env.action_space.n)]))\n",
    "\n",
    "print(\"Q-values for state (0, 0): %s\" % q_vals[(0, 0)], \"which is a list of Q values for each action\")\n",
    "print(\"As such, the Q value of taking action 3 in state (1,2), i.e. Q((1,2), 3), can be accessed by q_vals[(1,2)][3]:\", q_vals[(1,2)][3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test1 passed\n",
      "Test2 passed\n"
     ]
    }
   ],
   "source": [
    "def eps_greedy(q_vals, eps, state):\n",
    "    \"\"\"\n",
    "    Inputs:\n",
    "        q_vals: q value tables\n",
    "        eps: epsilon\n",
    "        state: current state\n",
    "    Outputs:\n",
    "        random action with probability of eps; argmax Q(s, .) with probability of (1-eps)\n",
    "    \"\"\"\n",
    "    # you might want to use random.random() to implement random exploration\n",
    "    #   number of actions can be read off from len(q_vals[state])\n",
    "    import random\n",
    "    \n",
    "    ############################################################\n",
    "    # TODO: Implement Epsilon Greedy exploration for choosing  #\n",
    "    # an action.                                               #\n",
    "    ############################################################\n",
    "    if random.random() < eps:\n",
    "        action = env.action_space.sample() #exploration\n",
    "    else:\n",
    "        action = np.argmax(q_vals[state]) #exploitation\n",
    "    return action\n",
    "    \n",
    "# test 1\n",
    "dummy_q = defaultdict(lambda: np.array([0. for _ in range(env.action_space.n)]))\n",
    "test_state = (0, 0)\n",
    "dummy_q[test_state][0] = 10.\n",
    "trials = 100000\n",
    "sampled_actions = [\n",
    "    int(eps_greedy(dummy_q, 0.3, test_state))\n",
    "    for _ in range(trials)\n",
    "]\n",
    "freq = np.sum(np.array(sampled_actions) == 0) / trials\n",
    "tgt_freq = 0.3 / env.action_space.n + 0.7\n",
    "if np.isclose(freq, tgt_freq, atol=1e-2):\n",
    "    print(\"Test1 passed\")\n",
    "else:\n",
    "    print(\"Test1: Expected to select 0 with frequency %.2f but got %.2f\" % (tgt_freq, freq))\n",
    "    \n",
    "# test 2\n",
    "dummy_q = defaultdict(lambda: np.array([0. for _ in range(env.action_space.n)]))\n",
    "test_state = (0, 0)\n",
    "dummy_q[test_state][2] = 10.\n",
    "trials = 100000\n",
    "sampled_actions = [\n",
    "    int(eps_greedy(dummy_q, 0.5, test_state))\n",
    "    for _ in range(trials)\n",
    "]\n",
    "freq = np.sum(np.array(sampled_actions) == 2) / trials\n",
    "tgt_freq = 0.5 / env.action_space.n + 0.5\n",
    "if np.isclose(freq, tgt_freq, atol=1e-2):\n",
    "    print(\"Test2 passed\")\n",
    "else:\n",
    "    print(\"Test2: Expected to select 2 with frequency %.2f but got %.2f\" % (tgt_freq, freq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will implement Q learning update. After we observe a transition $s, a, s', r$,\n",
    "\n",
    "$$\\textrm{target}(s') = R(s,a,s') + \\gamma \\max_{a'} Q_{\\theta_k}(s',a')$$\n",
    "\n",
    "\n",
    "$$Q_{k+1}(s,a) \\leftarrow (1-\\alpha) Q_k(s,a) + \\alpha \\left[ \\textrm{target}(s') \\right]$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test passed\n"
     ]
    }
   ],
   "source": [
    "def q_learning_update(gamma, alpha, q_vals, cur_state, action, next_state, reward):\n",
    "    \"\"\"\n",
    "    Inputs:\n",
    "        gamma: discount factor\n",
    "        alpha: learning rate\n",
    "        q_vals: q value table\n",
    "        cur_state: current state\n",
    "        action: action taken in current state\n",
    "        next_state: next state results from taking `action` in `cur_state`\n",
    "        reward: reward received from this transition\n",
    "    \n",
    "    Performs in-place update of q_vals table to implement one step of Q-learning\n",
    "    \"\"\"\n",
    "    ############################################################\n",
    "    # TODO: Implement an in-place q learning update.           #\n",
    "    ############################################################\n",
    "    \n",
    "    q_vals[cur_state][action] = q_vals[cur_state][action] + alpha * (reward + gamma * np.max(q_vals[next_state]) - q_vals[cur_state][action])\n",
    "    \n",
    "# testing your q_learning_update implementation\n",
    "dummy_q = q_vals.copy()\n",
    "test_state = (0, 0)\n",
    "test_next_state = (0, 1)\n",
    "dummy_q[test_state][0] = 10.0\n",
    "dummy_q[test_next_state][1] = 10.0\n",
    "q_learning_update(0.9, 0.1, dummy_q, test_state, 0, test_next_state, 1.1)\n",
    "tgt = 10.01\n",
    "if np.isclose(dummy_q[test_state][0], tgt,):\n",
    "    print(\"Test passed\")\n",
    "else:\n",
    "    print(\"Q(test_state, 0) is expected to be %.2f but got %.2f\" % (tgt, dummy_q[test_state][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "def greedy_eval():\n",
    "    \"\"\"evaluate greedy policy w.r.t current q_vals\"\"\"\n",
    "    test_env = KTAICrawlerEnv(horizon=np.inf)\n",
    "    \n",
    "    ret = 0.\n",
    "    done = False\n",
    "    H = 100\n",
    "    \n",
    "    current_state = test_env.reset()\n",
    "    for i in range(H):\n",
    "        \n",
    "        ############################################################\n",
    "        # TODO: Implement a greedy policy, that is, always choose  #\n",
    "        # the action with the highest q values.                    #\n",
    "        ############################################################\n",
    "        \n",
    "        ############################################################\n",
    "        # Hint: Look at the code on how to run the random policy   #\n",
    "        # at the beginning of this notebook.                       #\n",
    "        ############################################################\n",
    "\n",
    "        ############################################################\n",
    "        # Step 1: determine the action based on the q values at    #\n",
    "        # the current state.                                       #\n",
    "        ############################################################\n",
    "        action = np.argmax(q_vals[current_state])\n",
    "    \n",
    "        ############################################################\n",
    "        # Step 2: take the action and get the next state, rewards. #\n",
    "        ############################################################\n",
    "        next_state, reward, done, info = env.step(action)\n",
    "        q_learning_update(gamma, alpha, q_vals, current_state, action, next_state, reward)     \n",
    "        \n",
    "        ret += reward\n",
    "        current_state = next_state\n",
    "    return ret / H"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Itr 0 # Average speed: 0.05\n",
      "Itr 50000 # Average speed: 2.52\n",
      "Itr 100000 # Average speed: 3.21\n",
      "Itr 150000 # Average speed: 3.29\n",
      "Itr 200000 # Average speed: 3.21\n",
      "Itr 250000 # Average speed: 3.37\n",
      "Itr 300000 # Average speed: 3.29\n"
     ]
    }
   ],
   "source": [
    "# now with the main components tested, we can put everything together to create a complete q learning agent\n",
    "\n",
    "env = KTAICrawlerEnv() \n",
    "# initialize q_values to 0\n",
    "q_vals = defaultdict(lambda: np.array([0. for _ in range(env.action_space.n)]))\n",
    "# gamma discount factor\n",
    "gamma = 0.9\n",
    "# alpha learning rate\n",
    "alpha = 0.1\n",
    "# epsilon greedy exploration parameter\n",
    "eps = 0.5\n",
    "# decay = 0.01\n",
    "# min_eps = 0.5\n",
    "\n",
    "epochs_total = []\n",
    "rewards_total = []\n",
    "\n",
    "current_state = env.reset()\n",
    "epochs, rewards = 0, 0\n",
    "\n",
    "for itr in range(300001):\n",
    "    ############################################################\n",
    "    # TODO: use epsilon greedy actions and perform q learning  #\n",
    "    # update                                                   #\n",
    "    ############################################################\n",
    "    action = eps_greedy(q_vals, eps, current_state) #chose action based on explore vs exploit\n",
    "    \n",
    "    next_state, reward, done, info = env.step(action) #use action to renew info\n",
    "    \n",
    "    q_learning_update(gamma, alpha, q_vals, current_state, action, next_state, reward) #upate Q-table\n",
    "    \n",
    "    current_state = next_state #use new state to repeat\n",
    "    \n",
    "    rewards += reward \n",
    "    epochs+=1\n",
    "    \n",
    "    if itr % 50000 == 0: # evaluation with greedy evaluation\n",
    "        print(\"Itr %i # Average speed: %.2f\" % (itr, greedy_eval()))\n",
    "        \n",
    "    #eps = max(min_eps, eps - decay)\n",
    "\n",
    "    epochs_total.append(epochs)\n",
    "    rewards_total.append(rewards)\n",
    "\n",
    "# at the end of learning your crawler should reach a speed of >= 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHFCAYAAAAOmtghAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABN6klEQVR4nO3deVxU5f4H8M+wDYswCirjKCruC2qmhqDmjrmQpt00zVzSq5mWuS/dpOWHpmlWpl29ZXpzaZHMm2XgAmZomkvics1bhCtxVTY3EHh+f5w7A8MMMAwD58zh83695sXMc7bveebMOV+ec85zNEIIASIiIiKVcJE7ACIiIiJHYnJDREREqsLkhoiIiFSFyQ0RERGpCpMbIiIiUhUmN0RERKQqTG6IiIhIVZjcEBERkaowuSEiIiJVYXJDZKNPPvkEGo2mxFd8fHylLbtx48YYP368XdNu3boVq1evtjpMo9EgKirK7riqwr59+9C5c2f4+PhAo9Fg586dVse7du0aoqKicOrUKbuXlZiYiKioKGRkZNg9j6ioKGg0Grund2bG38gff/whdyhUzbnJHQCRs9m4cSNatWplUd6mTRsZoinb1q1bcebMGcycOdNi2OHDh9GgQYOqD8pGQgg89dRTaNGiBXbt2gUfHx+0bNnS6rjXrl3Da6+9hsaNG+Ohhx6ya3mJiYl47bXXMH78eNSsWdP+wIlIVkxuiMopJCQEnTt3ljsMh+jatavcIZTq2rVruHXrFp544gn07dtX7nAq3d27d+Ht7S13GCUSQuD+/fvw8vKSOxSiUvG0FJGDdezYET169LAoz8/PR/369TF8+HBT2a1btzBt2jTUr18fHh4eaNKkCRYvXoycnJxSl1FS8398fLzZKbJevXph9+7dSElJMTuFZmTttNSZM2cwdOhQ1KpVC56ennjooYewadMmq8vZtm0bFi9eDIPBAD8/P/Tr1w8XLlywoZaAQ4cOoW/fvvD19YW3tzfCw8Oxe/du0/CoqChTq9L8+fOh0WjQuHFjq/OKj49Hly5dAAATJkwwrWfRddu1axfCwsLg7e0NX19f9O/fH4cPHzZb3ty5cwEAwcHBFqcbP/vsM0RERKBevXrw8vJC69atsWDBAty5c8em9S1u/PjxqFGjBpKSkhAREQFfX19TApebm4s333wTrVq1glarRZ06dTBhwgT897//NU0/d+5c6HQ65Ofnm8pmzJgBjUaDFStWmMpu3rwJFxcXvP/++wCA+/fvY/bs2XjooYeg0+ng7++PsLAwfP311xYxajQaTJ8+HR9++CFat24NrVZr2haOHDmCbt26wdPTEwaDAQsXLsSDBw/sqgsiR2NyQ1RO+fn5yMvLM3sVPcBMmDABhw4dwsWLF82mi42NxbVr1zBhwgQA0kGmd+/e2Lx5M2bNmoXdu3fjmWeewfLly80SoIpYu3YtunXrBr1ej8OHD5teJblw4QLCw8Nx9uxZvPfee4iJiUGbNm0wfvx4LF++3GL8RYsWISUlBf/4xz+wfv16XLx4EZGRkWb1YU1CQgL69OmDzMxMfPTRR9i2bRt8fX0RGRmJzz77DAAwadIkxMTEAJAO2ocPH8ZXX31ldX4PP/wwNm7cCAB45ZVXTOs5adIkANKpuaFDh8LPzw/btm3DRx99hPT0dPTq1QuHDh0yLW/GjBkAgJiYGNM8Hn74YQDAxYsXMWjQIHz00UfYs2cPZs6cic8//xyRkZGlrmtpcnNz8fjjj6NPnz74+uuv8dprr6GgoABDhw7FsmXLMHr0aOzevRvLli1DXFwcevXqhXv37gEA+vXrh6ysLBw9etQ0v71798LLywtxcXGmsn379kEIgX79+gEAcnJycOvWLcyZMwc7d+7Etm3b0L17dwwfPhybN2+2iHHnzp1Yt24dXn31VXz//ffo0aMHzp07h759+yIjIwOffPIJPvzwQ5w8eRJvvvmm3XVB5FCCiGyyceNGAcDqy9XV1TTejRs3hIeHh1i0aJHZ9E899ZQIDAwUDx48EEII8eGHHwoA4vPPPzcb76233hIARGxsrKmsUaNGYty4cRaxJCcnm0174MABAUAcOHDAVDZ48GDRqFEjq+sEQCxZssT0edSoUUKr1YpLly6ZjTdw4EDh7e0tMjIyzJYzaNAgs/E+//xzAUAcPnzY6vKMunbtKurWrSuys7NNZXl5eSIkJEQ0aNBAFBQUCCGESE5OFgDEihUrSp2fEEIcO3ZMABAbN240K8/PzxcGg0G0a9dO5Ofnm8qzs7NF3bp1RXh4uKlsxYoVVuu1uIKCAvHgwQORkJAgAIhffvnFNGzJkiXCll3ruHHjBADx8ccfm5Vv27ZNABA7duywun5r164VQghx584d4eHhIV5//XUhhBBXrlwRAMT8+fOFl5eXuH//vhBCiMmTJwuDwVBiHHl5eeLBgwfiueeeEx07djQbBkDodDpx69Yts/KRI0cKLy8vkZqaajafVq1a2VR/RJWNLTdE5bR582YcO3bM7PXTTz+ZhgcEBCAyMhKbNm1CQUEBACA9PR1ff/01nn32Wbi5SZe67d+/Hz4+PnjyySfN5m+8K2rfvn1Vs0JF7N+/H3379kVQUJBFTHfv3rVo9Xn88cfNPrdv3x4AkJKSUuIy7ty5g59++glPPvkkatSoYSp3dXXF2LFjceXKFZtPbdniwoULuHbtGsaOHQsXl8JdXo0aNTBixAgcOXIEd+/eLXM+v//+O0aPHg29Xg9XV1e4u7ujZ8+eAIDz58/bHd+IESPMPn/zzTeoWbMmIiMjzVoHH3roIej1etNpMm9vb4SFhWHv3r0AgLi4ONSsWRNz585Fbm6uqUVq7969plYboy+++ALdunVDjRo14ObmBnd3d3z00UdW16NPnz6oVauWWdmBAwfQt29fBAYGmspcXV0xcuRIu+uByJGY3BCVU+vWrdG5c2ezV6dOnczGmThxIq5evWo6PbBt2zbk5OSY3c598+ZN6PV6i9uG69atCzc3N9y8ebPS16W4mzdvol69ehblBoPBNLyogIAAs89arRYATKdOrElPT4cQolzLqQjjvEpaXkFBAdLT00udx+3bt9GjRw/89NNPePPNNxEfH49jx46ZTpuVtr6l8fb2hp+fn1nZn3/+iYyMDHh4eMDd3d3slZqaihs3bpjG7devH44cOYI7d+5g79696NOnDwICAtCpUyfs3bsXycnJSE5ONktuYmJi8NRTT6F+/fr49NNPcfjwYRw7dgwTJ07E/fv3LWK0Vm/Gbbc4a2VEcuDdUkSVYMCAATAYDNi4cSMGDBiAjRs3IjQ01Ox28YCAAPz0008QQpglOGlpacjLy0Pt2rVLnL+npycAWFx4XPTAZ4+AgABcv37dovzatWsAUGpMtqpVqxZcXFwqfTlGxgSspOW5uLhYtEwUt3//fly7dg3x8fGm1hoAFeoPB4DV/nBq166NgIAA7Nmzx+o0vr6+pvd9+/bF3/72Nxw8eBD79u3DkiVLTOWxsbEIDg42fTb69NNPERwcjM8++8xs+SVdxG4txoCAAKSmplqUWysjkgNbbogqgfEUy86dO/HDDz/g559/xsSJE83G6du3L27fvm3RKZ3xos7Sbn023jV0+vRps/Jdu3ZZjKvVam1uWejbt6/pQF48Jm9vb4fcOu7j44PQ0FDExMSYxVVQUIBPP/0UDRo0QIsWLco935JajVq2bIn69etj69atEEKYyu/cuYMdO3aY7qAqbR7GA7xxuNHf//73csdZliFDhuDmzZvIz8+3aCHs3LmzWT8/jzzyCPz8/LB69Wqkpqaif//+AKQWnZMnT+Lzzz9HmzZtTC1ixnXx8PAwS1pSU1Ot3i1Vkt69e2Pfvn34888/TWX5+fmmi8GJ5MaWG6JyOnPmDPLy8izKmzZtijp16pg+T5w4EW+99RZGjx4NLy8vi+sRnn32WXzwwQcYN24c/vjjD7Rr1w6HDh1CdHQ0Bg0aZHGdRFFdunRBy5YtMWfOHOTl5aFWrVr46quvTNdZFNWuXTvExMRg3bp16NSpE1xcXErsp2fJkiX45ptv0Lt3b7z66qvw9/fHli1bsHv3bixfvhw6nc7WairV0qVL0b9/f/Tu3Rtz5syBh4cH1q5dizNnzmDbtm129fDbtGlTeHl5YcuWLWjdujVq1KgBg8EAg8GA5cuXY8yYMRgyZAimTJmCnJwcrFixAhkZGVi2bJlpHu3atQMAvPvuuxg3bhzc3d3RsmVLhIeHo1atWpg6dSqWLFkCd3d3bNmyBb/88otD6qOoUaNGYcuWLRg0aBBeeuklPPLII3B3d8eVK1dw4MABDB06FE888QQAKYnu2bMn/vWvfyE4OBhNmzYFAHTr1g1arRb79u3Diy++aDb/IUOGICYmBtOmTcOTTz6Jy5cv44033kC9evUs7vArySuvvIJdu3ahT58+ePXVV+Ht7Y0PPvjA7tviiRxO5guaiZxGaXdLARAbNmywmCY8PFwAEGPGjLE6z5s3b4qpU6eKevXqCTc3N9GoUSOxcOFC050uRsXvlhJCiF9//VVEREQIPz8/UadOHTFjxgyxe/dui7ulbt26JZ588klRs2ZNodFozO7kQbG7pYQQIikpSURGRgqdTic8PDxEhw4dLO5AMt4t9cUXX5iVG+9uKj6+NT/88IPo06eP8PHxEV5eXqJr167iX//6l9X52XK3lBDSnUatWrUS7u7uFuu2c+dOERoaKjw9PYWPj4/o27ev+PHHHy3msXDhQmEwGISLi4tZXSYmJoqwsDDh7e0t6tSpIyZNmiROnDhhsb7luVvKx8fH6rAHDx6It99+W3To0EF4enqKGjVqiFatWokpU6aIixcvmo377rvvCgBi8uTJZuX9+/cXAMSuXbss5r9s2TLRuHFjodVqRevWrcWGDRusxg1AvPDCC1Zj/PHHH0XXrl2FVqsVer1ezJ07V6xfv553S5EiaIQo0k5LRERE5OR4zQ0RERGpCpMbIiIiUhUmN0RERKQqsic3V69exTPPPIOAgAB4e3vjoYcewvHjx03DhRCIioqCwWCAl5cXevXqhbNnz8oYMRERESmZrMlNeno6unXrBnd3d3z33Xc4d+4cVq5ciZo1a5rGWb58OVatWoU1a9bg2LFj0Ov16N+/P7Kzs+ULnIiIiBRL1rulFixYgB9//BE//PCD1eFCCBgMBsycORPz588HIPWiGRgYiLfeegtTpkypynCJiIjICcia3LRp0wYDBgzAlStXkJCQgPr162PatGmYPHkyAOlBdU2bNsWJEyfQsWNH03RDhw5FzZo1sWnTJot55uTkmHUjXlBQgFu3biEgIMCujsGIiIio6gkhkJ2dDYPBYPbQW1snlo1WqxVarVYsXLhQnDhxQnz44YfC09NTbNq0SQghdRIFQFy9etVsusmTJ4uIiAir8zR2RMUXX3zxxRdffDn/6/Lly+XOL2R9/EJBQQE6d+6M6OhoAEDHjh1x9uxZrFu3Ds8++6xpvOItLqLYgwaLWrhwIWbNmmX6nJmZiYYNG+Ly5csWT98lIiIiZcrKykJQUJDZw2JtJWtyU69ePbOnJANA69atsWPHDgCAXq8HID3UrV69eqZx0tLSEBgYaHWeWq3W4uF2AODn58fkhoiIyMnYc0mJrHdLdevWDRcuXDAr+/XXX9GoUSMAQHBwMPR6PeLi4kzDc3NzkZCQgPDw8CqNlYiIiJyDrC03L7/8MsLDwxEdHY2nnnoKR48exfr167F+/XoAUrY2c+ZMREdHo3nz5mjevDmio6Ph7e2N0aNHyxk6ERERKZSsyU2XLl3w1VdfYeHChXj99dcRHByM1atXY8yYMaZx5s2bh3v37mHatGlIT09HaGgoYmNj7ToHR0REROqn+qeCZ2VlQafTITMzk9fcEBEROYmKHL9lf/wCERERkSMxuSEiIiJVYXJDREREqsLkhoiIiFSFyQ0RERGpCpMbIiIiUhUmN0RERKQqTG6IiIhIVWTtoZgU5vRp4L//BVxdgYICwOV/ue+DB9J7T08gJwcQArh/X/r84AGQny9N4+UF3LkD6HRAXp40rtv/NjF3d2m8ggJAo5H+3r8vzcvbW/qbny+N5+MjzcfFBcjNlcpyc6XPHh7SMgsKgNq1gXv3pPKMDGmYr680rRCFyzaug3G9cnOlYQ8eFC6roADQaqW4ASkWQIpVqy1cR41GWk5mZuG0Xl7S/HJzpXXS6aRyAPDzA+7eld7n5UnzyMuT5llQIMVQtB9NX19pPr16FdY/ERGVC5Mbktjx1FWqZOruPJyIqNLwX0MiIiJSFSY3REREpCpMboiIiEhVmNwQERGRqjC5ISIiIlVhckNERESqwuSGiIiIrNu4UeoqpKzXokVyR2pGI4S6O9PIysqCTqdDZmYm/Pz85A5HudjPjfKo+6dJRM6gPMcGB++zKnL8ZssNERERqQqTGyIiIlIVJjdERESkKkxuiIiISFWY3BAREZGqMLkhIiIiVWFyQ0RERKrC5IaIiIhUhckNERERqQqTGyIiIlIVJjdERESkKkxuiIiISFWY3BAREZGqMLkhIiIiVWFyQ0RERKrC5IaIiIhUhckNERERqQqTGyIiIlIVJjdERESkKkxu5PDWW4BGo6wXERGRSjC5kcOCBXJHQEREpFpMboiIiEhVmNwQERGRqjC5ISIiIlVhckNERESqwuSGiIiIVMVN7gCIqAS8RZ+IyC5suSEiIiJVYXJDREREqsLkhpTr559tG2/LltKHh4aWPlwI25ZDREROQdbkJioqChqNxuyl1+tNw4UQiIqKgsFggJeXF3r16oWzZ8/KGDEREREpnewtN23btsX169dNr6SkJNOw5cuXY9WqVVizZg2OHTsGvV6P/v37Izs7W8aISXHKanlhywwRUbUie3Lj5uYGvV5vetWpUweA1GqzevVqLF68GMOHD0dISAg2bdqEu3fvYuvWrTJHTUREREole3Jz8eJFGAwGBAcHY9SoUfj9998BAMnJyUhNTUVERIRpXK1Wi549eyIxMbHE+eXk5CArK8vsRURERNWHrMlNaGgoNm/ejO+//x4bNmxAamoqwsPDcfPmTaSmpgIAAgMDzaYJDAw0DbNm6dKl0Ol0pldQUFClrgM5AZ6WIiKqVmRNbgYOHIgRI0agXbt26NevH3bv3g0A2LRpk2kcTbGOzIQQFmVFLVy4EJmZmabX5cuXKyd4IiIikmzeLHcEZmQ/LVWUj48P2rVrh4sXL5rumireSpOWlmbRmlOUVquFn5+f2YtUji0zRERVY/x46+Vjx1ZpGGVRVHKTk5OD8+fPo169eggODoZer0dcXJxpeG5uLhISEhAeHi5jlOR0mPwQETmGkzwWRtZnS82ZMweRkZFo2LAh0tLS8OabbyIrKwvjxo2DRqPBzJkzER0djebNm6N58+aIjo6Gt7c3Ro8eLWfYpDRO8mMjInJ6TvLPoqzJzZUrV/D000/jxo0bqFOnDrp27YojR46gUaNGAIB58+bh3r17mDZtGtLT0xEaGorY2Fj4+vrKGTYpjZP82IiIqGrImtxs37691OEajQZRUVGIioqqmoBInZj8EBE5hpO0lCvqmhsiIiKiimJyQ86hdeuShznJfxJERE7PSVrCmdyQpYp0fPjyy8Drr5c93uuvA++/D7zxBjBpUsnj/fQTsHQpcOIEUNJdcmUlNyX9GAcNAo4ckd5fulR2zKX55pvSh/v7lzysY0dg/fqKLZ+IiEyY3JClS5fsz87HjAH+9reyx3vpJWD6dOCVVwAfn5LHe+QRYMECwNMT+PFH63HZG+v8+UBoqPQ+KMj2+UyYYFk2eHDp07RrJ82/cWPLYUePViyhJCKqKk7SUs7khhzLnkTDSZo5TZwtXiKiaobJDamfEpIRJ/lvh4hIDZjcVETTptJBq7wvcmzCoYTkxVbOFCsRkZNicmOvS5eA33+XOwqyhaMTCiYoRESKxuTGXjduyB2Bc2OCYI4tekTkDLRauSOwCZMbkl9FEx1nSpScKVYiouJWrgSaNDEv++QTWUIpjayPXyCqEjwtRUTkGN7ewG+/yR1FmdhyQ/KorhcUExFRpWNyQ0RERKrC5MZebC2omKIX0Fb2xbRKOi1V0rS8oJiIyGGY3FDVqlsXmDgRqFmzsMzRyUdMjGPn5whMXoiIqgwvKKaq9cknwMCBlbuMJ56wbTw5Wt+Y5BARVTq23DiLPXvkjsB5KeG0lHEans4kIqp0TG7IseQ4eDNhICKiIpjcUPVl7ykie5Ipno4iIqoyTG7sxdYCx6nsunSG74rJDxGRwzC5IceS4yBtb/LCU2hERKrE5IYcqzocvKvDOhIROTEmN+T8yko2lJCM8LQTEVGVYXJjr6o+YCrhAE32K+1WcH63RCQXX1+5I6gUTG7sxQNS9VUZ3z1bdpxHjx5Vu7xmzQAfn/JPl5Hh8FBsMm2a9fLRo8s/r5UrzT9HRwObN9s27cGD5V9edfP880BWFjB7ttyROByTG3sxuVEOZzgtRVTV5EqYlfL8NP7uy6bif6qY3BDJjTth51LVBwQVH4DswvpwHBXXJZMbe/GaG+ehhMcvGHci/B6dn4oPCA5Rmdt4eeqev7VqjcmNvfjDUQ5n/y6cPf7qRo6WG24jVBlUnKgzubEXdzaOw7pU9U6GZKLGa27Ks6/gfqVaY3Jjr6r+4Wg0wJYthZ/HjDEf/tBDlbdsT0/bx3V3L324Vuu4eRl5eZU+vHZt6+UeHrbHUlTNmuWfplYt6W/duvYtk5SD19xUHSYolUvF2xaTG7kJUfg6ebKw3MXFcrzRowvH/fRT8+FFp/3b36Rx/PwqFtvWrUDPnsDFi4Vl4eHAc89J75s3B779Vro1tmVL4OmngQ4dpGELFwK9elnO01pZUatXS38DA4F27cqOcfBg4C9/KXn4++8Dn3wivffwAJYvl94PHQp07Wo5/q5dQL16QIMGwPjxheVz5gDdugGHDgFvvCGth3H4xInS36Lxdu8uTQMAdeoA774rvd++XRq2apX0OSSkfAkfkdI5MiGpyLyYGFVrbnIH4LQq+4dTkYy6ItPu3Fn4/umnpRdgub7/+Efh+4EDLecTHW3f8l96SXrZonFj4JtvSh/n4YeBJk3M4587t+TxIyOll9HGjdbHO3DAcvibb0pJEwD88IP0d8UK8+latiwc9vLLpcdOyuQsLTdq/a9creslBxXXJVtu7MWO3IiILFVmyw3vliIbMbmxl1p/OGpLsJxlfZwlTnKelhs17qPUuE5yUvF+h8mNvSrjR1baPMuzPBVvsGaqy3qSssix3TnTts5rbkgBmNwolTPtzIiqE2fp50Zp+xClxUOq/k6Y3NiL19w4B2ep08r6L7Nbt9KHDxtm/rn4XXrOYsgQ28ft3l36a7xYvryMF41b07592dO/8Ubpw0eMMP88Zw4wY0bZ83W04ttGixa2TTd2bMnDIiLKF8Pjj5dv/KKMd246K3//yl/GyJGVvwyZOOmeTAHU2uQpRzKglrp0VN1dvw7cvAn85z/2z+PHH4E7d4D9+4FXXil5vC++ABYvLvycng5cvgwcOwb8+itw4UL5D0glmTev8H3NmtJ6WnPxInDrVsnz+eAD88/HjwNffQXcuGFbHAcPAv/+t9Sdwr//La3fd9/ZNi0g3fKfnAwEB1sOW7ECOHxYWoc//rA+fdH6LurECam+P/sMyMwE/vlP6c66CROAJ58sHK9oFwXr1gHnzpUd87lzwO+/lz3evn1StxK//w7ExJgPO39eWq+zZ6UuDUry6KNAx47Wh33zjbRdTZhQdizr1wOtWpl/r9Z+Yx99ZP0J4MX7t9q9W4r90qWSlzluHJCXZ363Z5MmluOVdsdleR09Cvz2m/mdqmfOSL/DonbsAN55x3L677+3LDMm8CUJCJCWaa07jOKKPs391VfLHl8heCu4vdR6KzjJo+h3ptdLf+39z61jR6k/IqNmzUoe183NfNl+ftKrQYPCssGDgdhY25fv6grk51uW9+tX2M+Qv3/hehZXWrwA0Lq1+ecWLaT1CAiwLT6NRrolH5D+Wjs4lKVxY+udR7q4lH3AKOn3WTQh8PMDnnmm8HPR/U3R+mnTxrI+rC3H09N6Mlbco49KdWnUoQPwyy/SexeXwmW3aQOMGlXyfJo2Ne97y8jdXeofy5bvqlEj6W/Rca3td439TJWlXTsgKEh67+0N3L1rOU6rVtL2GxkJLFoklRkMlomhMTZH6NJF+mvs6BOQfh/e3ubjtW0LDB9u2YVEnz7mnzUa6bd26FDJy/T0tJ60WfPpp8DSpVLdN2xo2zQKwOTGXmppbXBmTOJsw3pyrNLq09nrunj8cq6PnK3IZS3b2S8qL8/3rNE4VVJjxNNS9lJrcuPsO+fi1LY+zsCWOq/MHTU5jlr3c87I1u3cnt+DCr9nJjf2yshw/DyLbpTvveeY+RCVpawdG7cn66qyXop+R2o4EMl92r2iXWtUxnevhu9VQZjcyOnvfzf/XHTjnjIFuHbN+rCyONuPpKwHX5akRo3C9yU9ZNPV1b5524NJgKSkenDUdqnkelZSbEVjsbXuKzP+8s7b0bHYMj+l7DsrmlApaTuUCZMbeznioPnXv5Y8TKORHuBYHSxcKF1Mae1OAGu+/FK6uG7r1sKy+HjpIse9e6XPU6YAjz0mPVvKGThyZ+ToHbTSuz3gjrz6qKrvWu5tqiK/uao+LaxQvKCY5BcQIN0Ga6sRIyz7AgkPl27zNPrwQ8fEpgZl7SgdnbxUgx1nlSv6Hcl1OsyR5H6kREVbwivjO1DiRcxOjC03SuKojZc/AlIzJW/fSorNnlh4Wsqxy3QWStpuHYTJjbOorj86Kj+5d1S8W6pycV8gPzkuKK7Mu6VUSDHJzdKlS6HRaDBz5kxTmRACUVFRMBgM8PLyQq9evXC26KkHIrJUfCdZ1s6Op6WUTw0JjZIuKFd6fVbVHVpGSq8POygiuTl27BjWr1+P9sWey7J8+XKsWrUKa9aswbFjx6DX69G/f39kZ2fLFKmTkPs2SyJr1LIDdcZO/JRwt1R5l6OUR8FYi8PW52w5kp9f1S/Ticme3Ny+fRtjxozBhg0bUKtI99NCCKxevRqLFy/G8OHDERISgk2bNuHu3bvYWvQumepCqTtNcgw1fb88LUXWONP3VNZt9L17Axs2lDz9p5+aPwLFEYo/ssLdHdizx3I8pSaOVUz25OaFF17A4MGD0a9fP7Py5ORkpKamIqLIQ/u0Wi169uyJxMTEqg7TUlX/F6qW/3qp8pV3W+G2VT5y1Be/I8dyRH1OmlRykjBmjPR8J3vZEt+oUcCAAdaHyX3xtgLIeiv49u3bceLECRw7dsxiWGpqKgAgMDDQrDwwMBApKSklzjMnJwc5OTmmz1lZWQ6KtppQ4UZOVay6bENVfV2Emii1dcGZkkhua6WSreXm8uXLeOmll/Dpp5/C09OzxPE0xb5AIYRFWVFLly6FTqczvYKMT4GtTrjRkyOVd4dvSw/FjrwuTK7tXekHQnt6KC7Okeso935J7uWXR0VuW3em9axEsiU3x48fR1paGjp16gQ3Nze4ubkhISEB7733Htzc3EwtNsYWHKO0tDSL1pyiFi5ciMzMTNPr8uXLlboeRCZK2amUNw6lH6SVhi0K8qqKu6UcsQyl7A+sKR6bCrcv2U5L9e3bF0lJSWZlEyZMQKtWrTB//nw0adIEer0ecXFx6NixIwAgNzcXCQkJeOutt0qcr1arhVarrdTYK01pPwYVbnxUhJIfv8Dz97ar6nV31v1CRU5LVeY6O2t9Fledf4P/I1ty4+vri5CQELMyHx8fBAQEmMpnzpyJ6OhoNG/eHM2bN0d0dDS8vb0xevRoOUJ2HrwVnIpSyndaWY8QUMr6KZlaDtpyUGrdlbTdazSOO5XsxBT9bKl58+bh3r17mDZtGtLT0xEaGorY2Fj4+vrKHRqR86jos6WUtqNUyo5Yrmc82ZokKqWelMKWa5CM5c506tGe6arBtqGo5CY+Pt7ss0ajQVRUFKKiomSJh4jsoPZ+bphQVFxFtpHqUMdsfa8w2fu5cVrWsuWrV6VOlRIS7Jtns2bSX2/vwjLj3V7dulmO/9570t/ly6W/jzwi/R01Svq7Zo3ty/b3l/527Wr7NGSOO5XSKbV5v7yM6+FM6+NMsTqTyvrNV/T74r5IWS03Ts9gkF728vICsrMBtyJfy2+/AffuWe96e8YMYPTowp4rExOl6WvWlD6PHQsMGgTk5wMeHlJZbq70w3FzA65ckRIqIaTh9++zi2+5yLkzkvsCZKXNX4nsuV5JafUkdzyVcVpKzlvljeM74rSUCpNfJjdKU6OG+Wd3d+lVkqJdcru6FiY21oaXNi1QmAARFVUZByVnPi1VmqqIRQ0HIqV24mdNVdW3kh4sqgI8LUWkNo7eGSvtgmKlqA7rqYZEyhkp6ToyJ8XkhkjtytpxOdt/pnJzpjtpHDW92pR1WsqWcQF5Ly4v7bSUCpOV8mJyQ0SOVdmnHJTy+AVrlNSJn5LqBTCPR467pcozndLqrjyc6ZRfJWJyQ+QovH1ToqZ1ISKnxOSGyNl4eZU+vG/f8s3v4YdLH96mTfnmN2iQ9fLGjQvf9+tXvmmLqlevfPE4mnH5xnUo2qlo8dhKeSgwgMIbABo0KH28+vUL37dvX2aIAACXIrv3WrWkv489Ztu0RtbW0Rbh4ZZlxq4qAKBDh7LnYe3O05CQwi4zyqtoFxtDh1ofx1i3xvoCgD59Sp5nSfMpOq/SuLoWvi/6uy4aa0mM9Vl0HgAwZAjQrp31aerWlf5GRJiX/+8RRxZs+T0qFO+WshfPYZNczp0Ddu8G2raVbucfO1YqNxiARYuACRNKnnbCBGDjRvOykSOB27eB0FDr0/TvD3zyCZCXB5w8Cdy8KXVRcOwYsG8f8NprQNOmwIABwH//Czz7LHD2rDRuUW3bAm+9JZVHR0tlSUlSFwZDhwKffw4880zJsR84ANy4ATRpAnz2mRQ3UHJLUWqqNE8fH+n3umULsGBByfM/dQp46CEgLEyqi0ceAdLTpTsYfXyAJ58EpkyRlg8A//d/0oF2yBAgObkwtqLOnwe++UbqZuG334C5c6XykyeB48eBp54CNm0Cnnii5LgAab5ffAHUrg307FlYbm3dhwyRxnF1BWJjgbt3Cw9qW7ZIrx49gEOHpO4kjDZtspxX0XW0xalT0t/p06WD9fPPFw6bNKnw/YgRwIYNUtI2cqR0sH3kEWD+/MJxmjYtfH/ihFRnQ4ZI67x2LTBtGvD114Xj7N8PXLggLfP996WygweBRx+Vtt+iCdrHH0tJS9euwJEj0sH9/Hlg+HBpeO3a0rw9PaW6/OMP4N//Blq0AObNs5zP4MHAt99K29yIEdKwYcOAlSuB2bMLxw8NleY9ejSQkWGebHp7S/MQojDWhAQgKqpwfQDp93/wYGF9urkB338vxZiTI/3+/PyAjz6Sfp/37knf9+DB0nfy9deW+4hRo4DDhwv7RmvUSIq7tN+jwmmEUPdROisrCzqdDpmZmfBzZB8uX34J/OUv5mXl7YtC3VVf/Xz3XeF/OuX9bn/4QdoJF5/W2nZUfN7GcZ56SjroF7d5MzBuXOG0tWtLCYo9cdpq2DDzA489yyqtL45z56RkCZD6cTK2UtjSl4kaGNczIUHabmrUAO7ckcrKs97G+UydCqxbZ18MRZW0bbZrB5w+bfs8GzQALl8uXzxKVbSeUlKAhg3li6Usxli7dAGOHpU3FlTs+M3TUkRERFVBzQm3wjC5IXIUZ7mgWA0X/KphHRyJ3fUTmWFyQ6R2PHARKQN/i1WGyQ2RWnDHWf2o9TQHt2WqICY3FcUfIRHJTen7ofLGp9akjaoMk5uK4o+QqOop/WBOZA2PF1WGyY29uJFScc5yQTGpD/dHRGaY3FQUD0qkFEraFqvyYMsDe8VV9rbD78i5KGlfYicmN0TkfFSw83UI1gORVUxuKoo7F1I6bqNEVM0wuSFSC1seRggUPpvI+NBGZ6TTFb4vmrwZH/QYHFy18citd2/pr9wPFS2JrQl2jRrS3x49Ki8WORnXT+lKes6cE+GDMyuK55LJyPiU3qIHXlsZn+Jb/CnSp05JD8lcvLjkaU+cAPbsAV5+2frw4tvohg1SIjRmTPnjtFXRg9mqVWU/edya06dLfrKyXi89M8vb2/zp11u3Ss9IGj++/MtzZps2SQ+TND5E1VmdOiV9h0Uf6OnsLlyQnkM4dSoQECB3NKU7dw6IiQFeeknuSCqMD8601xdfSA8q1GgKDx58cCalp0tP3i2epNgiIwPQaqXpiwsPl57aC5R/u/nnP6UnBdszrb2KPjizIst88klgx46Kz0dtjPuX+Hjzp4TbO59p04APPrBv2qJKenBm+/bAL7+UPz6q1ipy/GbLDZEj1apl/7TOfJqosvB6ISKyA6+5sZfxPxTufInI2XE/RirD5KaiuFOgqsBTMkRENmNyQ0RERKrC5IZI7eRoXWRLU9VgyzGRVUxuKqphQ+vlxjtefH3Ny1u2rNx4iNSESRIR2YF3S1VUo0bSLZT+/ublFy4ABw5It7J++SXQurXUh8DIkcDOncBDD8kRLRERkeoxuXGEgQMty4KCCvsWMf7t0kX6O2pU1cRF6tGggdwRyIOnXUrnqJYt1jOpDJMbImfw/vtAQYHUyykREZWKyY29eC0AVSW9vrCnXmfAlgAqitsDVTFeUFxR/NGS0jERJ6Jqxq7k5t69e7h7967pc0pKClavXo3Y2FiHBUZERERkD7uSm6FDh2Lz5s0AgIyMDISGhmLlypUYOnQo1q1b59AAiaiC2LpIRNWMXcnNiRMn0KNHDwDAl19+icDAQKSkpGDz5s147733HBogERE5OZ4apSpmV3Jz9+5d+P6vc7rY2FgMHz4cLi4u6Nq1K1JSUhwaIBE5Id6iXDUcVT+sZ1IZu5KbZs2aYefOnbh8+TK+//57REREAADS0tLg5+fn0ACJiIiIysOu5ObVV1/FnDlz0LhxY4SGhiIsLAyA1IrTsWNHhwZIREQlcJbTPWwZoipmVz83Tz75JLp3747r16+jQ4cOpvK+ffviiSeecFhwREREROVldyd+er0eer3erOyRRx6pcEBOw1n+YyIiIqpmbE5uhg8fbvNMY2Ji7ArGKbG5lZTOmbdR/hNBRHaw+ZobnU5nevn5+WHfvn34+eefTcOPHz+Offv2QafTVUqgRERERLawueVm48aNpvfz58/HU089hQ8//BCurq4AgPz8fEybNo13SxERb1F2NqxnUhm77pb6+OOPMWfOHFNiAwCurq6YNWsWPv74Y4cFR0QOwFM7JDcmT1TF7Epu8vLycP78eYvy8+fPo6CgoMJBERGRijDBpipm191SEyZMwMSJE/Gf//wHXbt2BQAcOXIEy5Ytw4QJExwaIBE5IR7MqoajWkT4fZHK2NVy8/bbb2PBggV455138Oijj+LRRx/FO++8g3nz5mHFihU2z2fdunVo3749/Pz84Ofnh7CwMHz33Xem4UIIREVFwWAwwMvLC7169cLZs2ftCZmISH2YlBBZVe7kJi8vD//85z/x7LPP4urVq8jIyEBGRgauXr2KefPmmV2HU5YGDRpg2bJl+Pnnn/Hzzz+jT58+GDp0qCmBWb58OVatWoU1a9bg2LFj0Ov16N+/P7Kzs8sbtuNxp0JERKRI5U5u3Nzc8PzzzyMnJwcATK0u9oiMjMSgQYPQokULtGjRAv/3f/+HGjVq4MiRIxBCYPXq1Vi8eDGGDx+OkJAQbNq0CXfv3sXWrVvtWl6l4IVypHTcRqks3EZIZew6LRUaGoqTJ086NJD8/Hxs374dd+7cQVhYGJKTk5Gammp6KCcAaLVa9OzZE4mJiQ5dNhEpFA+66sDvkaqYXRcUT5s2DbNnz8aVK1fQqVMn+Pj4mA1v3769zfNKSkpCWFgY7t+/jxo1auCrr75CmzZtTAlMYGCg2fiBgYFISUkpcX45OTmmViUAyMrKsjkWIiIicn52JTcjR44EALz44oumMo1GAyEENBoN8vPzbZ5Xy5YtcerUKWRkZGDHjh0YN24cEhISzOZblHEZJVm6dClee+01m5dPRERE6mJXcpOcnOywADw8PNCsWTMAQOfOnXHs2DG8++67mD9/PgAgNTUV9erVM42flpZm0ZpT1MKFCzFr1izT56ysLAQFBTksXiIiIlI2u5KbRo0aOToOEyEEcnJyEBwcDL1ej7i4OHTs2BEAkJubi4SEBLz11lslTq/VaqHVaistPiKyAa+xICIZ2ZXcGJ07dw6XLl1Cbm6uWfnjjz9u0/SLFi3CwIEDERQUhOzsbGzfvh3x8fHYs2cPNBoNZs6ciejoaDRv3hzNmzdHdHQ0vL29MXr06IqETVS9eHvLHYH9atWSOwJlc5Z/5GrXljsCqmbsSm5+//13PPHEE0hKSjJdawMUXh9j6zU3f/75J8aOHYvr169Dp9Ohffv22LNnD/r37w8AmDdvHu7du4dp06YhPT0doaGhiI2Nha+vrz1hOxb7uSFnERkJDBsGPPJI1S3TUb+PN98ELl4EJk50zPzUYvFi4LffgNBQx8zPnpa2VauAvXuB998HnnsOKHINpsmuXcDKlcCGDRWPkagcNEKUfy8UGRkJV1dXbNiwAU2aNMHRo0dx8+ZNzJ49G2+//TZ69OhRGbHaJSsrCzqdDpmZmY59YvnWrcCYMUC/fkBcnOPmS6QGQ4dKBzaA/wgomTGpeeklYPVqWUMhKq4ix2+7Wm4OHz6M/fv3o06dOnBxcYGLiwu6d++OpUuX4sUXX3R4HzhEREREtrKrE7/8/HzUqFEDAFC7dm1cu3YNgHSh8YULFxwXHREREVE52dVyExISgtOnT6NJkyYIDQ3F8uXL4eHhgfXr16NJkyaOjpGIiIjIZnYlN6+88gru3LkDAHjzzTcxZMgQ9OjRAwEBAfjss88cGiARERFRediV3AwYMMD0vkmTJjh37hxu3bqFWrVqldp7MBERKRD326Qydl1zExcXh7t375qV+fv7M7EhIiIi2dnVcjNixAjk5OSgU6dO6NmzJ3r16oVu3bqZLjKuFnh7KxERkSLZ1XKTnp6O+Ph4PP744zh58iT+8pe/wN/fH127dsWCBQscHaOysbWKiIhIUexKblxdXREWFoYFCxZgz549SExMxOjRo3H8+HGsWLHC0TESERER2cyu01Lnz59HQkIC4uPjkZCQgPz8fHTv3h0rV65Ez549HR0jERERkc3sSm7atm2LOnXqYObMmfjb3/6Gtm3bOjouIiIiIrvYdVrqxRdfRP369REVFYWJEydi/vz5+O6773D79m1Hx0dERJWN1w6SytiV3KxevRonTpzAn3/+iVdeeQX5+fl49dVXUbt2bXTt2tXRMRIRERHZzK7kxqigoAB5eXnIzc1FTk4OHjx4gD/++MNBoRERERGVn13JzUsvvYQOHTqgbt26mDJlCq5du4a//vWv+OWXX5CamuroGJWJ/dwQEREpkl0XFF+9ehWTJ09Gr169EBIS4uiYnAvPVRMRESmKXcnNl19+6eg4iIiIiBzC7mtu/vnPf6Jbt24wGAxISUkBIF1o/PXXXzssOCIiqgJsgSaVsSu5WbduHWbNmoVBgwYhIyMD+fn5AICaNWti9erVjoyPiJxR7dpyR0BE1Zhdyc3777+PDRs2YPHixXB1dTWVd+7cGUlJSQ4Ljoic1LJlwIABwOefyx0JEVVDdl1zk5ycjI4dO1qUa7Va3Llzp8JBEZGTq1MH2LNH7iiIqJqyq+UmODgYp06dsij/7rvv0Lp164rGRERERGQ3u1pu5s6dixdeeAH379+HEAJHjx7Ftm3bEB0djY8++sjRMSoT+7khIiJSJLuSmwkTJiAvLw/z5s3D3bt3MXr0aNSvXx/vv/8+evTo4egYlY13GRARESmK3beCT548GSkpKUhLS0NqaiqOHj2KkydPolmzZo6Mj4iIKhv/SSOVKVdyk5GRgTFjxqBOnTowGAx477334O/vjw8++ADNmjXDkSNH8PHHH1dWrERERERlKtdpqUWLFuHgwYMYN24c9uzZg5dffhl79uzB/fv38e2336Jnz56VFScRERGRTcqV3OzevRsbN25Ev379MG3aNDRr1gwtWrRgx31ERESkGOU6LXXt2jW0adMGANCkSRN4enpi0qRJlRIYERERkT3KldwUFBTA3d3d9NnV1RU+Pj4OD4qIiIjIXuU6LSWEwPjx46HVagEA9+/fx9SpUy0SnJiYGMdFqFTs54aI1IJ3S5HKlCu5GTdunNnnZ555xqHBOCXuFIiIiBSlXMnNxo0bKysOIiIiIoewuxM/IiIiIiVickNERESqwuSGiIiIVIXJDREREakKkxsiouqOd32SyjC5sRf7uSEiIlIkJjcVxf94iIiIFIXJDREREakKkxsiIiJSFSY3REREpCpMboiIqjteO0gqw+SGiIiIVIXJDREREakKkxsiIiJSFSY39mInfkRERIoka3KzdOlSdOnSBb6+vqhbty6GDRuGCxcumI0jhEBUVBQMBgO8vLzQq1cvnD17VqaIreCFeERERIoia3KTkJCAF154AUeOHEFcXBzy8vIQERGBO3fumMZZvnw5Vq1ahTVr1uDYsWPQ6/Xo378/srOzZYyciEgFunaV/j77rLxxEDmYm5wL37Nnj9nnjRs3om7dujh+/DgeffRRCCGwevVqLF68GMOHDwcAbNq0CYGBgdi6dSumTJkiR9hEROpw6BCQng7Uri13JEQOpahrbjIzMwEA/v7+AIDk5GSkpqYiIiLCNI5Wq0XPnj2RmJgoS4xERKrh6srEhlRJ1pabooQQmDVrFrp3746QkBAAQGpqKgAgMDDQbNzAwECkpKRYnU9OTg5ycnJMn7OysiopYiIiIlIixbTcTJ8+HadPn8a2bdsshmmKXbQrhLAoM1q6dCl0Op3pFRQUVCnxEhERkTIpIrmZMWMGdu3ahQMHDqBBgwamcr1eD6CwBccoLS3NojXHaOHChcjMzDS9Ll++XHmBExERkeLImtwIITB9+nTExMRg//79CA4ONhseHBwMvV6PuLg4U1lubi4SEhIQHh5udZ5arRZ+fn5mr0oKvnLmS0RERBUi6zU3L7zwArZu3Yqvv/4avr6+phYanU4HLy8vaDQazJw5E9HR0WjevDmaN2+O6OhoeHt7Y/To0XKGXoj93BARESmKrMnNunXrAAC9evUyK9+4cSPGjx8PAJg3bx7u3buHadOmIT09HaGhoYiNjYWvr28VR0tERETOQNbkRthwakej0SAqKgpRUVGVHxARERE5PUVcUExERETkKExuiIiISFWY3BAREZGqMLkhIiIiVWFyYy/2c0NERKRITG4qiv3cEBERKQqTGyIiIlIVJjdERESkKkxuiIiISFWY3BAREZGqMLkhIiIiVWFyQ0RERKrC5MZe7OeGiIhIkZjcVBT7uSEiIlIUJjdERESkKkxuiIiISFWY3BAREZGqMLkhIiIiVWFyQ0RERKrC5IaIiIhUhcmNvdjPDRERkSIxuako9nNDRESkKExuiIiISFWY3BAREZGqMLkhIiIiVWFyQ0RERKrC5IaIiIhUhckNERERqQqTG3uxnxsiIiJFYnJTUeznhoiISFGY3BAREZGqMLkhIiIiVWFyQ0RERKrC5IaIiIhUhckNERERqQqTGyIiIlIVJjf2Yj83REREisTkpqLYzw0REZGiMLkhIiIiVWFyQ0RERKrC5IaIiIhUhckNERERqQqTGyIiIlIVJjdERESkKkxu7MV+boiIiBSJyU1FsZ8bIiIiRWFyQ0RERKrC5IaIiIhURdbk5uDBg4iMjITBYIBGo8HOnTvNhgshEBUVBYPBAC8vL/Tq1Qtnz56VJ1giIiJyCrImN3fu3EGHDh2wZs0aq8OXL1+OVatWYc2aNTh27Bj0ej369++P7OzsKo6UiIiInIWbnAsfOHAgBg4caHWYEAKrV6/G4sWLMXz4cADApk2bEBgYiK1bt2LKlClVGSoRERE5CcVec5OcnIzU1FRERESYyrRaLXr27InExEQZIyMiIiIlk7XlpjSpqakAgMDAQLPywMBApKSklDhdTk4OcnJyTJ+zsrIqJ0AiIiJSJMW23BhpivUjI4SwKCtq6dKl0Ol0pldQUFDlBMZO/IiIiBRJscmNXq8HUNiCY5SWlmbRmlPUwoULkZmZaXpdvny5UuNkJ35ERETKotjkJjg4GHq9HnFxcaay3NxcJCQkIDw8vMTptFot/Pz8zF5ERERUfch6zc3t27fxn//8x/Q5OTkZp06dgr+/Pxo2bIiZM2ciOjoazZs3R/PmzREdHQ1vb2+MHj1axqiJiIhIyWRNbn7++Wf07t3b9HnWrFkAgHHjxuGTTz7BvHnzcO/ePUybNg3p6ekIDQ1FbGwsfH195QqZiIiIFE7W5KZXr14QpVyYq9FoEBUVhaioqKoLioiIiJyaYq+5ISIiIrIHkxsiIiJSFSY39mI/N0RERIrE5Kai2M8NERGRojC5ISIiIlVhckNERESqwuSGiIiIVIXJDREREakKkxsiIiJSFSY3REREpCpMbuzFfm6IiIgUiclNRbGfGyIiIkVhckNERESqwuSGiIiIVIXJDREREakKkxsiIiJSFSY3REREpCpMboiIiEhVmNzYi/3cEBERKRKTm4piPzdERESKwuSGiIiIVIXJDREREakKkxsiIiJSFSY3REREpCpMboiIiEhVmNwQERGRqjC5sRf7uSEiIlIkJjcVxX5uiIiIFIXJDREREakKkxsiIiJSFSY3REREpCpMboiIiEhVmNwQERGRqjC5ISIiIlVhcmMv9nNDRESkSExuKor93BARESkKkxsiIiJSFSY3REREpCpMboiIiEhVmNwQERGRqjC5ISIiIlVhckNERESqwuTGXuznhoiISJGY3FQU+7khIiJSFCY3REREpCpMboiIiEhVmNwQERGRqjC5ISIiIlVxiuRm7dq1CA4OhqenJzp16oQffvhB7pCIiIhIoRSf3Hz22WeYOXMmFi9ejJMnT6JHjx4YOHAgLl26JG9gmZnyLp+IiIisUnxys2rVKjz33HOYNGkSWrdujdWrVyMoKAjr1q2TN7DvvpN3+URERGSVopOb3NxcHD9+HBEREWblERERSExMlCmq/xk+HKhZExg6VN44iIiIyIyb3AGU5saNG8jPz0dgYKBZeWBgIFJTU61Ok5OTg5ycHNPnrKysygluzhzpRURERIqi6JYbI02xXoCFEBZlRkuXLoVOpzO9goKCqiJEIiIiUghFJze1a9eGq6urRStNWlqaRWuO0cKFC5GZmWl6Xb58uSpCJSIiIoVQdHLj4eGBTp06IS4uzqw8Li4O4eHhVqfRarXw8/MzexEREVH1oehrbgBg1qxZGDt2LDp37oywsDCsX78ely5dwtSpU+UOjYiIiBRI8cnNyJEjcfPmTbz++uu4fv06QkJC8O2336JRo0Zyh0ZEREQKpBFCCLmDqExZWVnQ6XTIzMzkKSoiIiInUZHjt6KvuSEiIiIqLyY3REREpCpMboiIiEhVmNwQERGRqjC5ISIiIlVhckNERESqwuSGiIiIVIXJDREREamK4nsorihjH4VZWVkyR0JERES2Mh637elrWPXJTXZ2NgAgKChI5kiIiIiovLKzs6HT6co1jeofv1BQUIBr167B19cXGo3GofPOyspCUFAQLl++zEc7lIF1ZTvWle1YV7ZjXdmOdWW7yqwrIQSys7NhMBjg4lK+q2hU33Lj4uKCBg0aVOoy/Pz8+AOwEevKdqwr27GubMe6sh3rynaVVVflbbEx4gXFREREpCpMboiIiEhVmNxUgFarxZIlS6DVauUORfFYV7ZjXdmOdWU71pXtWFe2U2pdqf6CYiIiIqpe2HJDREREqsLkhoiIiFSFyQ0RERGpCpMbIiIiUhUmN3Zau3YtgoOD4enpiU6dOuGHH36QOySHioqKgkajMXvp9XrTcCEEoqKiYDAY4OXlhV69euHs2bNm88jJycGMGTNQu3Zt+Pj44PHHH8eVK1fMxklPT8fYsWOh0+mg0+kwduxYZGRkmI1z6dIlREZGwsfHB7Vr18aLL76I3NzcSlv3shw8eBCRkZEwGAzQaDTYuXOn2XCl1U1SUhJ69uwJLy8v1K9fH6+//rpdz2qxR1l1NX78eIvtrGvXrmbjVJe6Wrp0Kbp06QJfX1/UrVsXw4YNw4ULF8zG4bYlsaWuuG1J1q1bh/bt25s62QsLC8N3331nGq7abUpQuW3fvl24u7uLDRs2iHPnzomXXnpJ+Pj4iJSUFLlDc5glS5aItm3biuvXr5teaWlppuHLli0Tvr6+YseOHSIpKUmMHDlS1KtXT2RlZZnGmTp1qqhfv76Ii4sTJ06cEL179xYdOnQQeXl5pnEee+wxERISIhITE0ViYqIICQkRQ4YMMQ3Py8sTISEhonfv3uLEiRMiLi5OGAwGMX369KqpCCu+/fZbsXjxYrFjxw4BQHz11Vdmw5VUN5mZmSIwMFCMGjVKJCUliR07dghfX1/x9ttvV14FFVFWXY0bN0489thjZtvZzZs3zcapLnU1YMAAsXHjRnHmzBlx6tQpMXjwYNGwYUNx+/Zt0zjctiS21BW3LcmuXbvE7t27xYULF8SFCxfEokWLhLu7uzhz5owQQr3bFJMbOzzyyCNi6tSpZmWtWrUSCxYskCkix1uyZIno0KGD1WEFBQVCr9eLZcuWmcru378vdDqd+PDDD4UQQmRkZAh3d3exfft20zhXr14VLi4uYs+ePUIIIc6dOycAiCNHjpjGOXz4sAAg/v3vfwshpIOji4uLuHr1qmmcbdu2Ca1WKzIzMx22vvYqfsBWWt2sXbtW6HQ6cf/+fdM4S5cuFQaDQRQUFDiwJspWUnIzdOjQEqeprnUlhBBpaWkCgEhISBBCcNsqTfG6EoLbVmlq1aol/vGPf6h6m+JpqXLKzc3F8ePHERERYVYeERGBxMREmaKqHBcvXoTBYEBwcDBGjRqF33//HQCQnJyM1NRUszrQarXo2bOnqQ6OHz+OBw8emI1jMBgQEhJiGufw4cPQ6XQIDQ01jdO1a1fodDqzcUJCQmAwGEzjDBgwADk5OTh+/HjlrbydlFY3hw8fRs+ePc062BowYACuXbuGP/74w/EVYIf4+HjUrVsXLVq0wOTJk5GWlmYaVp3rKjMzEwDg7+8PgNtWaYrXlRG3LXP5+fnYvn077ty5g7CwMFVvU0xuyunGjRvIz89HYGCgWXlgYCBSU1NlisrxQkNDsXnzZnz//ffYsGEDUlNTER4ejps3b5rWs7Q6SE1NhYeHB2rVqlXqOHXr1rVYdt26dc3GKb6cWrVqwcPDQ5H1rbS6sTaO8bMS6m/gwIHYsmUL9u/fj5UrV+LYsWPo06cPcnJyAFTfuhJCYNasWejevTtCQkLMYuC2Zc5aXQHctopKSkpCjRo1oNVqMXXqVHz11Vdo06aNqrcp1T8VvLJoNBqzz0IIizJnNnDgQNP7du3aISwsDE2bNsWmTZtMF+XZUwfFx7E2vj3jKI2S6sZaLCVNW9VGjhxpeh8SEoLOnTujUaNG2L17N4YPH17idGqvq+nTp+P06dM4dOiQxTBuW+ZKqituW4VatmyJU6dOISMjAzt27MC4ceOQkJBQamzOvk2x5aacateuDVdXV4ssMi0tzSLjVBMfHx+0a9cOFy9eNN01VVod6PV65ObmIj09vdRx/vzzT4tl/fe//zUbp/hy0tPT8eDBA0XWt9Lqxto4xqZ5JdZfvXr10KhRI1y8eBFA9ayrGTNmYNeuXThw4AAaNGhgKue2ZamkurKmOm9bHh4eaNasGTp37oylS5eiQ4cOePfdd1W9TTG5KScPDw906tQJcXFxZuVxcXEIDw+XKarKl5OTg/Pnz6NevXoIDg6GXq83q4Pc3FwkJCSY6qBTp05wd3c3G+f69es4c+aMaZywsDBkZmbi6NGjpnF++uknZGZmmo1z5swZXL9+3TRObGwstFotOnXqVKnrbA+l1U1YWBgOHjxodrtlbGwsDAYDGjdu7PgKqKCbN2/i8uXLqFevHoDqVVdCCEyfPh0xMTHYv38/goODzYZz2ypUVl1ZU523reKEEMjJyVH3NlWuy49JCFF4K/hHH30kzp07J2bOnCl8fHzEH3/8IXdoDjN79mwRHx8vfv/9d3HkyBExZMgQ4evra1rHZcuWCZ1OJ2JiYkRSUpJ4+umnrd4+2KBBA7F3715x4sQJ0adPH6u3D7Zv314cPnxYHD58WLRr187q7YN9+/YVJ06cEHv37hUNGjSQ9Vbw7OxscfLkSXHy5EkBQKxatUqcPHnS1BWAkuomIyNDBAYGiqefflokJSWJmJgY4efnV2W3N5dWV9nZ2WL27NkiMTFRJCcniwMHDoiwsDBRv379allXzz//vNDpdCI+Pt7s9uW7d++axuG2JSmrrrhtFVq4cKE4ePCgSE5OFqdPnxaLFi0SLi4uIjY2Vgih3m2KyY2dPvjgA9GoUSPh4eEhHn74YbNbENXA2NeBu7u7MBgMYvjw4eLs2bOm4QUFBWLJkiVCr9cLrVYrHn30UZGUlGQ2j3v37onp06cLf39/4eXlJYYMGSIuXbpkNs7NmzfFmDFjhK+vr/D19RVjxowR6enpZuOkpKSIwYMHCy8vL+Hv7y+mT59udqtgVTtw4IAAYPEaN26cEEJ5dXP69GnRo0cPodVqhV6vF1FRUVV2+2lpdXX37l0REREh6tSpI9zd3UXDhg3FuHHjLOqhutSVtXoCIDZu3Ggah9uWpKy64rZVaOLEiaZjVZ06dUTfvn1NiY0Q6t2mNEJUUfebRERERFWA19wQERGRqjC5ISIiIlVhckNERESqwuSGiIiIVIXJDREREakKkxsiIiJSFSY3REREpCpMbojIKcXHx0Oj0SAjI6NC82ncuDFWr17tkJiISBmY3BCRUwoPD8f169eh0+nkDoWIFMZN7gCIiOzh4eFheqoxEVFRbLkhokojhMDy5cvRpEkTeHl5oUOHDvjyyy8BFJ5W2r17Nzp06ABPT0+EhoYiKSnJNH1KSgoiIyNRq1Yt+Pj4oG3btvj222/Npi96WmrHjh1o27YttFotGjdujJUrV5rFk5aWhsjISHh5eSE4OBhbtmyxiDkzMxN//etfUbduXfj5+aFPnz745ZdfKqF2iKiysOWGiCrNK6+8gpiYGKxbtw7NmzfHwYMH8cwzz6BOnTqmcebOnYt3330Xer0eixYtwuOPP45ff/0V7u7ueOGFF5Cbm4uDBw/Cx8cH586dQ40aNawu6/jx43jqqacQFRWFkSNHIjExEdOmTUNAQADGjx8PABg/fjwuX76M/fv3w8PDAy+++CLS0tJM8xBCYPDgwfD398e3334LnU6Hv//97+jbty9+/fVX+Pv7V2p9EZGDlPtRm0RENrh9+7bw9PQUiYmJZuXPPfecePrpp01PDN++fbtp2M2bN4WXl5f47LPPhBBCtGvXTkRFRVmdv3F645OHR48eLfr37282zty5c0WbNm2EEEJcuHBBABBHjhwxDT9//rwAIN555x0hhBD79u0Tfn5+Fk8qbtq0qfj73/9e/kogIlmw5YaIKsW5c+dw//599O/f36w8NzcXHTt2NH0OCwszvff390fLli1x/vx5AMCLL76I559/HrGxsejXrx9GjBiB9u3bW13e+fPnMXToULOybt26YfXq1cjPz8f58+fh5uaGzp07m4a3atUKNWvWNH0+fvw4bt++jYCAALP53Lt3D7/99lv5KoCIZMPkhogqRUFBAQBg9+7dqF+/vtkwrVZbarKg0WgAAJMmTcKAAQOwe/duxMbGYunSpVi5ciVmzJhhMY0QwjRd0bLi74uPUzzmevXqIT4+3mJY0SSIiJSNFxQTUaVo06YNtFotLl26hGbNmpm9goKCTOMdOXLE9D49PR2//vorWrVqZSoLCgrC1KlTERMTg9mzZ2PDhg0lLu/QoUNmZYmJiWjRogVcXV3RunVr5OXl4eeffzYNv3DhgtkFyQ8//DBSU1Ph5uZmEXPt2rUrWiVEVEXYckNElcLX1xdz5szByy+/jIKCAnTv3h1ZWVlITExEjRo10KhRIwDA66+/joCAAAQGBmLx4sWoXbs2hg0bBgCYOXMmBg4ciBYtWiA9PR379+9H69atrS5v9uzZ6NKlC9544w2MHDkShw8fxpo1a7B27VoAQMuWLfHYY49h8uTJWL9+Pdzc3DBz5kx4eXmZ5tGvXz+EhYVh2LBheOutt9CyZUtcu3YN3377LYYNG2Z2SouIFEzma36ISMUKCgrEu+++K1q2bCnc3d1FnTp1xIABA0RCQoLpguB//etfom3btsLDw0N06dJFnDp1yjT99OnTRdOmTYVWqxV16tQRY8eOFTdu3BBCWF5QLIQQX375pWjTpo1wd3cXDRs2FCtWrDCL5/r162Lw4MFCq9WKhg0bis2bN4tGjRqZLigWQoisrCwxY8YMYTAYhLu7uwgKChJjxowRly5dqtS6IiLH0QhR5KQ0EVEViY+PR+/evZGens7rWYjIoXjNDREREakKkxsiIiJSFZ6WIiIiIlVhyw0RERGpCpMbIiIiUhUmN0RERKQqTG6IiIhIVZjcEBERkaowuSEiIiJVYXJDREREqsLkhoiIiFSFyQ0RERGpyv8DiA+KHBSjD4UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(rewards_total,'r')\n",
    "\n",
    "plt.title('Evolution of total reward')\n",
    "plt.ylabel('Rewards')\n",
    "plt.xlabel('epsiode')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the learning is successful, we can visualize the learned robot controller. Remember we learn this just from interacting with the environment instead of peeking into the dynamics model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = KTAICrawlerEnv(render=True, horizon=500)\n",
    "prev_state = env.reset()\n",
    "ret = 0.\n",
    "done = False\n",
    "\n",
    "while not done:\n",
    "    action = np.argmax(q_vals[prev_state])\n",
    "    state, reward, done, info = env.step(action)\n",
    "    ret += reward\n",
    "    prev_state = state\n",
    "    time.sleep(0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can close the visualization GUI with the following method \n",
    "env.close_gui()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Create Assignment",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}