jupyter/CrawlingFRED.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Crawling Fred Big Data in a Rapid and Methodical Way"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Author: Yadong Zhang\n",
    "\n",
    "Date: 9-28-2018\n",
    "\n",
    "E-main: ydup@foxmail.com\n",
    "\n",
    "## What is Fred?\n",
    "\n",
    "<img src=\"../img/FRED.png\"  alt=\"fred\" />\n",
    "\n",
    "> [FRED](https://fred.stlouisfed.org/): Download, graph, and track 527,000 US and international time series from 87 sources.\n",
    " \n",
    "The data can be obtained by Tag, Category, Release, Source, Release Calendar. And there are many tools / [APIs](https://research.stlouisfed.org/docs/api/fred/) for you to crawl the data, such as [fredapi](https://github.com/mortada/fredapi) of python. And in this notebook, I will show you how to use fredapi with python to download the fred data in a rapid and methodical way. Let's go : )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Basic Test for the FredAPI\n",
    "Firstly, let's try to download a kind of series data with the help of [FredAPI](https://github.com/mortada/fredapi) :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install fredapi with Pip and Register to Get a API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fredapi import Fred\n",
    "fred = Fred(api_key='APIkey')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download Data with its Series ID"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x111f08350>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD8CAYAAABjAo9vAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJztnXd8FHX+/1+f3U0hCQmQBAg1gBTpYAwiTQGVcjYs2Bu2+53n6el5YDlsnHy9O3s7LJy9l9MDUUAUpAfpvYVeQgslpO1+fn/MzO7s7Mxsm5md2X0/Hw8e7M7Ozr6z5TXveX/ehXHOQRAEQSQPrkQbQBAEQRgLCTtBEESSQcJOEASRZJCwEwRBJBkk7ARBEEkGCTtBEESSQcJOEASRZJCwEwRBJBkk7ARBEEmGJxEvWlBQwIuLixPx0gRBEI5l2bJlhzjnheH2S4iwFxcXo6ysLBEvTRAE4VgYYzsi2Y9CMQRBEElG3MLOGMtkjC1hjK1kjK1ljD1hhGEEQRBEbBgRiqkBMJRzfpIxlgbgV8bY95zzRQYcmyAIgoiSuIWdC31/T4p308R/1AuYIAgiQRgSY2eMuRljKwAcBDCTc77YiOMSBEEQ0WOIsHPOvZzz3gBaAShljHVX7sMYu5MxVsYYK6uoqDDiZQmCIAgVDM2K4ZwfAzAHwAiVx6Zwzks45yWFhWHTMAmCIIgYMSIrppAx1ki83QDABQA2xHtcgiCSkzqvD16f+jIc5xyfle1CdZ3XYquSCyM89iIAcxhjqwAshRBj/58BxyUIIgnp+Mj36PDwdKzYdSzksQVbD+OhL1Zh0rT1CbAseYhb2DnnqzjnfTjnPTnn3TnnTxphGEEQyc1lr84P2fbfFXsAANNW77PanKSCKk8JgrAMZYjl9Z+3+m//tvMoPivbDQA4cqrWUruSDRJ2giAsY8vBk0H3/29GYDlux+FTVpuTtJCwEwRhGSdr6iPe91QU+xLBkLAThAOprffh7V+3o97rS7Qpumw5eBLF46f5F0o3Kzx2OVyRKNNt4g9mmpbUkLAThAN5c942PPW/dXhlzpZEm6LLLVOXAACueH0BAKC6VjuN8cDxmpBtxeOnYc2eSnOMS2JI2AnCgRyrEhYXX5i1OcGW6HOiWginSHnrtSpXGLX1wraCnHTVY8jj8ERkkLAThAPRqO+xHRmegMTsPFyFf/ywMWSf06IX3yRbXdh7tsqLy4bKqjr8tvNoVPF9p0PCTiSMDxfvwCNfr4bPKSplI+pknu/ol+Yl0BJ9MtICEnPFGwv8t8f0bem/Xe8T/haXi6keI95lhN92HcWY1xZg04ET8R3IQZCwEwnjka/X4MPFO9H+4en4dfOhRJvjKKTwBQCs3Xs8gZbok+Fx+29XnAjE0J8Z08N/u148sf+4dr/qMd74Zavq9kiZu0loOnj4ZOrkxpOwE7ZAqjgkIkMtVm0Hdh+tQvH4aViy/QgAwKdMdRHJ8Lgxpo/gtUtXH9NWCdWmt5xbbKhNU+eXAwA27LPvCdBoSNgJW/D5st2JNsFR1HntGb6as1Hwjt9bWA4AOFZVp7lvv/ZNAABfiJ/9LQPaAQBGdm+OJy/tZrhtWqGeZISEnSAciNdnT499xU4hX325+L9ea4Dyw1UAApk9xflZAIDmeZkYe3Zrw23La5Bm+DHtCgk7QTiQrkW5Qfdr6u3R5rZbC8GuC7o2C7uv0n+WYu1uFwuKzcfKmY/NwENfrIz7OE6EhJ2wDVW1qZOOFi/KRCLJU040Xpk4A8DoHkUh+zw/tlfQPkBwj3aPS5Clz+7qDwBolpsRky2n67z+pmJAaAOyZIaEnbANygZRhDbKVgIPfbkK2yoS//5Ji6WSZqvloPdrlw8AaJQVyFuXt0eQBL+0XRMM7lSIorwGhtj2dAr1eCdhJwgHUqdw2XccrsKlKv3NrUayijEWdF+OVLTkkXnsk7/f4A/FyLe7mHZmTSysVBnukYyQsBMJo3vL4Dixi6VO1kKsvLugHIOfnYPTtV40SHPj3qFn+B+TyvcTyVIxzVH6JNVEOU0UdmWWihQ2cbsD292MaY7RiwU7nPysgISdSBhr9gTnFedr9ApJVbw+jhlr9oNzDs45ZqzZh4nfrsXOI1X4z4JynK7z4qLuzRNtZhCzNxwMuq/mbGelCQujHoWwrxfzzNNcAVlyuVhM7RNmrjsQ/ZOSCBJ2wjZU6XT+S0Vuemcx7v5gGV6avQW/bKrA3R/8FrKPMoWvePw0WywSSt641FP9Lxd19j/mcQuyc3ZxE43nym4zxNRyYvG2w5qPcQNDO3aFhJ2wDTQOLZj5WwRx+mnDAdX0v4KcDNXc7C6PzTDdtnC4xbDaa+LouzsGtQ/Z54ymOarP9ciU3e1iMcXY3/p1u+Zj36RAlTMJO2EbXpi1CWXlRxJthu3wcUBt+aFbi1zkZHisNygC3IowixR2KWwYnLr46nV9Q54rf+r01ft1h3PEwv2fJn9uOwk7YSnTVu3D279ux9fLA/nFY0uEKsP5Ww6rtnVNdVbvqcQ1UxaFbHexQPaJ3VAKu8vFsHDCUMx+YEjQ9tE9i3ChopjJjL/pf38caPgx7Yw9T/dE0vKHj4LjxC3yMnFj/7b4tGwXAGDxdvLYI0USzyGdCjG6ZxEe+mJVgi0KoBR2AJr56Gr7Gk2nZg1Nfw07QR47kVD2VlYjzR38NRz774X4Znnyx0HjpUG64Je9e1spri4xvrdKPESTuipvQWwWHhfD337X1X+/8rR2c7JkgISdsIz1Gm1Tla0EFm8/gvs+XWGFSY5Gb1Ex0ZkfXLU0SZ01e82baSqdX1wuhtsGtvNvn/CVfa5uzICEnbCMkS+GTvpJd7uQlU4RQQA4dLIGL83eHHF6n9S/XEIe0TiV4NRR6W/o2DQHo3ro59rXy1oQPybzqgHg0t4ton5t+Ult4fhhmHn/4JB9pq9WH+qRLJCwEwlldM8iFBdkJdoMWzDhq9V4buYmLIkxM2jLpFH+2/uOnTbKrJiQzk11Xl9IqE3JYVma69AuTYMea9owA1np0XV6rJedGJvnZaJjisXXARJ2IsE8fkk3f85zqrNILKqp0Yk5L31kuOZjLhfDH87vAEBoqpVIpDBRnZeHFXY56Z7gfWvrfaiq9eLJ79ZFPD5RakHQMDN1rwTjFnbGWGvG2BzG2DrG2FrG2J+MMIxILrRivrmZHn8lYqoj9XpRdm6U6FCYHZIHrqRhplCw9MnSXcYap8L+ymoUj5+mOqtUCsVE4rHLyVAI+7sLdwAA3pm/HTe8vTiiY0ge+71DO4Y8JvWLT3aM+EXVA3iAc94VwDkA/sAY6xrmOUSKodXIya552FYjP/FphdjlZflaXNZbmCN6Xb82htilx+o9wqLnpyonkU/LdqHk6Zk4XedFmjvyz1jpsceC1xvcE17O5eKcVQB4/Nu1OJ2kbSzifhc55/s457+Jt08AWA+gpf6ziFRDTawimbKTKsjfn992HlXdp31hcAm+sjsmAOSI4Yd2+dnGGRcGtXPzgeM1OHSyFieq66Py2LMNWEivF8cGelROKONkmTH/WVCetAVxhl4DM8aKAfQBENk1E5EySN6dnDdvKkmAJfZEnrrYqZl6D5WWjYQCn/xsoQtmpkr/GKl0v97AVrdaSFcZ4doFRxPvD1es5PNx/H36euyvrNbcRznFSQ5jDH8d0cV//9DJmohtcxKGCTtjLAfAlwDu45yHJCwzxu5kjJUxxsoqKiqMelnCIej9EAFg+WMXYOotZwdtS3QutpUclxXMVNf5cPBE6PslvRuDOhYAAJ4Z0yNkH7eLoWtRrl/8zUSyR14tHE3YReL/nScs+H53T/iy/2U7j2LK3G144HPtOge1gR1Bx9gRsNfIIR52whBhZ4ylQRD1DznnX6ntwzmfwjkv4ZyXFBYWGvGyhIM4Ua1f6dc4Ox39O+QHbavzJuePTo2znp7lvz3hq9UonTQ7ZB9JqLIzPMjPTldN40tzu3DXkPb43+p9IY8ZjZomxnKl8JeLOmPlxAvRQ2WM3lltG6s+Z/2+E5rHC3js6vLWpkkgTEXCrgETVr/eBrCec/5c/CYRyUiTCDxI5aXzVhvM8LQTmeKAihv7t8WL1/TR3O9Pn6zA3E0VOGpyG2RlJ06fj6uKfb926n3XJRhjqu2HASEPXY4Ur9dr8VwnZhVpeezytR2f+d0MEoIRHvsAADcCGMoYWyH+GxXuSURqIcVhlQUocpT9RVJF2NXCLnp0aZ6LgWI4Rg+z+6Eoe57XaajkKyqteSPFq7hqiyTU89HinQCAWevVpyjJrwxnqKRqJgNxL0Fzzn9FYMQhQajywOdCD+w2TbSrTJUO1vp9x/G7ntGXlDsNtbCLEVTXW5vKp5XSWh+HW5yRFux7sgikRjrh7AuzriOx5eBJzaEfToUqQwhLuUSn9wdjDAvGD8XzY3sBAF6ds9Uqs5KKm/q3BQBc8rK1g5u14utZabH7j/IMFgAY9VJovyEtIm3V+9g3a6KyyQmQsBOWEi5bo0WjBkh3R9cbJFmRTxd64pJuET+vT5tGAIBajQpWs6jXWOzOy1KPn0eCVuxdi11Hqvy3rzyrleZ+n9x5jv/2Qp35qE6FhJ2wlOwIRrl55VWYFuRj25XcBh40EBdM83MiT1+MpiDISOIJuWgRTSXqzsNVGPTsHP99vYyXc9rn+/vqyKn3+nCsyvmzd0nYCUuJtmT88e/WmmSJ/RnQoQDDzhQWm6MZXGGVsMuzTmrrfdigk4IYK9H8LYP/MSfofscwcfMHLght0XDGI9+j95MzNWcHOAUSdsJS0sUfql52zEXdAulo74lNoFIRl4th4sXdcMM5bTD8zMjbL0gVqgCwx8T2vfKY+qJth1Wri81kW5isqUZZ+lc5Lp0q1+U7j8Vkk10gYScsJc3twvZnRuEdRZWpnAyVUvlkpqnYsXHj0yPwxd39gx4rbJiBpy/rEdWVTveWgUKf13/eYoyRYfC4GJrnZobf0UC2Vpwy7djHwxTU2Z3UbVjsIL5Zvgc5GR4Md3DTrIYZHuQ2SItpcHFlVR0On6rBqRqvanWi0+lSlIuiRnXI8LjRJt/YoSPzt1izMOh2MbQ12PZwVJwwts+LvODqWJWzhZ08dgfw77nb8GlZoDVqxYmasL1X7MS6vceRk+lBv/b6FYhy5CJx+WvzMfRfv+DiV341w7yE4/Nxf7zaqDGBo3sWAQC2HzLPq+3SPJBO6HEzfxZOuiwuvmjCMNNev6hR4Arhg0Wxheyu79fGn6l15RsL/dub5+r3vbc7jhb2QydrUqJRlMfFgoo/zp40C+c8Y05Ri9Gs23sco16ah32V1VF5WD/cF5hTuc1EcbIDrZtkoV2B0L9EyoKRi2YsTBRnh0682LzRCLmZgVREF2P+dEd52EjZEsBIqmu9qKn3orKqDo/GmIue4XGjVmViVW6UaZZ2w7HCvn7fcZQ8PQtPfLcu0aaYjtvF/AtVmw4Yn3lgJvLFu3kRjjYDAn1RlFz+2nwcOO6cq5VIeGZMD/zzKqEoy+1imDymB16+VrsXTCRI4mqm3yNPb/TxQI8W5RQkszhd58Xt75ah15M/hvSmbxHhCSXd41IdRbjrSGJnxsaLY4X9hVmbAAjN8pMdwWMXvnwXPj/Xv33LQfv3UvEanNu8fOcxzFiTnP09JK4pbRP3AGYpTfDJ/5nn+Hh5oHHb0VO1/m6cRkxB0mPeQ+cDEIRdchbW7AlOT9wbYagyw+NCrdcXUi/xvKgvTsWRws45xw9r1Rv8JCMrdh3D4m1HQsJOw5/7JUEWRY68+PFmsdQ9Uu4Y1E51++EkHY5gJGaLKyCctKUQ4e3vleFd0cmSwkpG8+5tpVjy8DDkiEVuemPt1IqP1Phi2W4AwNzNyTUjwpHCXl2XpL02Naj3cdT7OD4rC50tWV1n75mN8sv10nb5OnuGojWZp3leA9XtRACtlrVGouxYIJXmt2hkzuczpFMhmuZm+kfeLdl+RHNftbi5GlKo8JapS0Mec1KCghJHCruyVPjA8WqM/3IVaizuZmc1f/1ydci2DfvtHXOXx9jX7o2ugEUrzv7O/MhHraUq8iHhM9aYM3RDK8wWrnAoWj6/uz9ekq05eMQBGj+u075qf3Ne/N8RpyQoqOFIYVeuB9349mJ8snRXUoZnwmX92DkraPb6A3h2RmBYsHTZGylaZfROWFuwEz+a9LvQatN7Wryi/pe4IBwvZxc3wSW9Al1BY6mF0KK0WD8F958OHXbtSGFXsumA8EM3umDBDoQbD2dfWQfGvVsWdP/D2/tF9fyczEBO9/mdaZxirKwzoe9JndenWfkpDcOIZGpWLOiFmTY8NQLF+Vn4+I5zNPeR87iia+bwM4NbXbwyx5rKXaNxpLBream7j1apbncyZoSXdh+twpaD1odwiqKMvf5+SGAB7I0bzzLanJTBjLmeeuLq96hNCvPr9XjJTHPj57+cHzI/V4v2hcELvTuPJIeGOFPYNbY30IjJOplwi0AnNRYY9Rj4f3Mw/Lm54Xc0mGivoBukBz7PDI8bz17Z02CLkptZfx6MIZ0K8dSl3Q0/NtPpNimJvhPGqinDfZsOnMRlOsNgnIIzhV1D2RPVh9pMwg1LuOmdJRZZEh89WuYhM4bmXv+6qpc/TfIqncEJRCgdCnPwy6YKjJ2yyJTjj+nbEq9dHzrPVBJLPfG3C2pXHi9c0wff3jPAf/9kjbrzdPBEtW0TNpyphBrC/uLszaa83MpdxxKW+hRp2pbd+eL3/XUvobW44qxWeEL0OJ0gFLFwsqbelLRVs9+v567ujVE9ivDQiEBf84kXd7Ukh16NSKtN5Si/k43EaU89WzXybzuuMhT88MkalE6ajc6Pzoj6Na3AkcIunSXHj+wSZs/4qfP6cOmr8xOW+pQswp5qrXgjobrOi/s+WY7uE3/AHe+VhX9CHBidPVV5ug53vFeGnzceRC9RBHu2ysOtA9ph/hahGvTIKWuSGaT+/R/fGdmCqR5v3VQSsu3IqdCJSlPnl/tv23HikiOFXSpQaphpftdhtbO1laj1sTAKMys4K01+3/SqDp3CX79chW9W7AVgbk8XQHvQdKzU1Hkxc90B7Dl22n81u2q3UKcgvVQ0U5/i4dHRXbHs0eFo3Ti+tsEdm+agb5vGIdufVUl5lGfL9Pu7/fLdnSnsosfeUNZdLjvdjfM7F6Le4AG++wwMwWytOIk1UU6ZUcbYuxbl4sf7BwdtU/MoFmw5hIMn9G0/6+lZAICZ6w4YLpTyk4aaFxQvpZNmGX5Mq/lV1hRNr4rSCOoM/l3IxftUbXAMevKYHgCEdRUraJabifycjJhCfXLeurlE9RhzN+m3GzDT+YoVRwr7Kz8JZ0v5R9AsNxNzNlbgjEe+N/S1jFwcGfavX/C7l6PrKV6jaJ9QcbIGnZo1RPnk0f5tt0wNXkCt9/pw3VuLccNbi8Mef82eStzxXhke/9bY2aIPfL7Sf9uMASEnauodPej6yKlaHJadkMMtksfLa3O2Gno8qYp43uaKEM/8mtI2KJ88Gu0L9WeOGkW8MX1pgpXyOAU5kfdk33nYXmmSjhT2b1cKl6/HZJf7ZvXstupyUgvlD16tCEs5ePfGtwWhlwq39Nh9VCj5333M2C+mFTMjX7No7JsZlB+2tsd8tFW/4Vgk9oWZs6FCs/WDmbwwtrfhx2SKBM3Zfx4S8XOX7zpqtDlx4Thh3yrrQ5FmQaOjcJWfkRLr4pVy8fT5sYEybWlosdJGqRmTGit3BQvu3R8sA2DcCDXOOXw+juv7tQEAjBuo3qHRCLYfspeXFA11Fl++7ze4h30PccH0Lxd1xuV9WuLR0Wdiw1MjDH0NPS7r0xJbJo005DX/fnkPtC/IRn5OcKVsXlYg1LtC8bsZcEZwAVSiHUAljhP2PUcDTaU8bhdevKY3Prurv84z4mPslIXhd4oAuectD+9wzrF422FwzvHN8j0hnpVS2L9fHehFHi62J+Xofla2C8Xjp2H9vuO47LX5Mf8NkTDhq9Vo//B0fz+Xuwa3N+217JiNEClGL2ZaTZbopZ9d3ARuF8Ptg9pb7rl73C5DXnN412b46cHzdOtgdikqUpVC/sePl8dth5E4Tti/Xr7Hf/t3PYtwae+WKG0X+SzNaDEqW0G+OLlDFo/7cPFOjJ2yCP/6cRPu+3QFHpTFpgGg1hsc45e3sj0kW6BUuyKQxOMHcTBF+aFTun+PVlOnaPhkqdBaeLG4GGh0muNM2cLx7A0Hbd+2WAulsGemmfNTPCeKObPRIC3Gpnns5akazQfjhP5Gygw8zoG+bRph0uXGV/UagaOF3WoPIZ5cYHkPeSnPFwB+2nAQgHazIaXH3rpJoN9KS1nvlUHPztF8balQ5fcf/qZroxlVdEb/8Ds2a4hv/hCoCuzymD0LRMIhv/Ls0TIPr14XWsFpBP+5tRSA8VW70hVoMlZ7y2kqDrX+8rc9eEw2V5WDgzGG60qFkKPd2pkY8qkwxt5hjB1kjMU2UTYKpCG/14pvqJkohTyey2ev7Fhyj/3XMHNAlcIu94D7tAlUx+0+qj2jMdLwnxnFUGb88HtalEZnJg9/Heit/90fB2LYmcZnDgGC89OuIBvVBn+2/jF4SS7sUhuM71buxfuLdvivVDgXsvIkp+l0nddWmTFGfSr/AWDJyok0WOIJRbvN/9x6NrLS3XC7mGFVdkodjyVUUfL0TNz5XllQLuwi2eLmxb1CGw49/PVq/2tJcfR3bxM8rwmjAtW2kTbFmqkzkECOGWFfM4RdmWucLNW5ZpGZ5ja8TkEKGSa7x67MSpPqUDgPdZgueN4+oyoN+VQ453MBmFthoUDq+SxxXuemuGtwB3h9HKcM+hIr253G0v700Mla/LjuACZ8FfDQNuw/4RejfZWhnvZHi3fi541CiEZaTO3XrgnKJ49GVnog1ie/rUa0cdvNB+w9jUmLJ74zNgffSuRZTmax99hpzFpv3LANeYaIx53cMXalMyctmi7cdhhLy4+GPGb09KhYsex0yxi7kzFWxhgrq6iIf3CsWoOjl38SmoBNX2XMKDClkOt57KdrvRj3n6URX46NeEFom7tgq3qaoZR+KF2hhLvkvebs1v7beQ2ENK3qOl9Upf0Tvg4dvecEPly8M9EmRI0UUhzSqWmYPePH6PYONbIF6zRXcnvs0m9JYvaGg9ioGEd5nZjae7rOi6H/sofXbtmnwjmfwjkv4ZyXFBaaMw1HioEfrzbmi6x00DVGPAIAft54ELM3HMTfp6+P6NjhCqo2KwZhaJVL3zqgGEDgkphzHvRDHjD5J7TN1++hMbpnEYDQlK54GVvSOvxOMfL+uFL/7XB/nx0pbdcEjbPSTJsypIZRIUp5C4vcBub3a0okzRUdI1+avRkXiU5ZTobwt0+6zH6ZMUl1uh3SSThhNMoy5scS4rHr/DCkR9bvj3wMmVoYRmJemEVViYkXd0Pz3Ex/aEd5UXGypj5osVbOtHsHYvYDQ9CqsXqhUzSs2VOJ4vHTgrY9ZeIXflDHQpRPHo0GaW5c1K25aa9jFvU+DrfF3q5RufObZTNnk7WVspwHLuikul3q0658D/R+11bhOGFvX5CNgWcUqD4mZcp8utSYS/NoFk8lzdcSUTWMmtGa7nH5UxWjWeDt1iIPHQpz/G1X4+G9heWqdpmNiwWHBpyC18th1brjfcM7AjCuWZWN56ebwh+HdQzqzRSO698M36PJbIxKd/wYwEIAnRljuxlj44w4rpLaeh+2HTqFDRpecbsCYX6hUeW9yrJvvcVTHsNYaXk2x8juzdG7dWwCm+Zmfm9bEvibxKlDSjJUxNYIj/ezsuCKWamww2waZ6cHFW2pMWfjQSzbYenafljqfRweizx2Kdyz9aA9FvaShRevCfSrkdeXbDt0ypBiv3gwKivmWs55Eec8jXPeinP+thHHVXK7OIzg0En1UvLiAiHWOriTMTH8GWv3B92PxGOPhivfCLQreP2Gs/DZXf39KVQ3y4S5Zyv9vG23i/lPOpO/3wAAOFWj7sWqeW3y8L1RqYNWxb3T3C7UhfkR3Tp1Ka543ZjWEEbx5W+7seeYNZfs0sn80leNaSchFe38TlybSRWU1aeDOwZ0ZteR4M/yVY2CQ6twVCgmXF9kKXPEqEtOZW93PWGP5yph4sVdAQihi81PjwQA5InrBPcN74g/a8T45K/t4xzvLij3Z4iEy4SQXx3IY4Sx9F9RZgKluRlaN7FG2D0uZngP/mTD6NCJlNJ3p4l9gOzIy9f2CbovH7au5LmZm8w2RxdHCXs4GGNI97gM8zqVvwe9UIxaiCNScmUDQzxuF9wuBq+YgvPGL1s1UyIlGGPwcWCirKe6Ms9fzoLxQ/HRHeqhki0x5OEO/kdwO4MrzzIvG0aJx+0yrANnsiLPqDpkwNQsKRXXbh0NzebL3/YE3Q/X0mTPsdMJq0Z1lLDfNkBoAbvs0eGa+2S4XYb1PJm9/mDQfT2PPZ6Fwsv6tAy673ExfwZDnZeHzWF3sdBUtu/X7Ffdt3vLXLRo1ECzuGm7AX3td1jYa1x4r8hj10PudIx6cV7cx5N+X/E4M05EGsgBAGcXB4/QU1tTGjD5pxCnxyoc9clkpbvhYkC+zmSTEzX1mDq/3JBRYMrKTT2P3a2SZ67s4Rzpcz0uBq+Xw+sT/oUr23aJHruSZ6/siev7tcHGpwPdHqbeUhq6I+AP9zRtGP2kdyXKnu9m4nGzhC9UxUKam+H353Ww5LXk/YUOGpCJdbRKCPOlmMOOe84/w3/7cUVLk4EdC7DhqREhrU4ShaOEvdbri9gzPqoyBzRaeirSAKPVjy0xZiGcqvXirV+3++Pk4eaJuFjoSee16/vi6pLWmHR5D2R43Njw1AgseWQYChuqnxQbiUMF/vrlqphsVtpvFWkul+5JXC76xeOn2aJtAuccdV5uyaAYwPiWwFKBkiTwqULj7HSUTx6N8smj0a1FaEJDZpobN59bbL1hKjhL2Ot9EXeTM6JwYur87UH3o12EUqv0++6egegXYf94KTf81TAj4FbursTPG4MXlkf1CM5YyExz63rjUi672mDsaOnUzJpZl4DgsdfrxNiV/dqrJvNFAAAe6klEQVQveH6u2SaFRTrXeCxKZO8sti8wihFiemyfGNNzCfNxlLDnNUiLeECucg5oLCjTKvVy1SM5jbRs1AA9WuVhyo0lEb3+AXGcWXWd+THkbi1y4z6GVDh2fhfz+59IeMKkO6oN4qiu82LvsdMJ6wopXWGohe/MoCivATZPGmnIsVbvrvSnAVt1YnI6RoSFo8VRjR7uv6AT7g+T+icxdf52w/LZJaL22BX3M8RL4sz0wA9i6i1naz6/MIop6fESz4+0bX4WdhyuwuQreiDd40ITg1o6RALnHMd1UjvV+pDLh3NEU1FoFNLCuF7mktGkuV3oUJgdd7uN139x7gBxq/jj0DPw8k+B9+nTpbtwwznqBYNmkXSn3K5Fguc5Z2MFRr80D3sNLALRE3a1hyoVMci3bxZEXL6YpefdSj9C5eBcu9G1KBcdm+agVeMsNG2YaaknN2/zIX8mz39X7MEvmypw8ES1v6FZuNF5f/50RVRXd49/uxa/ezm+zBKplYRVlacSWytOYdmOo+F31IFSS8OzU9FMTz4xzSqSTtjlmQZr9x7HW/O26+ytzxlNg8M+eqEYNdHfVxmYDD+oY4G/5UGkSIIzpo+xY820yNIpuNDDx7llYQUll/dp6S/n/tMnK3DzO0tQOmm2f1TgsDBtVL9avgcjo0gB/M+CcqzZczzmfHCfj+P8f/4MIDC0wUlEOrQllVGuUynXv6wg6YS9OD9YPOdujv1NlTJNpEwcPY9dLRXynfmxn1QA4HNxyMaynfF5WZEi5bZHu4Dq44nr8idUnlrvReqFf/SYszFQG2F0n3SzcWJaaSJQhotP13ktH8CRdMKubIy/5eBJTP5+Q0xfSkmrPaI3GkkoJjtGr1eNgpzYenKURph1o0TyQvs+NTOq5/l8PGxKplnEUnn63T0DQ7Z9tnRXVMeINZVVfpmel5Wms6d5KEOEkWJE1WoqIIWD5Vg9gCPphL2NSvOpN37ZihdnRd+7QfLC3aI3qheKkfbV2kPZX/2+4R3x3NXqY9EkYZZ+SMp8+nCofbEiYVBH9XbI4UhkKCbNrV15euPboe1Tv7tnIHqoNFV76MtVOFO2qBqOYzGK40JZe4jmufEXg0XDgxcKnuTWQ7GdlFKtXW+shGs1YAVJJ+wAVItwfo1hAUMSa1ckHnuU3/r7hnfCmL7qsXP5mDu5HVp4ZKJakJMe8wr8e7cFqlKjSdFKbCjGhXovx6Bnfwp5TDqZSsVXTRtmqIq6xOk6b8Sf4xkx5uoPlJ08bxvYLqZjxEqnZkI++9q98acCE/p8fMc5GGph2q+SpBR2tQEW0rSTaGiRJyzKtRE7Fer95CWn0QivJjsjOAtV3iRMDXmTp7JHLwhZ9I0UuTj/7b9rIn7e5gMnDKn0jQWhF70vpG2qnGNVdVj88DDMemCIf5tWodvkGRsiet1Y483y01+4z9VoloutHp6Mcfi33MGYfu8gQ2xKVvp3yMc7ilRmowbrREJSCrsaWk2v9JAmMl3auwUAfa/8oyVCu9zTBkzzCdd/XUlbE1rkfrwk8pjz3srqkBQvq/C4WURtmpvlZgYJ6YanRuDd20qx4akRQfv9+5dtEfVJlxZsT9d6o4o9y78fVkyYktOiUXwjEOUns64GFLSlAu0LA8kcWy1cQE16YZ8wsguA0Cb50SDFj6UsFTXkor9czGK5Jca+EUV5DaJajHx/XD+8el1fbDGgulCqQB3Tt2WYPe1BrLngLhfDkE6FqvHQAZNDwzpKJJG74vUFKHl6Fuq9voi8+L9Pj+yKwAyuiPMzTUQFpdOZdFkP/+1wIVUjSWph3/DUCNw1RMhrj3Q4tBz/4qmosh8t1p6lKh8U/Ino7Uabtx782pHv2zwvE6N7FhlSGPTKdX0BRLeQmu524e4h1nQqVKKs3hzZvbk/dCbRMUxoSum1R4K0YLtOrDXo8tgM//R6u5LpiW9Rj9Ido6d/h3x/FpbUIsQKklrYld7YwSjfWOkEG8lAgdGyplvSQq0TfwjSQmykl+ucc6HrpoXl8XKUJ7PXbzgLcx86P2ibFILQIjPNjdmy+DsAVNXqr8koc+frfTzmFEirkK/FvLugHMero8vsWbXbeQVVdkCSj/s/XWnZVU9SCvv5nQvRWCVHONpcUqXHrsdYWSaLtL+Vl15G4REF+sVZm4O2X/XGAtVcb+nklaiGUJ4IPpu/jugSdp8OiuZyj3+rv8C4ard6z/naeh/eX1iO695cpPp4QY51fXT0mPjtWvR8/MeonlNu4QCVZEKezPDdyr2WvKajmoBFyivX9UWVSk/waDNjJFmOJN4tzyiRRN6Buu4/KSkXEJeWH8XS8qO4WpGKWe8X9kTlsaufUH68fzAufH4uPrqjX0wLfZ+V7Uaz3Ez0bNUIF3RtFvL48Wr171KnR7/XPe457fPxv1X7kJ9tD4GPhpJiob5Ca6wioY48cvDg5ytxeZ+WpqcHJ6XHnp3hUc1lHxdl3rC0IBrtbMcO4kq4Ez12tY6Sy3VaGviFPUEFSvKPZuXEC/23OzVriPLJo3Fuh9iKrgDg5Z+24I73yoK2SZXFHZvl4Ko3FkR9TM6F92rWn4eE39kE4klT/Oo3IXnghMZJjQiPjwMPfL7S9NdJSmFXUj55NApyMnD4ZE3Ybn9ylDF2vctoefxeep4DQ+yqnsT01fuC7nPOUS52VKwXY4ZWdyqUkM+lVbaTiJZ/33hW2PYNUrriI1+vwdLy6Hv4eH0c7Quz0ThBHrvy6mXxNqESlnOu22rA5+P47wohjGCHKVROo4ts2Iky7GcGKSHsgFCe/82KvWG7/cmRhNkrKrVy8Iacm95Z4r8t6bncY1cOvw2H1BbgrZsiG8phJsoQ1o1vL8F5//wZM9cdwDu/Co3O9FJBzSSWwjMtLurW3J8VpMayHUejOlmrLZ77OI/6CtBo5FdXY6cIawGvztmCXk/+6G93rOSnDYETaL0TPZYE880fBvhvl7SNTgtiISlj7HpEUnwiIQlzJJN25EOCpef5xB/AzPsHh83MUPL6DX3xxbLdGHZm4sqSJeTFR8Xjp/lvV56uw0viQIFoM46MwoweNd1a5AaV3dfWC7N2r3g9utBLndcHtys4M8vHow/tGY2aMP/zR6GX0l3vL8P0P4WGa16eExgccXkfZ9Q42Al5nL1fe/PnK6SMxx4L0tc/kv4hRXmBhk4+RSimQ2FOSJuAcLTNz8YDF3ZOWA8WOSVt1btFPiiLFf5To6GZ2UQ6AzcaLusdLFzjv1yFUzpXBi0bNcCUG8/C9mdGBW1XS23zcY4ERa38XHWWdn//dRpDR1buCmQBtc2PvT4jlZl+7yD8/OB5lrwWCbsOkqBHcuEp9/Ck50meuw20OSqGdmkaVOQTyYmtbxvzLy/ViKeiWAvlSfir5XvQbeIPmvvfcm4xLuzWPOQkrBWKcSf4C3H3ecHFZOFy9uWoZQgRkdG1RS6K4yhajAYSdh38i6BRxhR9nOOzsl14cfZmMJa4zoex0iDNHdTHpG8EMcGGUV6RGMWfxaEGuQYKfN+20bVJ1sq2Ugt5JLITpkSHwpygWa/PhGlz8MYvW/23n7ikm2l2EcZhiLAzxkYwxjYyxrYwxsYbcUyjGRNDXNAfK492iDUH1opjzxIdT42Faav3BVVRbtyvnwXx+MVdg6oaraS4IBs9W+Xh3zcat8jcpXnkee8LJwwN+tvLJ4/GpMu7A9Dw2BM4lESL9xft8N9WO0FO/j4g/PKQI2Ff4hZ2xpgbwKsARgLoCuBaxljXeI9rNM+N7R31cySPvTbKMmDOgXcXCj8Wu/2Io+FYlZAFFK4M+nqLJ7DLSXO78O09A9G/g3UDvx8a0dl/uygvdFE8TQyiq3vsiRtKouSdW0JPhr3bNMZpleI+iURfbRCRYYTHXgpgC+d8G+e8FsAnAC414LgJR/LYzxVFI9Jhz/I0RydPde/9pDAirzbM36BV/elkHhBDPG+qpJte368tXru+L+ZoLIRJwq2WTVV5ui6mFtJmMLRLaLx87qYKnPm3GX7box0gQ9gDI36RLQHIm4jsFrcFwRi7kzFWxhgrq6iwfmo3ILSibdU4urRDQOjS6HYxVNV6sf1Q+H4ZyfRbeGveNrw0e3PI9gwxBn/rgGKLLbKGPw7riPLJo0MWC/u1a4K8BmkY1aNIs3tn2Q6hcOkFlXGMdV5fxA5CIpE+8yXbjyTYEiIWLHO1OOdTOOclnPOSwsJCq142CDdjmpPh1dLZfLKWAlK8dMFWoXPj7z9YFtIoS+KhL1cZYW7CuKl/ILTy9LT1qvtIwy26t4huKIgTubhXC//th0edGXZ/KWXw+zX7Qx6rPF3niCucV+ZswbIdR/wFTISzMOIbtgeAvDNUK3Gb7fh82W6cqK4PiSEu33kU3Sb+gNd+3hK0XQqRysOKUqra92v243mZR1Za3ATtLUplMpsnL+2u+/gzYwLDA6rr458YZXcmy/7eXq3DZ8xIOd/KUMzMdQdw4HgNvrWow18kfHi7dkOvK15faKElhJEYIexLAXRkjLVjjKUDuAbAtwYc1zTO/FvwNPqZ6w4AAJ6dsTFou1o/dq3sDzstipmNNDIQCJ8xkwxkZ3hwbWlrvHGDdruBcPh8PKShmB3o3jJwxbXibxdoZr28fXMJNj4d/UASIjHELeyc83oA9wD4AcB6AJ9xzmOblpsgNh0QUvvOaR9cYanWnXHfsWp8uHhHyHaOxPUkTwSl7YT3KlWqEJ8Z0xMjuus3CJOQn/gkHpMNB09Uzr8acmekUVY6Fk4YhiUPDwvZb3CnQmTEOYGJsA5DvmGc8+kAphtxLDPp3boRVuwKHZAwa73gsedo/ODkHvvzKgtigHASUI5pS2aGn9kUS7YfQa8oB2+nArcNKMbHS4LHKH4oG6uYkWYfB0DtO980N9RrT1RbZiI27PMNs4Bnr+zpv61WPHKqJjheLFWcRpK66+PmNKSyK7cPbI///XGgf/gCEaBNfpbu42b0t4mH0uIm6Nysoebj4wa2o/x1h2Gfa0IL6CT78q7ZUxmyENY0N3jIRGCCUgRfas5TKhTjcrGg+CwRIJxwt4wh5dZMPru7v+Zjv/zlvJQJtyUTKSXscuSNnm4dUIyp88tDhkX4m3hFcDwfT9ywCTMY0qkQv2wK1BsU52fhlev64miVdk96QkDu3XLO8fGS4Fmxr14f+yKsVWyZNBLV9T7N8CRhb5JHiSJk/EhpsHEgFDN1fjkA4MvfgodFcJV0Ry2SLSsmR9EzpE1+Nrq3zMOgjompQXAaUnOyrRUn8fDXq4Mea9rQ/v1WPG4XibqDSTlhb91YiH9G0v6Fcx5xd0bO1VMhi8PEW+3Ko6ODC3FevrZPgixxJlIR0vDn5gZtl3dVJAizSDlhl8KfaounSnw8sjCMsC9XzRywS1+QaFE2t4p3nmiqoZYhFUuHUYKIhZQTdmkhVC1HXQlH5PMpucbIM4+DUyClYpXmKulvhD5qbQOyMigPnLCGlBN2KQ4u99i1mjkJQxEiO64QYw/d/tI1zg1h7KsU5pi+cE30LY9THbUTem4mXfUQ1pBywi7Fwb1BrXXVA+48imk3HOp57FaNwjKTcP3YiVDUPHYnDl0hnEnKCbvUxEs+7k4SLqUuc879MfbeOs2fOOfYcvAkpq8O7uYnZUY4Fam97MnqyGdiEgIrVSqcl5RTC1zCGlJP2EX1PnwqkI9d7w2MwJMLPkcgFPP+uFLV4wmZM+qemNNj00M6CamN0Y4GJIDfdoYKez1d+RAWkXLCLl0O3/X+MgDA8eq6IJGv8wV+fHVen3/MWUOV+Gi6xwXGGCqr1Hu8O/3K+8GLOuOsto0xuFNBok1xHBf3Cm0YptYcjCDMIOWEXRkHH/XivKD79V6Oeq8Pv/9gGVbtrtQd0vyKmNvNZcVOl8iGMji9v0aHwhx8+ftzVU9qhD5XlwRGFEiCrhfOIwgjcWaSdRwo17R2Hz0ddL/ey7H/eJV/+k2jLG1Rk04S8szJRdsO+28nUSEqESXSWs6AM/Lx9GXdcV1pG3TUabRFEEaScsIu96KLx08LefxUbb0//ALoZzIoH+vduhH2HhNOFL1a5eHCbs3jNZdwKI2z0/HBuH7o2ToPbhdDD2pvTFhIyoViwtUljXu3LCgHWc/pVmr+Zb1b+IMyb95UQr02UpyBHQsod51ICCkn7IdP1qhuf2hEZwDA/cM7BhUv6cXJb5m6FJzLI+xClky45xEEQZhJygn7nmOnVbdfeVYrZHhcePDzlRj07Bz/9nD6LHWGFPaVefqk6wRBJIiUixWke9TPZY2z0lFT70ONYrJ8uAXQnUeq/F46ED7UQxAEYTYp57GrlXoDwI7Dp1S3hysD/3jJTpQfrgIAvDR7c3RTlwiCIEwg5YQdKh71lkkjNSewh5PnmnofPlsqTMg5fKoWbcX+607u6kgQhLNJuVAMV1F2j9uFzDQNYQ/jeRflZaKmPjAE+82bSrBsx1HKhiAIImGknMeujIFLE20y0wJvhTwOHy6iMuCMgqC4fEFOBi6i/HWCIBJIygl7nzaNVbfLc87P7ZDvvx0uVv7Fst3+oiSCIAg7kHLC3rl5Q3xy5zkh2xlj+Oj2fpgwsou/26OwPbDPw6O6hDwPAFburjTcToIgiFhJOWEHgI37T6huP/eMAlxzdhvM33rIv03usVOmC0EQTiAlhV3qsnd1SauQx+58vywoDi+XcqomJQjCCaRcVgwA9GrdyL9oqmTNnuCwilzLSdYJgnACcXnsjLGrGGNrGWM+xliJUUYlklO13qD7LCgUo//cNMpdJwjCBsQbilkDYAyAuQbYYkvkYq43dAMApt87yGRrCIIgwhOXsHPO13PONxpljB1o3aSB//ZF3ZqBQd7YK3C7X7smIc/Na0BFSQRBJJ6UXDzV4kR1HXYdCeSkcx4cY5c77Go9Z8J59ARBEFYQVtgZY7MYY2tU/l0azQsxxu5kjJUxxsoqKipit9hEHvpiVdB9H1e04pV575f1aRnyfA8JO0EQNiBsVgznfLgRL8Q5nwJgCgCUlJTYsrmtcv4pwINj7OLtHi3zVEMx5LETBGEHKBQjQy7MgzoW4MDxGkUoRhxeDa7aQ8ZNee4EQdiAeNMdL2eM7QbQH8A0xtgPxpiVGOTZivM2H8LqPZVB1aa1XqHZ1/7KatUqVDd57ARB2IC4CpQ4518D+NogWxKOmljLY+wz1x0AABw6WUvCThCEbaFQjIzj1XUh24JbCgRuq2k4hWIIgrADJOwyNh04GbLNpRJjB9T7xtDiKUEQdoCEPQzB6Y4BSMMJgrArJOxhkAs4oxa+BEE4ABL2MMiLkuTFpiTsBEHYFRJ2DXq0zAOgbNsrC8vQO0cQhE0heZJx77CO/tvSQqhc2H2yCRzksRMEYVdI2GX0FL10ILBQKhdwn6wRAi2eEgRhV0jYZcgb2EjCLS866tUqT/Y4KTtBEPaEhF0GVwm1yDNhcjIDhbqk6wRB2BUSdhnBHrug3PL+MUV5DUIeJwiCsBsk7DJkDrvfI5cLeJsmWf7bSmFvm58FgiAIO0DCHoSg7Bd2beYXbnmbAA55qMZaywiCICKFhF2G5LEzBrjEd0Yu4B5X4O1S9oohnScIwi6QsMuQ/HEGFoixy5S9cZb2sGpq2UsQhF2Iqx97siH32CXknnnT3EzV5901pD2uObuNmaYRBEFEDAm7DCmGzlhgcVQv+6VV4wYoLW6CCSPPtMQ+giCISKBQjAb+AiWdCIuLMdhyKjdBECkNCbsK8hi7nse+80gVvl6+xyqzCIIgIoKEXQUO7o+t01QkgiCcBgm7jHM7FCCvQRruGtzBH4pR03V5oVJOBi1TEARhL0iVZDTJTsfKiRcCgGq6IwAsnDDUL+ZLHh6GjDS3tUYSBEGEgYRdA5cLKM7PwpOXdg/aLu8Xo5X+SBAEkUgoFKMBA4PbxZDmpreIIAhnQaqlAWOgVEaCIBwJCbsGLsaCuj0SBEE4BRJ2DVwMOFVTD6+P1J0gCGdBwq4BYwwHT9Tg0W9WJ9oUgiCIqIhL2Blj/2CMbWCMrWKMfc0Ya2SUYYlGGpP38ZJdCbaEIAgiOuL12GcC6M457wlgE4AJ8ZtkD6QIDLXjJQjCacQl7JzzHznn9eLdRQBaxW+SPdh/vBoAKMZOEITjMDLGfhuA7w08XkJZsv1Iok0gCIKIibCVp4yxWQCaqzz0COf8v+I+jwCoB/ChznHuBHAnALRp45yhFBSJIQjCaYQVds75cL3HGWO3APgdgGGca2d+c86nAJgCACUlJY6JbzjGUIIgCJG4esUwxkYAeAjAEM55lTEm2QtqKUAQhNOIV7VeAdAQwEzG2ArG2BsG2GQrPBSLIQjCYcTlsXPOzzDKELtCwk4QhNOgOEMYKBRDEITTINUKg0dvmjVBEIQNIWEPg8dFbxFBEM6CVCsMaeSxEwThMEjYw5BNw6oJgnAYJOxh6N8+P9EmEARBRAUJexiyyGMnCMJhkLATBEEkGSTsGtw7rCMAgJZOCYJwGiTsGqSL2TDpHnqLCIJwFhRA1mDcwPY4UV2PcQPbJdoUgiCIqCBh16BBuhsTRp2ZaDMIgiCihuIMBEEQSQYJO0EQRJJBwk4QBJFkkLATBEEkGSTsBEEQSQYJO0EQRJJBwk4QBJFkkLATBEEkGYxzbv2LMlYBYIdBhysAcMigYxkB2aMP2aMP2aNPqtvTlnNeGG6nhAi7kTDGyjjnJYm2Q4Ls0Yfs0Yfs0YfsiQwKxRAEQSQZJOwEQRBJRjII+5REG6CA7NGH7NGH7NGH7IkAx8fYCYIgiGCSwWMnCIIgZNhS2Blj7zDGDjLG1si29WKMLWSMrWaMfccYyxW3X88YWyH752OM9WaMNVRsP8QYe8ECe9IYY++K29czxiaI21szxuYwxtYxxtYyxv5k0fujZU9nxftznDF2nwX2pDPGporbVzLGzhO3ZzHGpjHGNojvz2SL3h8tewz5/mh97oyxJoyxmYyxzeL/jcXtjDH2EmNsC2NsFWOsr7i9t2j/WnH72BjfG6PsOV/x/lQzxi6zwJ4u4vtQwxh7MNxxEmiPYb+vmOCc2+4fgMEA+gJYI9u2FMAQ8fZtAJ5SeV4PAFs1jrkMwGCz7QFwHYBPxNtZAMoBFAMoAtBX3N4QwCYAXRNlj+J4bgD7IeTImm3PHwBMFW83FT8Xl2jb+eL2dADzAIxMlD1GfX+0PncAzwIYL24fD+D/xNujAHwPYdzuOQAWi9s7Aego3m4BYB+ARomyR3HMJgCOAMiywJ6mAM4GMAnAg+GOkyh7jPx9xfQbsOqFYniDixU/zEoE1gRaA1in8py/A5iksr0TgF3S8820B8C1AL6DMJ0qX/xiNFE53n8BXGAHewBcCGC+FZ8XgFcB3CjbbzaAUpXjvQjgDjvYY8T3R/m5A9gIoEjcVgRgo3j73wCule3v309xnJUQhT7R9gC4E8CH8doSiT2y/R6HhpDKj2MHe4z4fUX7z5ahGA3WArhUvH0VhB+nkrEAPlbZfg2AT7n4LptszxcATkHwqHYC+Cfn/Ij8iYyxYgB9ACy2gz0Q3h+1980Me1YCuIQx5mGMtQNwFhSfJWOsEYCLIYhswu2BQd8fxefejHO+T3xoP4Bm4u2WEE4iErvFbfLjlEK4qtlqB3tg0PcnQnuiPU7C7YE5vy9dnCTstwH4f4yxZRAukWrlDzLG+gGo4pyvUXmuGW+slj2lALwQLpfbAXiAMdZeZmcOgC8B3Mc5P24De9IBXALgcwNt0bPnHQjiUAbgBQALRPskezwQPquXOOfbEm2PSNzfH73PXTxhRHTSYIwVAXgfwK2cc59N7OkB4IdYbTHYHkN+XwbaY9bvSxfHDLPmnG+AcEkDxlgnAKMVu6j++BhjvQB4OOfLLLLnOgAzOOd1AA4yxuYDKAGwjTGWBuHL8iHn/KtE2yM+PhLAb5zzA1bYwzmvB3C/tB9jbAGE8JDEFACbOecxLXQbbY8R3x+Nz/0AY6yIc75PFMeD4vY9CL5iaCVuAxMWfKcBeIRzvijR9ohcDeBr8ftlhT3RHidh9oiY8vsKh2M8dsZYU/F/F4BHAbwhe8wF4Qv2icpTr4UJl0E69uwEMFR8LBvCgtMGxhgD8DaA9Zzz5xJtj+yplr4/TMh+yRZvXwCgnnO+Trz/NIA8AIZnD8Rij0hc74/O5/4tgJvF2zdDiOVK229iAucAqBTFJB3A1wDe45x/kWh7ZM+z+v2J9jgJsUeGKb+vsFgZ0I/0n/hG7ANQB+EyeRyAP0HwpDYBmAzZQhaA8wAs0jjWNgBdrLIHQA6Ey661ANYB+Iu4fSCEy7dVAFaI/0Ylyh7xsWwAhwHkWfj+FENYiFoPYBbETAEIniAXt0vvz+2Jsseo74/W5w5hIXs2gM3i6zYR92cQFnS3AlgNoETcfoP496yQ/eudKHtk790eqGQRmWhPc/EzPQ7gmHg7V+s4ibLHyN9XLP+o8pQgCCLJcEwohiAIgogMEnaCIIgkg4SdIAgiySBhJwiCSDJI2AmCIJIMEnaCIIgkg4SdIAgiySBhJwiCSDL+P3TMPU3I9BuiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x110019610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "data = fred.get_series('T10Y2Y')\n",
    "plt.plot(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "frequency                                                                Daily\n",
       "frequency_short                                                              D\n",
       "id                                                                      T10Y2Y\n",
       "last_updated                                            2018-09-27 15:51:16-05\n",
       "notes                        Series is calculated as the spread between 10-...\n",
       "observation_end                                                     2018-09-26\n",
       "observation_start                                                   1976-06-01\n",
       "popularity                                                                 100\n",
       "realtime_end                                                        2018-09-28\n",
       "realtime_start                                                      2018-09-28\n",
       "seasonal_adjustment                                    Not Seasonally Adjusted\n",
       "seasonal_adjustment_short                                                  NSA\n",
       "title                        10-Year Treasury Constant Maturity Minus 2-Yea...\n",
       "units                                                                  Percent\n",
       "units_short                                                                  %\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the series information\n",
    "fred.get_series_info('T10Y2Y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download Data with its Catogory\n",
    "You can choose a category index and then search the most popular series, and any item of the table can be download with its _series id_.\n",
    "\n",
    "But here, you may be confused about the category index. How do I know the meaning of the index? \n",
    "Don't worry. I will solve this for you in the next section : )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>frequency</th>\n",
       "      <th>frequency_short</th>\n",
       "      <th>id</th>\n",
       "      <th>last_updated</th>\n",
       "      <th>notes</th>\n",
       "      <th>observation_end</th>\n",
       "      <th>observation_start</th>\n",
       "      <th>popularity</th>\n",
       "      <th>realtime_end</th>\n",
       "      <th>realtime_start</th>\n",
       "      <th>seasonal_adjustment</th>\n",
       "      <th>seasonal_adjustment_short</th>\n",
       "      <th>title</th>\n",
       "      <th>units</th>\n",
       "      <th>units_short</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>series id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>TERMCBAUTO48NS</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>TERMCBAUTO48NS</td>\n",
       "      <td>2018-07-09 19:51:02</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2018-05-01</td>\n",
       "      <td>1972-02-01</td>\n",
       "      <td>66</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>Finance Rate on Consumer Installment Loans at ...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RIFLPBCIANM60NM</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>RIFLPBCIANM60NM</td>\n",
       "      <td>2018-07-09 20:01:02</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2018-05-01</td>\n",
       "      <td>2006-08-01</td>\n",
       "      <td>52</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>Finance Rate on Consumer Installment Loans at ...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>TERMAFCNCNSA</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>TERMAFCNCNSA</td>\n",
       "      <td>2012-06-26 17:01:23</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1971-06-01</td>\n",
       "      <td>38</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>New Car Average Finance Rate at Auto Finance C...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                frequency frequency_short               id  \\\n",
       "series id                                                    \n",
       "TERMCBAUTO48NS    Monthly               M   TERMCBAUTO48NS   \n",
       "RIFLPBCIANM60NM   Monthly               M  RIFLPBCIANM60NM   \n",
       "TERMAFCNCNSA      Monthly               M     TERMAFCNCNSA   \n",
       "\n",
       "                       last_updated  \\\n",
       "series id                             \n",
       "TERMCBAUTO48NS  2018-07-09 19:51:02   \n",
       "RIFLPBCIANM60NM 2018-07-09 20:01:02   \n",
       "TERMAFCNCNSA    2012-06-26 17:01:23   \n",
       "\n",
       "                                                             notes  \\\n",
       "series id                                                            \n",
       "TERMCBAUTO48NS   For further information, please refer to the B...   \n",
       "RIFLPBCIANM60NM  For further information, please refer to the B...   \n",
       "TERMAFCNCNSA     For further information, please refer to the B...   \n",
       "\n",
       "                observation_end observation_start popularity realtime_end  \\\n",
       "series id                                                                   \n",
       "TERMCBAUTO48NS       2018-05-01        1972-02-01         66   2018-09-28   \n",
       "RIFLPBCIANM60NM      2018-05-01        2006-08-01         52   2018-09-28   \n",
       "TERMAFCNCNSA         2011-01-01        1971-06-01         38   2018-09-28   \n",
       "\n",
       "                realtime_start      seasonal_adjustment  \\\n",
       "series id                                                 \n",
       "TERMCBAUTO48NS      2018-09-28  Not Seasonally Adjusted   \n",
       "RIFLPBCIANM60NM     2018-09-28  Not Seasonally Adjusted   \n",
       "TERMAFCNCNSA        2018-09-28  Not Seasonally Adjusted   \n",
       "\n",
       "                seasonal_adjustment_short  \\\n",
       "series id                                   \n",
       "TERMCBAUTO48NS                        NSA   \n",
       "RIFLPBCIANM60NM                       NSA   \n",
       "TERMAFCNCNSA                          NSA   \n",
       "\n",
       "                                                             title    units  \\\n",
       "series id                                                                     \n",
       "TERMCBAUTO48NS   Finance Rate on Consumer Installment Loans at ...  Percent   \n",
       "RIFLPBCIANM60NM  Finance Rate on Consumer Installment Loans at ...  Percent   \n",
       "TERMAFCNCNSA     New Car Average Finance Rate at Auto Finance C...  Percent   \n",
       "\n",
       "                units_short  \n",
       "series id                    \n",
       "TERMCBAUTO48NS            %  \n",
       "RIFLPBCIANM60NM           %  \n",
       "TERMAFCNCNSA              %  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fred.search_by_category(33058, limit=10, order_by='popularity', sort_order='desc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Strategy of Crawling Data\n",
    "Once we get the methods of downloading a specific series data, we are anxious to crawl more data, such as we want to crawl the top 10 popular data of each category.\n",
    "Here are many problems for this achievement.\n",
    "1. How many kinds of categories are there?\n",
    "2. What are the names / index of these existing category?\n",
    "3. How can I download these so many data in a rapid way?\n",
    "\n",
    "And in the order of questions, I will show you the strategy and solutions.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Establish a Tree of the Catogories Name / Index\n",
    "\n",
    "Let's go through a branch of the catogory tree by click to figure out the strategy for establishment.\n",
    "\n",
    "__The First category, let's choose money, bank and finance__\n",
    "https://fred.stlouisfed.org/categories\n",
    "<img src=\"../img/category-1.png\"  alt=\"category-1\" />\n",
    "__And we click into the href, and find that a number of 32991 is added to the tail of URL__\n",
    "https://fred.stlouisfed.org/categories/32991\n",
    "<img src=\"../img/category-2.png\"  alt=\"category-2\" />\n",
    "__We select the interest rates, and find that the number of 32991 changes into 22__\n",
    "https://fred.stlouisfed.org/categories/22\n",
    "<img src=\"../img/category-3.png\"  alt=\"category-3\" />\n",
    "__Finally, we choose the automobile loan rates, and find that the number of 22 changes into 33058__\n",
    "https://fred.stlouisfed.org/categories/33058\n",
    "<img src=\"../img/category-4.png\"  alt=\"category-4\" />\n",
    "\n",
    "And after the observation, the structure of this branch is shown here. \n",
    "\n",
    "<img src=\"../img/branch.png\" alt=\"branch\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So let's crawl the index from the website instead of clicking them manually. This is the url text of the branch.\n",
    "\n",
    "__First Order Category and Second Order Category in https://fred.stlouisfed.org/categories__\n",
    "\n",
    "<div class=\"fred-categories-group\">\n",
    "<p class=\"large fred-categories-parent\">\n",
    "    <a href=\"/categories/32991\"><strong>Money, Banking, &amp; Finance</strong></a> <span class=\"category-count\">(7,400+)</span>\n",
    "</p>\n",
    "<p class=\"fred-categories-children\">\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/22\">Interest Rates</a> <span class=\"category-count\">(810+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/15\">Exchange Rates</a> <span class=\"category-count\">(70+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/24\">Monetary Data</a> <span class=\"category-count\">(250+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/46\">Financial Indicators</a> <span class=\"category-count\">(2,100+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/23\">Banking</a> <span class=\"category-count\">(1,600+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/32360\">Business Lending</a> <span class=\"category-count\">(2,200+)</span></span>\n",
    "        <span class=\"fred-categories-child\"><a href=\"/categories/32145\">Foreign Exchange Intervention</a> <span class=\"category-count\">(3+)</span></span>\n",
    "        <br class=\"clear\">\n",
    "    <!--[if IE]><br><!endif-->\n",
    "</p>\n",
    "</div>\n",
    "\n",
    "__Third Order Category in https://fred.stlouisfed.org/categories/22__\n",
    "\n",
    "<li><a href=\"/categories/33058\">Automobile Loan Rates</a>&nbsp;(3)</li>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Version 1.0\n",
    "'''\n",
    "Crawl the first and list of second category in the mainpage, \n",
    "but some of the second category data can't be matched.\n",
    "'''\n",
    "import requests\n",
    "import re\n",
    "cateTable = []\n",
    "mainPage = requests.get('https://fred.stlouisfed.org/categories')\n",
    "# First RES for the name and index of category - 1\n",
    "fstRes = r'<p class=\"large fred-categories-parent\">\\n    <a href=\"/categories/(.*?)\"><strong>(.*?)</strong></a>(.*?)<br class=\"clear\">'\n",
    "# Second RES for the name and index of category - 2\n",
    "sndRes = r'<a href=\"/categories/(.*?)\">(.*?)</a>(.*?)<span class=\"category-count\">(.*?)</span>'\n",
    "# Third RES for the name and index of category - 3\n",
    "trdRes = r'<li><a href=\"(.*?)/categories/(.*?)\">(.*?)</a>' # Some category has a space before the '/'\n",
    "\n",
    "fstContent =  re.findall(fstRes, mainPage.text, re.S|re.M)\n",
    "for value1st in fstContent:\n",
    "    cate2 = re.findall(sndRes, value1st[2], re.S|re.M)\n",
    "    for cate2item in cate2:\n",
    "        # Give them names\n",
    "        cate1Index = value1st[0]\n",
    "        cate1Name = value1st[1]\n",
    "        cate2Index = cate2item[0]\n",
    "        cate2Name = cate2item[1]\n",
    "        cate2NumString = cate2item[3]\n",
    "        if cate2NumString == '':\n",
    "            # Some of the second category which don't have the 'category-count' is the finanl category\n",
    "            cateTable.append([int(cate1Index), cate1Name, int(cate2Index), cate2Name, int(cate2Index), cate2Name])\n",
    "        else:\n",
    "            # Get the page of category-3\n",
    "            cate3Page = requests.get(str('https://fred.stlouisfed.org/categories/'+ cate2Index))\n",
    "            # \n",
    "            trdContent =  re.findall(trdRes, cate3Page.text, re.S|re.M)\n",
    "            for value3rd in trdContent:\n",
    "                cate3Index = value3rd[1]\n",
    "                cate3Name = value3rd[2]\n",
    "                try:\n",
    "                    # There are some illegal value in the cate3Index that should be removed.\n",
    "                    # If it can't be converted into integer, we ignore this\n",
    "                    cateTable.append([int(cate1Index), cate1Name, int(cate2Index), cate2Name, int(cate3Index), cate3Name])\n",
    "                except:\n",
    "                    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Version 2.0\n",
    "'''\n",
    "Crawl three pages\n",
    "but some of the second category data can't be matched.\n",
    "'''\n",
    "import requests\n",
    "import re\n",
    "cateTable = []\n",
    "mainPage = requests.get('https://fred.stlouisfed.org/categories')\n",
    "# First RES for the name and index of category - 1\n",
    "fstRes = r'<p class=\"large fred-categories-parent\">\\n    <a href=\"/categories/(\\d+)\"><strong>(.*?)</strong>'\n",
    "# Second RES for the name and index of category - 2\n",
    "sndRes = r'<a href=\"( *)/categories/(\\d+)\">(.*?)</a>&nbsp;(.*?)</li>'\n",
    "# Third RES for the name and index of category - 3\n",
    "trdRes = r'<li><a href=\"(.*?)/categories/(\\d+)\">(.*?)</a>' # Some category has a space before the '/'\n",
    "\n",
    "fstContent =  re.findall(fstRes, mainPage.text, re.S|re.M)\n",
    "for value1st in fstContent:\n",
    "    cate1Index = value1st[0]\n",
    "    cate1Name = value1st[1]\n",
    "    # Crawl the second category page\n",
    "    cate2Page = requests.get(str('https://fred.stlouisfed.org/categories/'+ cate1Index))\n",
    "    cate2 = re.findall(sndRes, cate2Page.text, re.S|re.M)\n",
    "    for cate2item in cate2:\n",
    "        # Give them names\n",
    "        cate2Index = cate2item[1]\n",
    "        cate2Name = cate2item[2]\n",
    "        cate2NumString = cate2item[3]\n",
    "        if cate2NumString == '':\n",
    "            # Some of the second category which don't have the 'category-count' is the finanl category\n",
    "            cateTable.append([int(cate1Index), cate1Name, int(cate2Index), cate2Name, int(cate2Index), cate2Name])\n",
    "        else:\n",
    "            # Get the page of category-3\n",
    "            cate3Page = requests.get(str('https://fred.stlouisfed.org/categories/'+ cate2Index))\n",
    "            trdContent =  re.findall(trdRes, cate3Page.text, re.S|re.M)\n",
    "            for value3rd in trdContent:\n",
    "                cate3Index = value3rd[1]\n",
    "                cate3Name = value3rd[2]\n",
    "                if cate3Index == cate1Index:\n",
    "                    cateTable.append([int(cate1Index), cate1Name, int(cate2Index), cate2Name, int(cate2Index), cate2Name])\n",
    "                else:\n",
    "                    try:\n",
    "                        # There are some illegal value in the cate3Index that should be removed.\n",
    "                        # If it can't be converted into integer, we ignore this\n",
    "                        cateTable.append([int(cate1Index), cate1Name, int(cate2Index), cate2Name, int(cate3Index), cate3Name])\n",
    "                    except:\n",
    "                        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "metadata": {},
   "outputs": [],
   "source": [
    "treeDF = pd.DataFrame(cateTable, \n",
    "                      columns=['cate1Index', 'cate1Name', 'cate2Index', 'cate2Name', 'cate3Index', 'cate3Name'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cate1Index</th>\n",
       "      <th>cate1Name</th>\n",
       "      <th>cate2Index</th>\n",
       "      <th>cate2Name</th>\n",
       "      <th>cate3Index</th>\n",
       "      <th>cate3Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>220</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32217</td>\n",
       "      <td>Commodities</td>\n",
       "      <td>32217</td>\n",
       "      <td>Commodities</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>222</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>224</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32415</td>\n",
       "      <td>Food and Beverages</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32416</td>\n",
       "      <td>Housing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>226</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32417</td>\n",
       "      <td>Apparel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>227</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32418</td>\n",
       "      <td>Transportation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>228</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32419</td>\n",
       "      <td>Medical Care</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>229</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32420</td>\n",
       "      <td>Recreation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32421</td>\n",
       "      <td>Education and Communication</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32422</td>\n",
       "      <td>Other Goods and Services</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32423</td>\n",
       "      <td>Commodity and Services Groups</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32424</td>\n",
       "      <td>Special Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>33913</td>\n",
       "      <td>Cryptocurrencies</td>\n",
       "      <td>33913</td>\n",
       "      <td>Cryptocurrencies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>237</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "      <td>10</td>\n",
       "      <td>Population, Employment, &amp;amp; Labor Markets</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>33717</td>\n",
       "      <td>Health Care Indexes</td>\n",
       "      <td>33717</td>\n",
       "      <td>Health Care Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32261</td>\n",
       "      <td>House Price Indexes</td>\n",
       "      <td>32261</td>\n",
       "      <td>House Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>244</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>33583</td>\n",
       "      <td>Commodity Based</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>33584</td>\n",
       "      <td>Industry Based</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>246</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>247</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32221</td>\n",
       "      <td>Import Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32224</td>\n",
       "      <td>Export Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32231</td>\n",
       "      <td>Services Price Indexes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     cate1Index cate1Name  cate2Index                             cate2Name  \\\n",
       "220       32455    Prices       32217                           Commodities   \n",
       "222       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "224       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "225       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "226       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "227       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "228       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "229       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "230       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "231       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "232       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "233       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "234       32455    Prices       33913                      Cryptocurrencies   \n",
       "235       32455    Prices           4                 Employment Cost Index   \n",
       "237       32455    Prices           4                 Employment Cost Index   \n",
       "238       32455    Prices       33717                   Health Care Indexes   \n",
       "240       32455    Prices       32261                   House Price Indexes   \n",
       "242       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "244       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "245       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "246       32455    Prices       32220                         Trade Indexes   \n",
       "247       32455    Prices       32220                         Trade Indexes   \n",
       "248       32455    Prices       32220                         Trade Indexes   \n",
       "249       32455    Prices       32220                         Trade Indexes   \n",
       "\n",
       "     cate3Index                                    cate3Name  \n",
       "220       32217                                  Commodities  \n",
       "222           9         Consumer Price Indexes (CPI and PCE)  \n",
       "224       32415                           Food and Beverages  \n",
       "225       32416                                      Housing  \n",
       "226       32417                                      Apparel  \n",
       "227       32418                               Transportation  \n",
       "228       32419                                 Medical Care  \n",
       "229       32420                                   Recreation  \n",
       "230       32421                  Education and Communication  \n",
       "231       32422                     Other Goods and Services  \n",
       "232       32423                Commodity and Services Groups  \n",
       "233       32424                              Special Indexes  \n",
       "234       33913                             Cryptocurrencies  \n",
       "235           4                        Employment Cost Index  \n",
       "237          10  Population, Employment, &amp; Labor Markets  \n",
       "238       33717                          Health Care Indexes  \n",
       "240       32261                          House Price Indexes  \n",
       "242          31                 Producer Price Indexes (PPI)  \n",
       "244       33583                              Commodity Based  \n",
       "245       33584                               Industry Based  \n",
       "246       32220                                Trade Indexes  \n",
       "247       32221                         Import Price Indexes  \n",
       "248       32224                         Export Price Indexes  \n",
       "249       32231                       Services Price Indexes  "
      ]
     },
     "execution_count": 311,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "print np.shape(cateTable)\n",
    "treeDF[treeDF.loc[:,'cate1Name']==\"Prices\"].drop_duplicates()\n",
    "#treeDF[treeDF['cate2Index'] == 5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 312,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save the table into csv file\n",
    "import sys\n",
    "reload(sys)\n",
    "sys.setdefaultencoding('utf8')\n",
    "# Convert the name to utf8 string\n",
    "treeDF['cate1Name'] = treeDF['cate1Name'].astype(str)\n",
    "treeDF['cate2Name'] = treeDF['cate2Name'].astype(str)\n",
    "treeDF['cate3Name'] = treeDF['cate3Name'].astype(str)\n",
    "treeDF = treeDF.drop_duplicates()\n",
    "treeDF.to_csv('../data/cateTree.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 313,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cate1Index</th>\n",
       "      <th>cate1Name</th>\n",
       "      <th>cate2Index</th>\n",
       "      <th>cate2Name</th>\n",
       "      <th>cate3Index</th>\n",
       "      <th>cate3Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>220</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32217</td>\n",
       "      <td>Commodities</td>\n",
       "      <td>32217</td>\n",
       "      <td>Commodities</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>222</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>224</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32415</td>\n",
       "      <td>Food and Beverages</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32416</td>\n",
       "      <td>Housing</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>226</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32417</td>\n",
       "      <td>Apparel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>227</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32418</td>\n",
       "      <td>Transportation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>228</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32419</td>\n",
       "      <td>Medical Care</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>229</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32420</td>\n",
       "      <td>Recreation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32421</td>\n",
       "      <td>Education and Communication</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32422</td>\n",
       "      <td>Other Goods and Services</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>232</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32423</td>\n",
       "      <td>Commodity and Services Groups</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>9</td>\n",
       "      <td>Consumer Price Indexes (CPI and PCE)</td>\n",
       "      <td>32424</td>\n",
       "      <td>Special Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>234</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>33913</td>\n",
       "      <td>Cryptocurrencies</td>\n",
       "      <td>33913</td>\n",
       "      <td>Cryptocurrencies</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>237</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>4</td>\n",
       "      <td>Employment Cost Index</td>\n",
       "      <td>10</td>\n",
       "      <td>Population, Employment, &amp;amp; Labor Markets</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>33717</td>\n",
       "      <td>Health Care Indexes</td>\n",
       "      <td>33717</td>\n",
       "      <td>Health Care Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>240</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32261</td>\n",
       "      <td>House Price Indexes</td>\n",
       "      <td>32261</td>\n",
       "      <td>House Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>242</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>244</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>33583</td>\n",
       "      <td>Commodity Based</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>245</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>31</td>\n",
       "      <td>Producer Price Indexes (PPI)</td>\n",
       "      <td>33584</td>\n",
       "      <td>Industry Based</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>246</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>247</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32221</td>\n",
       "      <td>Import Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>248</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32224</td>\n",
       "      <td>Export Price Indexes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>249</th>\n",
       "      <td>32455</td>\n",
       "      <td>Prices</td>\n",
       "      <td>32220</td>\n",
       "      <td>Trade Indexes</td>\n",
       "      <td>32231</td>\n",
       "      <td>Services Price Indexes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     cate1Index cate1Name  cate2Index                             cate2Name  \\\n",
       "220       32455    Prices       32217                           Commodities   \n",
       "222       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "224       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "225       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "226       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "227       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "228       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "229       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "230       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "231       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "232       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "233       32455    Prices           9  Consumer Price Indexes (CPI and PCE)   \n",
       "234       32455    Prices       33913                      Cryptocurrencies   \n",
       "235       32455    Prices           4                 Employment Cost Index   \n",
       "237       32455    Prices           4                 Employment Cost Index   \n",
       "238       32455    Prices       33717                   Health Care Indexes   \n",
       "240       32455    Prices       32261                   House Price Indexes   \n",
       "242       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "244       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "245       32455    Prices          31          Producer Price Indexes (PPI)   \n",
       "246       32455    Prices       32220                         Trade Indexes   \n",
       "247       32455    Prices       32220                         Trade Indexes   \n",
       "248       32455    Prices       32220                         Trade Indexes   \n",
       "249       32455    Prices       32220                         Trade Indexes   \n",
       "\n",
       "     cate3Index                                    cate3Name  \n",
       "220       32217                                  Commodities  \n",
       "222           9         Consumer Price Indexes (CPI and PCE)  \n",
       "224       32415                           Food and Beverages  \n",
       "225       32416                                      Housing  \n",
       "226       32417                                      Apparel  \n",
       "227       32418                               Transportation  \n",
       "228       32419                                 Medical Care  \n",
       "229       32420                                   Recreation  \n",
       "230       32421                  Education and Communication  \n",
       "231       32422                     Other Goods and Services  \n",
       "232       32423                Commodity and Services Groups  \n",
       "233       32424                              Special Indexes  \n",
       "234       33913                             Cryptocurrencies  \n",
       "235           4                        Employment Cost Index  \n",
       "237          10  Population, Employment, &amp; Labor Markets  \n",
       "238       33717                          Health Care Indexes  \n",
       "240       32261                          House Price Indexes  \n",
       "242          31                 Producer Price Indexes (PPI)  \n",
       "244       33583                              Commodity Based  \n",
       "245       33584                               Industry Based  \n",
       "246       32220                                Trade Indexes  \n",
       "247       32221                         Import Price Indexes  \n",
       "248       32224                         Export Price Indexes  \n",
       "249       32231                       Services Price Indexes  "
      ]
     },
     "execution_count": 313,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "treeDF[treeDF.loc[:,'cate1Name']==\"Prices\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the Top 10 Popular Series Data According to the Catogery Index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method - 1: Download with Single Thread\n",
    "This method spend very long time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from fredapi import Fred\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "import numpy as np\n",
    "fred = Fred(api_key='APIkey')\n",
    "treeDF = pd.read_csv('../data/cateTree.csv', index_col=0)\n",
    "treeDF.index = range(treeDF.shape[0])\n",
    "allData = []\n",
    "dataInfo = []\n",
    "error = []\n",
    "treeDF.index = range(treeDF.shape[0])\n",
    "for index in tqdm(range(treeDF.shape[0])):\n",
    "    try:\n",
    "        popu = fred.search_by_category(treeDF.loc[index, 'cate3Index'], limit=10, order_by='popularity', sort_order='desc')\n",
    "        for detailIndex, detialValues in popu.iterrows():\n",
    "            allData.append(pd.DataFrame(fred.get_series(detailIndex), columns=[detailIndex]))\n",
    "        allDataIn = pd.concat(allData, axis=1)\n",
    "        allDataIn.to_csv('../data/allData.csv')\n",
    "    except:\n",
    "        print treeDF.loc[index, 'cate3Index'], 'can not be found'\n",
    "        error.append(treeDF.loc[index, 'cate3Index'])\n",
    "        np.savetxt('../data/errorLog.csv', error, delimiter=',')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Method - 2: Download with Multi Thread\n",
    "We try to use multi thread to crawling the data\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time, threading\n",
    "\n",
    "from fredapi import Fred\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "import numpy as np\n",
    "fred = Fred(api_key='APIkey')\n",
    "treeDF = pd.read_csv('../data/cateTree.csv', index_col=0)\n",
    "treeDF.index = range(treeDF.shape[0])\n",
    "# Divide the task into multi threads\n",
    "totalWork = treeDF.shape[0]\n",
    "threadNum = 10\n",
    "# How many data to be crawled in the single thread\n",
    "singleWork = int((totalWork)/threadNum)+1\n",
    "\n",
    "def download(rangeCato, name):\n",
    "    print 'thread %s is running...' % threading.current_thread().name\n",
    "    allData = []\n",
    "    dataInfo = []\n",
    "    error = []\n",
    "    for index in rangeCato:\n",
    "        if index < treeDF.shape[0]:\n",
    "            try:\n",
    "                popu = fred.search_by_category(treeDF.loc[index, 'cate3Index'], limit=10, order_by='popularity', sort_order='desc')\n",
    "                for detailIndex, detialValues in popu.iterrows():\n",
    "                    allData.append(pd.DataFrame(fred.get_series(detailIndex), columns=[detailIndex]))\n",
    "                allDataIn = pd.concat(allData, axis=1)\n",
    "                allDataIn.to_csv('../data/allData'+name+'.csv')\n",
    "            except:\n",
    "                print treeDF.loc[index, 'cate3Index'], 'can not be found'\n",
    "                error.append(treeDF.loc[index, 'cate3Index'])\n",
    "                np.savetxt('errorLog'+name+'.csv',error,delimiter=',')\n",
    "    print 'thread %s ended.' % threading.current_thread().name\n",
    "\n",
    "threadList = [threading.Thread(target=download, args=(range(index*singleWork, (index+1)*singleWork), str(index),), name='DownloadThread'+str(index)) for index in range(threadNum)]\n",
    "\n",
    "for index, value in enumerate(threadList):\n",
    "    value.start()\n",
    "for index, value in enumerate(threadList):\n",
    "    value.join()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"../img/multi-thread.png\" alt=\"multi-thread\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Concat the data file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "allData = []\n",
    "threadNum = 10\n",
    "for index in range(threadNum):\n",
    "    allData.append(pd.read_csv('../data/threadData/allData'+str(index)+'.csv', index_col=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "allDF = pd.concat(allData, axis=1)\n",
    "allDF.to_csv('../data/allData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(24092, 829)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "allDF.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Generate the Information Table of Each Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time, threading\n",
    "\n",
    "from fredapi import Fred\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "import numpy as np\n",
    "fred = Fred(api_key='APIkey')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Info = []\n",
    "for value in tqdm(allDF.columns):\n",
    "    Info.append(fred.get_series_info(value))\n",
    "# This spend much time. Let's combine the work of crawling information into section 3.2.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>frequency</th>\n",
       "      <th>frequency_short</th>\n",
       "      <th>id</th>\n",
       "      <th>last_updated</th>\n",
       "      <th>notes</th>\n",
       "      <th>observation_end</th>\n",
       "      <th>observation_start</th>\n",
       "      <th>popularity</th>\n",
       "      <th>realtime_end</th>\n",
       "      <th>realtime_start</th>\n",
       "      <th>seasonal_adjustment</th>\n",
       "      <th>seasonal_adjustment_short</th>\n",
       "      <th>title</th>\n",
       "      <th>units</th>\n",
       "      <th>units_short</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>TERMCBAUTO48NS</td>\n",
       "      <td>2018-07-09 14:51:02-05</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2018-05-01</td>\n",
       "      <td>1972-02-01</td>\n",
       "      <td>66</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>Finance Rate on Consumer Installment Loans at ...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>RIFLPBCIANM60NM</td>\n",
       "      <td>2018-07-09 15:01:02-05</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2018-05-01</td>\n",
       "      <td>2006-08-01</td>\n",
       "      <td>52</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>Finance Rate on Consumer Installment Loans at ...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>TERMAFCNCNSA</td>\n",
       "      <td>2012-06-26 12:01:23-05</td>\n",
       "      <td>For further information, please refer to the B...</td>\n",
       "      <td>2011-01-01</td>\n",
       "      <td>1971-06-01</td>\n",
       "      <td>38</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>New Car Average Finance Rate at Auto Finance C...</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>BA3M</td>\n",
       "      <td>2006-06-07 15:42:44-05</td>\n",
       "      <td>Averages of business days, discount basis.</td>\n",
       "      <td>2000-06-01</td>\n",
       "      <td>1941-01-01</td>\n",
       "      <td>31</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>3-Month Bankers Acceptance Rate (DISCONTINUED)</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Monthly</td>\n",
       "      <td>M</td>\n",
       "      <td>BA6M</td>\n",
       "      <td>2006-06-07 15:42:45-05</td>\n",
       "      <td>Averages of business days, discount basis.</td>\n",
       "      <td>2000-06-01</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>18</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>2018-09-28</td>\n",
       "      <td>Not Seasonally Adjusted</td>\n",
       "      <td>NSA</td>\n",
       "      <td>6-Month Bankers Acceptance Rate (DISCONTINUED)</td>\n",
       "      <td>Percent</td>\n",
       "      <td>%</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  frequency frequency_short               id            last_updated  \\\n",
       "0   Monthly               M   TERMCBAUTO48NS  2018-07-09 14:51:02-05   \n",
       "1   Monthly               M  RIFLPBCIANM60NM  2018-07-09 15:01:02-05   \n",
       "2   Monthly               M     TERMAFCNCNSA  2012-06-26 12:01:23-05   \n",
       "3   Monthly               M             BA3M  2006-06-07 15:42:44-05   \n",
       "4   Monthly               M             BA6M  2006-06-07 15:42:45-05   \n",
       "\n",
       "                                               notes observation_end  \\\n",
       "0  For further information, please refer to the B...      2018-05-01   \n",
       "1  For further information, please refer to the B...      2018-05-01   \n",
       "2  For further information, please refer to the B...      2011-01-01   \n",
       "3         Averages of business days, discount basis.      2000-06-01   \n",
       "4         Averages of business days, discount basis.      2000-06-01   \n",
       "\n",
       "  observation_start popularity realtime_end realtime_start  \\\n",
       "0        1972-02-01         66   2018-09-28     2018-09-28   \n",
       "1        2006-08-01         52   2018-09-28     2018-09-28   \n",
       "2        1971-06-01         38   2018-09-28     2018-09-28   \n",
       "3        1941-01-01         31   2018-09-28     2018-09-28   \n",
       "4        1976-01-01         18   2018-09-28     2018-09-28   \n",
       "\n",
       "       seasonal_adjustment seasonal_adjustment_short  \\\n",
       "0  Not Seasonally Adjusted                       NSA   \n",
       "1  Not Seasonally Adjusted                       NSA   \n",
       "2  Not Seasonally Adjusted                       NSA   \n",
       "3  Not Seasonally Adjusted                       NSA   \n",
       "4  Not Seasonally Adjusted                       NSA   \n",
       "\n",
       "                                               title    units units_short  \n",
       "0  Finance Rate on Consumer Installment Loans at ...  Percent           %  \n",
       "1  Finance Rate on Consumer Installment Loans at ...  Percent           %  \n",
       "2  New Car Average Finance Rate at Auto Finance C...  Percent           %  \n",
       "3     3-Month Bankers Acceptance Rate (DISCONTINUED)  Percent           %  \n",
       "4     6-Month Bankers Acceptance Rate (DISCONTINUED)  Percent           %  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat(Info, axis=1).T.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final Version of the Crawling Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time, threading\n",
    "\n",
    "from fredapi import Fred\n",
    "import pandas as pd\n",
    "from tqdm import tqdm\n",
    "import numpy as np\n",
    "fred = Fred(api_key='APIkey')\n",
    "treeDF = pd.read_csv('cateTree.csv', index_col=0)\n",
    "treeDF.index = range(treeDF.shape[0])\n",
    "# Divide the task into multi threads\n",
    "totalWork = treeDF.shape[0]\n",
    "threadNum = 10\n",
    "# How many data to be crawled in the single thread\n",
    "singleWork = int((totalWork)/threadNum)+1\n",
    "\n",
    "def download(rangeCato, name):\n",
    "    print 'thread %s is running...' % threading.current_thread().name\n",
    "    allData = []\n",
    "    dataInfo = []\n",
    "    error = []\n",
    "    for index in rangeCato:\n",
    "        if index < treeDF.shape[0]:\n",
    "            try:\n",
    "                popu = fred.search_by_category(treeDF.loc[index, 'cate3Index'], limit=10, order_by='popularity', sort_order='desc')\n",
    "                for detailIndex, detialValues in popu.iterrows():\n",
    "                    dataInfo.append(fred.get_series_info(detailIndex))\n",
    "                allData.append(pd.DataFrame(fred.get_series(detailIndex), columns=[detailIndex]))\n",
    "                allDataIn = pd.concat(allData, axis=1)\n",
    "                dataInfoIn = pd.concat(dataInfo, axis=1)\n",
    "                allDataIn.to_csv('allData'+name+'.csv')\n",
    "                dataInfoIn.to_csv('info'+name+'.csv')\n",
    "            except:\n",
    "                print treeDF.loc[index, 'cate3Index'], 'can not be found'\n",
    "                error.append(treeDF.loc[index, 'cate3Index'])\n",
    "                np.savetxt('errorLog'+name+'.csv',error,delimiter=',')\n",
    "    print 'thread %s ended.' % threading.current_thread().name\n",
    "\n",
    "threadList = [threading.Thread(target=download, args=(range(index*singleWork, (index+1)*singleWork), str(index),), name='DownloadThread'+str(index)) for index in range(threadNum)]\n",
    "\n",
    "for index, value in enumerate(threadList):\n",
    "    value.start()\n",
    "for index, value in enumerate(threadList):\n",
    "    value.join()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concat All The Data and Information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 345,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "allData = []\n",
    "threadNum = 20\n",
    "for index in range(threadNum):\n",
    "    allData.append(pd.read_csv('../data/threadData/allData'+str(index)+'.csv', index_col=0))\n",
    "allDF = pd.concat(allData, axis=1)\n",
    "# allDF.to_csv('../data/allData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "metadata": {},
   "outputs": [],
   "source": [
    "allDF.to_csv('../data/allData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "infoData = []\n",
    "\n",
    "for index in range(threadNum):\n",
    "    itemPD = pd.read_csv('../data/threadData/info'+str(index)+'.csv', index_col = 0)\n",
    "    itemPD = itemPD.T\n",
    "    itemPD.index = itemPD['id']\n",
    "    itemPD = itemPD.drop(['id'], 1)\n",
    "    infoData.append(itemPD)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 348,
   "metadata": {},
   "outputs": [],
   "source": [
    "infoDF = pd.concat(infoData, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 354,
   "metadata": {},
   "outputs": [],
   "source": [
    "infoDF.to_csv('../data/infoAll.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 349,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5959, 14)"
      ]
     },
     "execution_count": 349,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "infoDF.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 356,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(61812, 5839)"
      ]
     },
     "execution_count": 356,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#allDF.to_csv('../data/allData.csv')\n",
    "allDF.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 350,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataCate = []\n",
    "\n",
    "for index in range(threadNum):\n",
    "    dataCatePD = pd.read_csv('../data/threadData/dataCate'+str(index)+'.csv', index_col = 0)\n",
    "    dataCatePD.index = dataCatePD['itemID']\n",
    "    dataCatePD = dataCatePD.drop(['itemID'], 1)\n",
    "    dataCate.append(dataCatePD)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 351,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataCatePD = pd.concat(dataCate, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 355,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataCatePD.to_csv('../data/dataCate.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 353,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5841, 6)"
      ]
     },
     "execution_count": 353,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataCatePD.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Draw the Tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "tree = pd.read_csv('../data/cateTree.csv', index_col = 0)\n",
    "#tree.loc[tree.shape[0]] = ['end','end','end','end','end','end']\n",
    "tree.index = range(tree.shape[0])\n",
    "tree.loc[tree.shape[0]] = ['end','end','end','end','end','end']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [],
   "source": [
    "node3List = []\n",
    "node2List = []\n",
    "node1List = []\n",
    "lastRow = tree.iloc[0]\n",
    "\n",
    "for index, row in tree.iterrows():\n",
    "    node3 = {\"name\": row['cate3Name'], \"parent\": row['cate2Name']}\n",
    "    if lastRow['cate2Name'] != row['cate2Name']:\n",
    "        node2 = {\"name\": lastRow['cate2Name'], \"parent\": lastRow['cate1Name'], \"children\": node3List}\n",
    "        node2List.append(node2)     \n",
    "        if lastRow['cate1Name'] != row['cate1Name']:\n",
    "            node1 = {\"name\": lastRow['cate1Name'], \"parent\": \"Fred\", \"children\": node2List}\n",
    "            if row['cate1Name'] != 'end':\n",
    "                node1List.append(node1)\n",
    "            node2List = []\n",
    "            lastRow = row\n",
    "        node3List = []\n",
    "        lastRow = row\n",
    "    if row['cate2Name'] != 'end':\n",
    "        node3List.append(node3)\n",
    "    '''\n",
    "    if row['cate3Name']\n",
    "    node3List.append(node3)\n",
    "    node2 = {\"name\": row['cate2Name'], \"parent\": row['cate1Name'], \"children\": node3List}\n",
    "    node2List.append(node2)\n",
    "    node1 = {\"name\": row['cate1Name'], \"parent\": \"Fred\", \"children\": node2List}\n",
    "    \n",
    "    last3Name = row['cate3Name']\n",
    "    '''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 343,
   "metadata": {},
   "outputs": [],
   "source": [
    "dictObj = {\"name\": \"Fred\", \"parent\": \"null\", \"children\": node1List}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 344,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import json\n",
    "jsObj = json.dumps(dictObj) \n",
    "fileObject = open('jsonFile.json', 'w')\n",
    "fileObject.write(jsObj)\n",
    "fileObject.close()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "cate1Index                  3008\n",
       "cate1Name     U.S. Regional Data\n",
       "cate2Index                 32061\n",
       "cate2Name            BEA Regions\n",
       "cate3Index                 32066\n",
       "cate3Name                 Plains\n",
       "Name: 602, dtype: object"
      ]
     },
     "execution_count": 326,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tree.loc[602]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prepare the data for a round tree chart with d3.js-v4\n",
    "# Remove the parent element\n",
    "node3List = []\n",
    "node2List = []\n",
    "node1List = []\n",
    "lastRow = tree.iloc[0]\n",
    "\n",
    "for index, row in tree.iterrows():\n",
    "    node3 = {\"name\": row['cate3Name']}\n",
    "    if lastRow['cate2Name'] != row['cate2Name']:\n",
    "        node2 = {\"name\": lastRow['cate2Name'], \"children\": node3List}\n",
    "        node2List.append(node2)     \n",
    "        if lastRow['cate1Name'] != row['cate1Name']:\n",
    "            node1 = {\"name\": lastRow['cate1Name'], \"children\": node2List}\n",
    "            if row['cate1Name'] != 'end':\n",
    "                node1List.append(node1)\n",
    "            node2List = []\n",
    "            lastRow = row\n",
    "        node3List = []\n",
    "        lastRow = row\n",
    "    if row['cate2Name'] != 'end':\n",
    "        node3List.append(node3)\n",
    "dictObj = {\"name\": \"Fred\",\"children\": node1List}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "jsObj = json.dumps(dictObj) \n",
    "fileObject = open('treeRound.json', 'w')\n",
    "fileObject.write(jsObj)\n",
    "fileObject.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "384px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}