Multiple_Linear_Regression_with_Regularization.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_WYgBy_zmJE"
      },
      "source": [
        "# Multiple Linear Regression\n",
        "\n",
        "This tutorial demonstrate implementation of multiple linear regression. \n",
        "\n",
        "The key learnings of this tutorial are as follows:\n",
        "\n",
        "1. Modeling Linear regression\n",
        "            - Key learnings\n",
        "                    * loading data set\n",
        "                    * loading libraries from scikit learn\n",
        "                    * creating training and test set enviornment\n",
        "                    * implement linear regression model\n",
        "                    * evaluate linear regression model\n",
        "                    * printing linear regression model in form of equation\n",
        "                    * analysing results\n",
        "                    \n",
        "2.  Applying Hypothesis testing\n",
        "            - Key learnings\n",
        "                    * applying Hypothesis testing\n",
        "                    * analysing results\n",
        "\n",
        "3. Overcoming Collinearity, model overfitting and complexity using Regularization    \n",
        "            - Key learnings\n",
        "                    * understanding effect of Collinearity on linear regression model\n",
        "                    * analysing correlation among attributes\n",
        "                    * practical understanding on output of linear regression model in\n",
        "                      presence of correlated festures\n",
        "                    * implement, analyse Ridge regularization to avoid  collinearity,    \n",
        "                      model overfitting and model complexity\n",
        "                    * implement, analyse Lasso regularization to avoid  collinearity,    \n",
        "                      model overfitting and model complexity\n",
        "                    * discovering relevant features using Lasso model \n",
        "                    * implement, analyse Elasticnet regularization to avoid  collinearity                       , model overfitting and model complexity\n",
        "                    * Analysing results of regularization\n",
        "                    * comparing results of regularization with linear regression model\n",
        "               \n",
        "4. Hyperparamter tuning via cross validation                     \n",
        "            - Key learnings\n",
        "                    * applying cross validation\n",
        "                    * tunning parameters of regularization techniques using cros  \n",
        "                      validation\n",
        " \n",
        "The data set used for demonstration is Moneyball which can downloaded form https://www.kaggle.com/wduckett/moneyball-mlb-stats-19622012/data . The data has been gathered from baseball-reference.com. It contains following features:\n",
        "\n",
        "1. RA: runs allowed\n",
        "2. RS:  runs scored\n",
        "3. OBP: On Base Percentage\n",
        "4. SLG: Slugging Percentage\n",
        "5. BA: Batting Average\n",
        "6. OOBP: opponent’s OBP\n",
        "7. OSLG: opponent’s SLG\n",
        "8. W:  wins in that season\n",
        "\n",
        "The features from 1-7 are used as indicator variables to predict the outcome W(i.e., wins in season). \n",
        "\n",
        "The step by step practical learning on implementing and analysing multiple linear regression to predict W is demonstrated below.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FpW-nH8dzmJK"
      },
      "source": [
        "# 1. Loading libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9ixQpzSnzmJK"
      },
      "outputs": [],
      "source": [
        "import pandas as pd # importing pandas\n",
        "import numpy as np  # importing mumpy\n",
        "from sklearn import linear_model   # imports linear_model from scikit learn\n",
        "from sklearn import metrics # for model evaluation\n",
        "from sklearn.model_selection import train_test_split # using scikit learn for hold-out\n",
        "import matplotlib.pyplot as plt # for visualization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ys1WGZoXzmJM"
      },
      "source": [
        "# 2. Loading data set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "brSW8IDZzmJM"
      },
      "outputs": [],
      "source": [
        "# Loading data set from local machine.\n",
        "dataset =pd.read_csv('moneyball.csv')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FBXdnyYjzmJM"
      },
      "source": [
        "## 3. Applying Hold-out method to create Train and Test split"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OrHGeRwBzmJM"
      },
      "outputs": [],
      "source": [
        "# My_data contains all data points from My_data set from from first feature to 6th feature(indicator features)\n",
        "My_data = dataset.iloc[:,0:7] \n",
        "\n",
        "# My_target contains class information which is 7th feature in the data set of \n",
        "\n",
        "My_data_target=dataset.iloc[:,7]\n",
        "\n",
        "\n",
        "X_train, X_test, Y_train, Y_test = train_test_split(My_data, My_data_target, test_size=0.7, random_state=10)\n",
        "\n",
        "\n",
        "#print(My_data.head())\n",
        "\n",
        "#print(My_data_target.head())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0U7QAkDtzmJN"
      },
      "source": [
        "## 4.  Building Multiple Linear Regression model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mfx9YG_JzmJN",
        "outputId": "cbf26814-fdc4-42b0-ecc2-1c586e7df2a5"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/base.py:503: RuntimeWarning: internal gelsd driver lwork query error, required iwork dimension not returned. This is likely the result of LAPACK bug 0038, fixed in LAPACK 3.2.2 (released July 21, 2010). Falling back to 'gelss' driver.\n",
            "  linalg.lstsq(X, y)\n"
          ]
        }
      ],
      "source": [
        "#Create a Mulitple Linear Regression model.\n",
        "\n",
        "LR_model = linear_model.LinearRegression()\n",
        "\n",
        "#Train the model using the training sets\n",
        "\n",
        "LRfitted = LR_model.fit(X_train, Y_train)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FnAqGzAuzmJO"
      },
      "source": [
        "## 4.1 Understanding the learnt Multiple Linear Regression model\n",
        "\n",
        "1. Getting the intercept \n",
        "2. Getting the coefficients/weights \n",
        "3. Printing the  Multiple Linear Regression model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DA2smcXnzmJO",
        "outputId": "c8b48d4e-022e-4b65-9335-a51bb558eff3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The intercept of the model :  70.86405009579792\n",
            "\n",
            "\n",
            "The coefficients learned against each feature present in the data set \n",
            "\n",
            "  Feature_name  Coefficients\n",
            "0           RS      0.024529\n",
            "1           RA     -0.038842\n",
            "2          OBP     41.828077\n",
            "3          SLG     21.093629\n",
            "4           BA     -8.165570\n",
            "5     Playoffs      3.691844\n",
            "6           RD      0.063371\n",
            "\n",
            "\n",
            "The linear regression model \n",
            "\n",
            "W = 70.86 +  RA X 0.02 + OBP X -0.04 + SLG X 41.83 + BA X 21.09 + Playoff X -8.17 + RD X 3.69 + W X 0.06\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# printing the intercept of the model learned\n",
        "\n",
        "print(\"The intercept of the model : \", LRfitted.intercept_)\n",
        "\n",
        "# printing the coefficients of the model learned against features present in the data set\n",
        "print(\"\\n\")\n",
        "Feature_names = list(My_data.columns.values)\n",
        "table_coeff= pd.DataFrame({'Feature_name':Feature_names , 'Coefficients': LRfitted.coef_})  \n",
        "print(\"The coefficients learned against each feature present in the data set \\n\")\n",
        "print(table_coeff)\n",
        "\n",
        "\n",
        "\n",
        "# printing the Linear regression model\n",
        "print(\"\\n\")\n",
        "print(\"The linear regression model \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(LRfitted.intercept_,2), \"+\", \" RA\" \" X\", round(LRfitted.coef_[0],2)\n",
        "     , \"+ OBP\" \" X\"  ,round(LRfitted.coef_[1],2) , \"+ SLG\" \" X\"  ,round(LRfitted.coef_[2],2)\n",
        "    , \"+ BA\" \" X\"  ,round(LRfitted.coef_[3],2) , \"+ Playoff\" \" X\"  ,round(LRfitted.coef_[4],2)\n",
        "     , \"+ RD\" \" X\"  ,round(LRfitted.coef_[5],2), \"+ W\" \" X\"  ,round(LRfitted.coef_[6],2)\n",
        "     )\n",
        "print(\"\\n\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F3LEB8fTzmJO"
      },
      "source": [
        "\n",
        "## 5.  Hypothesis Testing\n",
        "\n",
        "This test helps to find the importance of variables( significance) with respect to the hypothesis. To do this, we need to calculate the p value for each variable and if it is less than the desired cutoff( 0.05 is the general cut off for 95% significance) then we can say with confidence that a variable is significant. We can calculate the p-value using another library called ‘statsmodels’."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vz_8Iz3UzmJP",
        "outputId": "76b85bca-d7e3-4a42-9d4f-29aaf6ebdf0d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "RS          0.01\n",
            "RA          0.00\n",
            "OBP         0.00\n",
            "SLG         0.00\n",
            "BA          0.37\n",
            "Playoffs    0.00\n",
            "RD          0.00\n",
            "dtype: float64\n"
          ]
        }
      ],
      "source": [
        "import statsmodels.api as sm\n",
        "\n",
        "\n",
        "model = sm.OLS(Y_train, X_train)\n",
        "\n",
        "Statsmodel = model.fit()\n",
        "print(round(Statsmodel.pvalues,2))\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vl8L_zpHzmJP"
      },
      "source": [
        "The hypothesis testing reveals that all features other than BA  are statically significant"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DBN4NeW_zmJP"
      },
      "source": [
        "## 5. Apply Model to the Test Set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qf-nJtX8zmJP"
      },
      "outputs": [],
      "source": [
        "Prediction_test = LRfitted.predict(X_test)\n",
        "Prediction_train = LRfitted.predict(X_train)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FvmUsK4IzmJP"
      },
      "source": [
        "## 6. Evaluate Model Performance on Test set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M8etmjowzmJQ",
        "outputId": "e4c677e9-2039-4dc7-b1c3-664558c0548f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The coefficients for each indicator feature in data set are\n",
            "     Actual   Predicted\n",
            "437      89   86.145600\n",
            "131      84   96.168832\n",
            "633      82   82.632314\n",
            "195      68   67.280907\n",
            "230      94   94.471001\n",
            "731      89   89.677985\n",
            "75       74   73.464351\n",
            "513      74   83.075028\n",
            "353      80   78.518031\n",
            "546      79   74.532047\n",
            "584     100  102.520583\n",
            "43       77   74.239904\n",
            "605      92   87.210294\n",
            "832      97   92.862680\n",
            "242      83   85.244639\n",
            "634      83   90.119332\n",
            "176     104  105.150550\n",
            "171      85   89.843636\n",
            "688      81   84.759397\n",
            "899     103  102.501339\n",
            "644      84   84.205446\n",
            "294      81   82.851394\n",
            "277      77   77.297412\n",
            "750      97  101.469155\n",
            "425      81   80.285909\n",
            "887      80   73.897726\n",
            "849      66   66.880678\n",
            "449      91   89.049624\n",
            "700      85   84.512430\n",
            "1        92   98.666981\n",
            "..      ...         ...\n",
            "532      89   86.807573\n",
            "359      96   99.456046\n",
            "187      85   89.657080\n",
            "207      72   70.909260\n",
            "165      71   76.852171\n",
            "611      87   89.157063\n",
            "809      81   79.503607\n",
            "269      74   74.402068\n",
            "696      79   81.288865\n",
            "288      75   74.500695\n",
            "626      57   58.905280\n",
            "704      71   69.719994\n",
            "251      83   83.394992\n",
            "512      75   74.532745\n",
            "36       95   97.235475\n",
            "394      84   84.800217\n",
            "297      87   89.204310\n",
            "19       95   93.316249\n",
            "310      89   97.075461\n",
            "553      95  100.353442\n",
            "267      86   84.477814\n",
            "112      69   73.434027\n",
            "625      79   79.299225\n",
            "291      86   77.480392\n",
            "808      76   79.071582\n",
            "795      61   62.909936\n",
            "189      81   81.906440\n",
            "247      84   78.376706\n",
            "5        88   88.504824\n",
            "563      56   58.270960\n",
            "\n",
            "[632 rows x 2 columns]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "Text(0,0.5,'Predicted values')"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm4XGWZ7/3vb+8kQBKmbJBmyg4KL84gpEVexaGDLXC0QQ4qdGwDqDmAfYnag9AcW/vtkz7qsU83PYhGBoMJKk4tdvOiiKI2Th0UGeWAEiDIEAIIIbaB5D5/PKvYa1dWVa2ah/37XFddVbXWqrWeql173bWe4X4UEZiZmVUb63cBzMxsMDlAmJlZIQcIMzMr5ABhZmaFHCDMzKyQA4SZmRVygBhAkl4taX0br/+EpA90skzZfj8kaXWn92vDRdKnJf2P7PFRkm7v0XFD0oEd3ueibL+zOrnfUeEA0SWS1kn6jaRNkh7I/qnmd+E4p0r69/yyiDgjIv6608dqxzAEl/yJz8qJiO9FxMGNtiv6ntrgc4DorjdExHzgUOAlwLl9Lo+1YRR/ZY7ie7LOcYDogYh4APg6KVAAIGkHSR+TdI+kB7NqoZ2KXi/pHEm/kPSEpFslvTFb/jzgE8CR2ZXKY9nyab+EJb1T0p2SHpF0haR9cutC0hmS7pD0mKR/lqQ6b2dHSZ/PyvITSYfk9rWPpC9J2iDpLknvzpYfA/wF8JasnD+T9BpJN+Vee7Wk/8g9/56kE+rtN1s3lvt8Nkq6XNKCbF2l+mBZ9jk/LOm8Gp/xcmAp8OdZGb+WLV8n6f2SbgSelDSruqqj4PN+vaQbss/z+5JeXOOYF0j6WNWyr0p6X/b4/ZLuyz7r2yUtqfN3ye9jnaRzs+/Ko5IukbRjtu7VktZn+34AuKRRmSW9JPtbPyHp88COuXXTqkMl7S/py9nfaqOkf6rzPa37PyDpzyTdL+lXkk6v837fImlt1bL3Sroie/xfJP1U0uOS7pX0oQaf3dG559OufCW9LPt8Hsu+x6/OrTtV0i+zz+kuSUtrHWdoRIRvXbgB64Cjs8f7ATcB5+fW/x1wBbAA2Bn4GvA/s3WvBtbntn0TsA8poL8FeBLYO1t3KvDvVcf+NPA/sse/BzwMHAbsAPwj8N3ctgH8K7AbsBDYABxT4z19CHgKOAmYDfwpcFf2eAy4HvhLYA7wbOCXwOtyr12d29dOwH8Ce2SvfxC4L/ssdgJ+A0yU2O/ZwA+zz3gH4JPAZ7N1i7L396lsn4cAvwWeV+P9PfO5Vf0dbwD2B3bKfWYH1vi8XwI8BBwBjAPLsn3sUHC8VwL3Asqe7569732Ag7N1++Tey3Oa+O7dnJV5AXBdrnyvBp4GPpJ9XjvVK3P2md8NvDf7O52UfQfy+1ufPR4Hfkb6bs8jBZJX1Pme1vsfOCb7Trww29dl1Z97bj9zgSeAg3LL/gM4OVfGF5G+Sy/O9ntC1XdkVvX/bfX3FtgX2Agcl+3rtdnzPbMyPg4cnG27N/CCfp+H2j6P9bsAo3rLvmibsi9uANcAu2XrRDrJPye3/ZHAXdnjZ/7pauz7BuD47HHRP96nc//AFwEfza2bn/2DL8qeR+WfOHt+OXBOjeN+CPhh7vkYcD9wFOnkck/V9ucCl+Reu7pq/feAE4GXAd/Ijn0M8BrgxmybRvu9DViSW7d39v5m5f7598ut/3HlxFHw/p753Kr+jqdXLasXIC4A/rpq+9uBVxUcT8A9wCuz5+8EvpU9PpB00j4amN3Cd++M3PPjgF/kvltbgB1z62uWmRTEfkUWxLJ136c4QBxJ+oExq6BMp5L7ntL4f+Bi4MO5df9P9edetf/VwF9mjw8i/d/NrbHt3wN/lz2ufEfKBIj3A5+p2tfXSQF1HvAY8F/JfkiMws1VTN11QkTsTPonei7p1zKkXxxzgeuzS9XHgKuy5duR9Lbc5f9jpF9VexRtW2Af0i9AACJiE+lXz765bR7IPd5MCiK13Jvb1zZgfXaMSWCfShmzcv4FsFedfX2H9Nm8Mnt8Lemk9KrsOSX2Owl8JbfuNmBr1XGbeX9133MJk8CfVJV3f9JnNE2kM8zngFOyRX8IrMnW3Qm8h3SCekjS55SrGmyyzHdXHX9DRPxnyTLvA9yXlTW/vyL7A3dHxNMlytfof2CfgvdQz2VM/xz/JSI2A0g6QtK3s2qvXwNnUP7/J28SeFPV5/QK0tX8k6Sr+zOA+yX9m6TntnCMgeIA0QMR8R3Sr8xKffPDpKqEF0TEbtlt10gN2tNImiRVkfwxMBERu5GqDyrtBI3S8f6K9MWu7G8eqermvhbfzv65fY2RqnZ+Rfpnviv3fnaLiJ0j4rg65awOEN9h+wDRaL/3AsdWrd8xIlp5f7U+y+rlm0knt4rfyT2+F1hRVZ65EfHZGvv+LHBS9nc+AvjSMweNuCwiXkH6+wWpWqis/XOPF5L+RrXeT70y3w/sK01rl1pY45j3AgtV3PBdfcxG/wP3F7yHeq4G9pR0KClQXJZbdxmpKmv/iNiV1B5Sq53tSer/bT9T9TnNi4gPA0TE1yPitaSr2J+T/m+HmgNE7/w98FpJh2S/vD8F/J2kZwFI2lfS6wpeN4/0z7Uh2+400hVExYPAfpLm1DjuZ4HTJB0qaQfgb4AfRcS6Ft/H4ZJOzE4C7yHV6f+QVHXzRNb4uZOkcUkvlPS7uXIuyoJKxfdJde0vBX4cEbeQToZHAN/Ntmm0308AK7ITLJL2lHR8i+/tQVIbRyM3AH+YleUYUkCr+BRwRvarVZLmZY2kOxftKCJ+SjpZXgh8PSIqDbgHS/q97G/2n6ST6bYm3su7JO2n1GB/HvD5OtvWK/MPSG0W75Y0W9KJpL9XkR+TTuwfzvaxo6SXZ+umfU9L/A9cDpwq6fmS5gIfrPdmI+Ip4AvA/yK1aVydW70z8EhE/Kekl5KuMGq5ATg5e6+LSW0uFauBN0h6Xfa331GpkX4/SXtJOj77AfZbUvVyM3+vgeQA0SMRsQG4lNTYCqk+807gh5IeB75JOllWv+5W4G9J/6gPkhrbrstt8i3gFuABSQ8XvP6bwAdIv0zvB54DnNzGW/kq6VL6UeCPgBMj4qmI2Aq8ntRT6y6mTnq7Zq/7Qna/UdJPsrI9CfwEuCUitmTrf0Cqpngo26bRfs8n/Tr8hqQnSMHqiBbf20XA87Pqg3+ps93ZwBtIdc5LgWe2jYi1pLaEfyJ9RneS6t/ruYzU1pD/1bsD8GHS+30AeBZZN2lJSyXdUmKf3yA16P8CqDm+o16Zs7/LidnzR0h/+y/X2M9W0udyIKltZX22PRR/T2v+D0TE/0/6UfWtbJtvNXi/lfd8NPCFqmqus4D/L/t+/CUp+NTyAdL/yKPAX5H7m0TEvcDxpCrODaQrij8jnUfHgPeRrtQeIf1oOLNEmQdapfeEmY0ISeuAd2Q/Dsxa5isIMzMr5ABhZmaFXMVkZmaFfAVhZmaFhjpR1x577BGLFi3qdzHMzIbK9ddf/3BEFA7MzRvqALFo0SLWrl3beEMzM3uGpEYj0wFXMZmZWQ0OEGZmVsgBwszMCjlAmJlZIQcIMzMr5ABhZjZA1qyBRYtgbCzdr1nTv7IMdTdXM7NRsmYNLF8Omzen53ffnZ4DLO3DDNe+gjAzGxDnnTcVHCo2b07L+8EBwsxsQNxzT+PlvayCcoAwMxsQC2tMrFpZXqmCuvtuiJiqgupWkHCAMDMbECtWwNy505fNnZuWQ++roBwgzMwGxNKlsHIlTE6ClO5XrpxqoC5TBdVJ7sVkZjZAli6t3WNp4cJUrVS0vBt8BWFmNiQaVUF1mgOEmdmQaFQF1WmuYjIzGyL1qqA6zVcQZmZWyAHCzMwKOUCYmVmhrgUISRdLekjSzbllb5J0i6RtkhZXbX+upDsl3S7pdd0ql5mZldPNK4hPA8dULbsZOBH4bn6hpOcDJwMvyF7zcUnjXSybmY2oQUqXPey6FiAi4rvAI1XLbouI2ws2Px74XET8NiLuAu4EXtqtspnZaOpEriIHmCmD0gaxL3Bv7vn6bJmZWWnt5irqdTK8QTcoAaI0ScslrZW0dsOGDf0ujpkNkHZzFQ3afAz9NigB4j5g/9zz/bJl24mIlRGxOCIW77nnnj0pnJkNh0bpshvpdTK8QTcoAeIK4GRJO0g6ADgI+HGfy2RmQ6bdXEXtBphR081urp8FfgAcLGm9pLdLeqOk9cCRwL9J+jpARNwCXA7cClwFvCsitnarbGY2mpYuhWXLYDzrAzk+np6XTU3R62R4g04R0e8ytGzx4sWxdu3afhfDzAZEpZE5344wd25zCe3WrEltDvfck64cVqzoXe6jXpF0fUQsbrTdoFQxmZmVUq8baicamZcuhXXrYNu2dD9qwaEZzuZqZkOj+gqh0g0V0oncjcyd5SsIMxsaja4QFiwofl2t5VafA4SZDQ1fIfSWA4SZDY1a3U3HxtJt48bi9Y88Urzc6nOAMLOhUdQNFWDr1pQao5aZOo6hXQ4QZjY0qudklhq/pnocg5PxlecAYWY91e4JOt8Ntd5Vg5QCSX4MhLO9NscD5cysZzoxkC2v3hVE0alt0aIUFKpNTqag00iny98vHihnZgOnVjfVZcta+0U+MdHccmd7bY4DhJn1TK0TcaWRudkqn/PPhzlzpi+bMyctL+Jsr81xgDCznilzIm7mimLpUrj44qlG68nJ9LxWdY+zvTbHAcLMeqZWN9VqzVxRNJM7qboXVHUjdivlH+Vsrw4QZtYzrXRTrb6iOOus5noRVfc6gtaT8bUbYIaNezGZWd/ssUft0c9lzZ4Nu+ySRksvXAjHHQdXXpnaBRYsgEcfTcEgv/0ll4zuSb0M92Iys4HXiRQYTz2VgkylSuqCC6bGOWzcOD04VLZ/29tmxjiGdjlAmFnf9KtxtzLIrpWBcjOJA4SZ9c1xxxUvnzcv1fGP9eAMNcrjGNrlAGFmXVUvNcWVVxa/Zo890q/83XfvRQlHdxxDuzyjnJl1TbszwPUqTbcnFCrmKwgz65pGqSkaDTwr00YxZ05KrVGmy6w1p2sBQtLFkh6SdHNu2QJJV0u6I7vfPVsuSf8g6U5JN0o6rFvlMrPeKUqMl1/eaOBZ0frZs6cCQmXk9MMPpyqpycnWyukJhYp18wri08AxVcvOAa6JiIOAa7LnAMcCB2W35cAFXSyXmfXI+HjtdWNj6Upit92mL1+0aGqMQtHAtHe8A+bPL95n2ZHa1UY1VUa7uhYgIuK7QHVcPh5YlT1eBZyQW35pJD8EdpO0d7fKZmatO+ssmDUrnbBnzUrPa9m6tfa6SjfTX/1q+vJbb4Uddphq1L7uuql1mzbBRRfVns+hOqBMTGyfzK/aKKfKaFev2yD2ioj7s8cPAHtlj/cF7s1ttz5bth1JyyWtlbR2w4YN3SupmW3nrLPSQLTKiX/r1vR8552Leym1WuWzZUvtgW9btkzftrqbaj4308MPb5/M78wzZ06qjHb1rRdTRISkpvN8RMRKYCWkVBsdL5iZ1bRyZfHyTZvSfXUvpRUrtp9gpxvqdVNdutQBoFW9voJ4sFJ1lN0/lC2/D9g/t91+2TIz67F64xbqVRlV5H/RV1f51GuTaIfbELqj1wHiCmBZ9ngZ8NXc8rdlvZleBvw6VxVlZj3SiTmbYfov+nyVz6pVrTUi1+M2hO7pZjfXzwI/AA6WtF7S24EPA6+VdAdwdPYc4Ergl8CdwKeAOs1eZtYtnZpSs9Yv+uorilZVUoW7DaG7utmL6ZSI2DsiZkfEfhFxUURsjIglEXFQRBwdEY9k20ZEvCsinhMRL4oI5/A265F8lVK9cQvNnNDzOZaqq6zyvZJatWDB1HwO0Nz8EFaeU22YzWDVqTA6pZJjqSjVxgUdGOW0cWMKCAsWwOOPpxTelf3nG8mtPZ4wyGwGW7So9lVDI1Jqp6i3fmysXMN2p01OTl1d2PY8YZDZiKrXy6hZ7WQxbZTaIqI/wQGcnbVTHCDMhkinehlVtNs9tNXUFvWMjU3NAzE21lpjtru9doYDhNkQ6VQvo4p2T/Cd6JVUNClQZZrQysxvzXC3185xgDAbIo3mT2hWUe6iMoPZ8lVL+XEOExPljy2lpHvVc0ZXP28kn+7b3V47ywHCbIg0mj+hFdW5i1atmh4wZs+evn2nfqFv2wa/+U1rrx0fL073vW6dg0MnOUCYDZFG8yd0QnXAeMc7pq4qxsdh2bLaJ+Gy8ypU0nW30og9Z04KYg4I3ecAYTZEiuZH6HSVSr6X1B57pPTa+eytF12Ulhf1opo3r9wxNm1qvc2iUZtEJ3t5zXgRUfcGvByYlz1+K/C/gclGr+vF7fDDDw8za87q1RGTkxFSul+9evq6sbGIdBoud5s7d2ofUnOvbfU2OVn7vc2dW7t8lgBro8Q5tuFAOUk3AocALybNEnch8OaIeFXXolZJHihn1pw1a+C006ZGHkNqY7jkknQVMn8+PPlk8/utDEzr5bzQUmp7WbFi6gqq1sA/D5ybrpMD5Z7OIs7xwD9FxD8DO7dbQDPrvbPPnh4cID0/++z0uJXgAFO9qNpJ5x0Bq1eX73ZbNA6k0728ZroyAeIJSecCfwT8m6QxYHaD15hZl7RTx75xY+3lReMRylqwIN1X8iC1ojJH9bJl9XtRVcuPA+lGL6+ZrMxX4i3Ab4HTI+IB0mQ+/6urpTKzQp0eSZ3XTlq2SoC58kpYsqS1K4nK+1m1KlUbVXpRXXJJ44F4lSuEXvTymklKJeuTNAkcFBHflDQXGI+IJ7peugbcBmEzTbt17L1oI5g7d6pnVavHq/V+yrz/NWvSFcU992zfRmFJ2TaIMo3U7wSWAwsi4jmSDgI+ERFLOlPU1jlA2EwzNlb8S18qNwJ5fLz5kcqtqByn3uklon4AKXptUXryfECycjrZSP0uUlfXxwEi4g7gWe0Vz8xa0W4de7PBodUrgK1b6weHSkqOWlVRtZb3YhyITSkTIH4bEVsqTyTNAoZ3EgmzIbZixfaNtuPjaeBZpdH6rLNqN2I3kyspIgWUM89sr3dStTlz4Pzz0+NaI6nrjbDOj/T2SOruKhMgviPpL4CdJL0W+ALwte4Wy8xqqf5Vv3VraiSuNPJecEFnGrErAQZgv/3ab7/I506qnNRrzSdRb54J650ybRBjwNuB3wcEfB24MMq0bneZ2yBspml1Brh+DGQrOn41tyn0R8faICJiW0R8KiLeFBEnZY/bCg6SzpZ0s6RbJL0nW7ZA0tWS7sjud2/nGGajIj/uodXpQe++u71xDu2YPXt6FVj+asZtCoOtzBXEXRS0OUTEs1s6oPRC4HPAS4EtwFXAGaSeUo9ExIclnQPsHhHvr7cvX0HYqCv6hT0spDSA7oknYMuWqeW+Qui/TvZiWgz8bnY7CvgHYHUbZXse8KOI2BwRTwPfAU4kpfJYlW2zCjihjWOYjYSiGeSGQaXb7fz504MDtDcDnvVWmSqmjbnbfRHx98B/aeOYNwNHSZrIBt0dB+wP7BUR92fbPADsVfRiScslrZW0dsOGDW0Uw2zwDWsOoUrFRK0qsVaryqy3ZjXaQNJhuadjpCuKhq+rJSJuk/QR4BvAk8ANwNaqbUJSYd1XRKwEVkKqYmq1HGbDYOHC4T6Zjo8Xd1ntZLdZ654yJ/q/zT1+GlgHvLmdg0bERcBFAJL+BlgPPChp74i4X9LewEPtHMNsFKxYMZxtEJWJg1oZ52CDo2GAiIjXdPqgkp4VEQ9JWkhqf3gZcACwDPhwdv/VTh/XbNhUGnIruYX637m8nB13TPeTk7VzJ9ngq9mLSdL76r0wIv53yweVvgdMAE8B74uIayRNAJcDC4G7SZMS1Z3h1r2YbKapNQ6iTO6jXqo0Uhf1wpozB3beOc1f7WR6/dGJXkw7N7i1LCKOiojnR8QhEXFNtmxjRCyJiIMi4uhGwcFspsiPg3j44eJtli/vTRK+siq5oarHOUxMpCCWH/l9+um157i2PiszL+mg3jwntQ2DenNAl3lt9RzLRbfx8d7NB13mduaZxe9ncrLxaz2HdPfRwTmpdySl2ngBsGMusJze1chVgquYbNC1kkoiP5/B2NhwNujWSq1RK1152ddbZ3RyoNxngN8BXkca1LYf0PfJgsyGQdFAt3oDxapnjBvG4AC1x2+UTUs+rOM/Rk2ZAHFgRHwAeDIiVpEGyR3R3WKZjYZaJ7pKbqTqOvdhHTldrVYgKJoStJnXW2+VCRBPZfePZXmUdsUTBtkMkm8kbrYRtd6JrqiRdpgHxVXMmVM+Od/ExPbzW3gO6QHSqJECeAewO/Aq4JekAWz/rUwDR7dvbqS2bitqJG6mEbVsI/Oo3MbGtl82e3b9z6udRnxrDR1spB6PiIGsCXUjtXVbrXEHzTSi5hudyzTQjqKJidpddK33OtlIfZeklZKWSP2absSsP2q1ITTTiJqfInMYRxCPjaUTfKVKqJV5JTZu7Hy5rPvK/KmfC3wTeBewTtI/SXpFd4tlNhhqtSHUa1uo12ZRtpF2EFQm8Ln00vTrf9u2dH/ppVNtCDbayqT73hwRl0fEicChwC6k7q5mI6/ohF6vEbW6m2r1nNDVjbSDbNu2dOVTPV6jlSuiiYlOl856odTFoqRXSfo4cD1psFxb2VzNhkWZKTHzVwzLlhWPe1i2bOqK4rrrevkOWlem11aZK6I5c+D88ztaNOuRhgFC0jrgPcD3gBdFxJsj4kvdLphZv1RXEcHUL+ZKw3Rl/R57pG6qjQa2bd06dUVxwQVT2w+yoisgmP75nHdeCn75AHrmmdOfX3yxk/ENqzK9mHaJiMd7VJ6muBeTdVqj1BjDPEd0Oyq9tlpJHWKDp2wvpoYBYpA5QFinNerWWmv9TCDVzg3l3EnDpZPdXM1mjEbdWmdyjqB6VWgz+XMZZQ4QZjmNurU6R1Axfy6jqeaUo92cUc5sUBXNAZ3v1jqsc0RD/SqiWtuXqYF27qTRVWZGucXAmcC+2e0M4LDuF82s9xp1a126NPXaGR9Pz1sZVdwv27bBqlXlB+rVG+cwPl6726+Njppf74j4q4j4K9L8D4dFxJ9ExJ8Ah5PmjTabcdasgQsvnPoVPkjTfJbR7EC9WgMFV62qPZDORkeZ3z97AVtyz7dky8wGUjvpuYtGQleuGCR461vhqaca72dUlBkoaKOrZhtEzqXAjyV9JXt+ArCqnYNKei8pjXgANwGnAXsDnwMmSCO2/ygittTciVmB6n76lYFe0PoUn8M6q1uRVsZxLF3qgDBTlRoHIekw4Kjs6Xcj4qctH1DaF/h34PkR8RtJlwNXAscBX46Iz0n6BPCziLig3r48DsKqNZueu5sD38o28vZKRPlxHB7XMNo6PQ5iLvB4RJwPrJd0QFulS1cuO0male37fuD3gC9m61eRrlTMmtJseu5uTvE5aO0TZWesc68kqyiTi+mDwPuBc7NFs4HVrR4wIu4DPgbcQwoMvyZVKT0WEU9nm60n9ZgqKs9ySWslrd2wYUOrxbAR1Wx67pk0wKve1Yx7JVmRMlcQbwT+AHgSICJ+Rer+2hJJuwPHAwcA+wDzgGPKvj4iVkbE4ohYvOeee7ZaDBtRRb1u6s2RPEoDvFqd0Me9kqyWMl+lLdkcpgEgaV6bxzwauCsiNkTEU8CXgZcDu2VVTpC61t7X5nFsBqrudTMxkX45b9xYnJ10mCbwaaTZCX18xWCNlAkQl0v6JOkE/k7S7HIXtnHMe4CXSZqbTWG6BLgV+DZwUrbNMuCrbRzDZrD8hDbz52/fLXXz5tT2UNk2P/CtU+bM6ez+mlFmQp/JSV8xWGNlZpT7GKnx+EvAwcBfRsQ/tHrAiPhRtr+fkLq4jgErSe0c75N0J6mr60WtHsOsolGj9Zo1qXql011Zt2zp34xx+XEgmzbB7NnT17sR2soqMx/ERyLi/Y2W9YO7uVqRRuMaYDDTdzebK6laRHG33TlzYOed4ZFHUpvLihW+apjpOtnN9bUFy45tvkhmSTsjncvsOz8SuuhkO3v21C/oQerF1GyupLzKnM9F3Xa3bElVba5SsmbVDBCSzpR0E/BcSTfmbneRqobMmlaUyqJ6Sst2lBnXEAFnnz2YifaKUlssWTLVRlK5ysjLz/nc7DgQs3rq/YtcBryB1Fj8htzt8IjwbxBrSdEJPN9o3K4yJ8Knn57q1TRII51rOe20VOaIdBWQ76VUPedzs+NAzOop0wbxMuCWiHgie74L8Lyssbmv3AYxfMbGik/KUmdGHg9Sm0KzJFiwAJ54IlULVTQz57PnjLYyOtkGcQGwKfd8U7bMrGnd+IVb3Wunn11M21EZr7GlKkXl5s2pK26ZNhtnX7VOKhMgFLnLjIjYRrkssGbbqTW/QKvdLqvbNCpVRxMTUwPlhjVg5G3dWr7NJj8Owo3S1o4yAeKXkt4taXZ2Oxv4ZbcLZqOp079wi9o0nnpqqtfOww+nOvrK8ebPb/899Fsn22zM6ikTIM4A/l9S6ov1wBHA8m4WykZbJ3/h1mqUvvvuqSqZ666bWj6Mc0kXca8k64UyI6kfioiTI+JZEbFXRPxhRDzUi8KZQf1xE/PqZAarVMlccMFUFdSgpOAeGyuXK6lWChD3SrJeqNmWIOnPI+Kjkv6RLFFfXkS8u6slM6PxDHFPPtm/srVj27apAFBvpHetXklOlWG9UO8K4rbsfi1pvobqm1nXNRo3MQzjGGqpNdI7HwDcK8n6qdSUo4PK4yBGX61xEzB4U3q2Y3w8XVU4V5L1QtlxEPWqmL5GQdVSRUT8QYtlsyGTT37X6xPYggWp62qRUQkOkILDoLSPmFXUG8/wsez+ROB3mJpm9BTgwW4WygZHozYA64wFC/pdArPt1WyDiIjvRMR3gJdHxFsi4mvZ7Q+Bo3pXROunTuROaid76yOPlN/WzDqrzDiIeZKeXXki6QDSPNI2A7SbHbTd7K3D3J2zKPNqLQ6ENojKfH3fC1wr6VpJ3yFNDfqe7hbLBkW7uZPKXIHUu8JYsWL7GdGGRSXzaplUH8McCG10NcypFBFXSToIeG626OcR8dvuFssGxYoV7fXDLzM6W+ybAAAS0ElEQVTlZ6M2jmFsjK4Mgqu8h0oj/4IF8Pjj0+fJ9rgGG1gRUfcGzAX+O/Cp7PlBwOsbva4Xt8MPPzys+1avjpicjJDS/erV5ddPTlZmXZh+m5wst35ionj9INwiGq9v5fM06zZgbZQ4x5aZD+LzpIFxb4uIF0qaC3w/Ig7tZuAqw+Mg+q/R/ANnnZVSXVQ780z4+Mfrp5uIaJyOop/qzSFdGQltNog6OR/EcyLio8BTABGxGWj531bSwZJuyN0el/QeSQskXS3pjux+91aPYb3TqI3hyiuLX1dZXivXUK3lg6TMSGizYVYmQGyRtBPZoDlJzwFaboOIiNsj4tDsCuRwYDPwFeAc4JqIOAi4JntuA65RG0Oj9UUn2PzyiYnWy9ZL4+NOhWGjp0yA+CBwFbC/pDWkk/efd+j4S4BfRMTdwPHAqmz5KuCEDh3DuqhRL6dG6ycni9dXlh/a94rMciojoT1Bj42SugFCkoCfk0ZTnwp8FlgcEdd26PgnZ/sE2Csi7s8ePwDsVaNMyyWtlbR2w4YNHSqGNaPRFJ/5KpYDDyzex4MPTr2+uhtr/vXf/nZHi15Tu3mdPBLaRlGZRuqbIuJFHT+wNAf4FfCCiHhQ0mMRsVtu/aMRUbcdwo3UvVfUKD17NuyySxrstXBhCgrXXlu7+qhaJSFfRKqqefWr4c47UzVUr7q4VhrNFy1KXW2bNTGRZq8zGwadbKT+iaTf7UCZqh0L/CQiKnmdHpS0N0B270mJBlCjKT6POw6uuaZ8cID0ukog2Lo1vb4y8rrbxsenggOk8rfCI6FtFJUJEEcAP5T0C0k3SrpJ0o0dOPYpTFUvAVwBLMseLwO+2oFjWIc1anReubJ3ZWnHxEQKQE8/DS9/+VSVWavl90hoG0UNR1IDr+v0QSXNA14L/Lfc4g8Dl0t6O3A38OZOH9fat3BhcRVM5QTZzJVDP23cmALCggXwxBOwZUta3kr53a3VRlW9+SB2BM4ADgRuAi6KiKc7cdCIeBKYqFq2kdSryQZYo9QbwzSJT0TtuSbKkDzBj422elVMq4DFpOBwLPC3PSmRDbRGU2DOq5Hnd/784QkcZZx5pru12uirFyCeHxFvjYhPAifhOSBGVrPzNSxdmk6MRSfIJ58sfk1lea1xD4MmP/BtyZKpkd3Vjdpmo6xeG8Qz+SYj4mkNclIca1mnZ4xr1EZRVEU1aPK5pMxmsnpXEIdkeZIel/QE8OLKY0mP96qA1l2dmDEub8WKdILNmzUr9XKSYNkyOPLIqSqqQTB7durV5FQZZtPVm3J0PCJ2yW47R8Ss3ONdellI6552Z4yrVt1GMX9+6kpaPc7huONSFVU/VQLCJZekQW5uUzCbruSEiDaq2p0xrki+jeI3vynept/jJYYhW6xZvzlAzHBFVUKd7NffKFtrN0WkBuVax29ljmyzmcQBYoZr1G21FfleUfV0sw2icuxa81HktdPmYjbKHCCsbrfVIvW6xa5ZA6ef3ptcSvUCTKV9o2xbSqttLmajzAHCGsoHhD32mB4Aqqtozj57Km1Ft23b1ng+ibJtKc6lZLY9Bwirq/qKYOPG7QNAvoqm3dQVZVWqkBq1oRStr+ZcSmbFHCCsrrJXBO1W0YyPpyuCslOMbtuWgsR556WxFbXaUIraWM48s7NtLmajquGEQYPMEwZ1X9lf9ZUTvNTa+IZK+oo1a+C009IcE2V55LNZczo5YZBZQ5Vuo0XBYWxs+kjlermNli6FV76yuWNX90JqNreUmRVzgLC6J9RaVT5jY+mEX2vAWT7Z3aWXTh+p/M1vTo2urp6wZ/78NNK6WZUqrkpuqVqN6GZWngPECGrmF3SjE+r55xe/7tJL0wm/VnVSZV2jbrPVx6+VDbaRSi+kTueWMpvJHCBGTLO/oBudUK+7rvh1leXtpuooOn6z8r2QOp1bymwmc4AYMWV+QeevMIpSc0PjOaYry9tN1dHKiXvOnNrZV7uRW8pspnKAGDGNfkFXX2HU0miO6crydlN1lD1xz5s3tf+LL66dfbXbuaXMZhIHiBHT6Bd0mSqdZk+ozabqyCs6oY+PTw2Eq/Ry2rSp3P67kVvKbKbqS4CQtJukL0r6uaTbJB0paYGkqyXdkd3v3o+yDbtGv6DrVek0e0Kt1QjeTCN50Ql91aqpbrNPP9389J7tBCwzy4mInt+AVcA7ssdzgN2AjwLnZMvOAT7SaD+HH354zESrV0dMTkZI6X716vLrJycj0ql3+m1ysvhYtbbP3+bOnTrG6tXpea31ZtZ/wNooc64us1Enb8CuwF1ko7hzy28H9s4e7w3c3mhfMzFAtHsCbvb1RdvXCzDNBiAz672yAaIfVUwHABuASyT9VNKFkuYBe0XE/dk2DwB7Fb1Y0nJJayWt3bBhQ4+KPDhq9VJatqz1Kp16VUrV29dSqbpyN1Oz0dGPADELOAy4ICJeAjxJqlJ6RhbhCvvYRMTKiFgcEYv33HPPrhd20NQ60eZnSDvttJSWu1bAqK6jh/ptBvnta6XXrjSCd6ObqVNnmPVHPwLEemB9RPwoe/5FUsB4UNLeANn9Q30o28Arc6J96qmUdrsSMKoHylXP73DaaeUH1hU1gs+Zk3oZjY3VTvd93HGl3t52nDrDrH96HiAi4gHgXkkHZ4uWALcCVwDLsmXLgK/2umzDoMz8BtXyA+WqT7gbN26fObVeaoqlS1N1ViUH09jY9IC0aVPx68pM/VnEqTPM+qcv6b4lHQpcSOrB9EvgNFKwuhxYCNwNvDkiHqm3n5ma7nvNmnSCvOeedIKuNZgtr5KGe9Gi2qOni7YvOvby5c2nx2g1DfjYWPGAvlb3Z2YDnu47Im7I2hFeHBEnRMSjEbExIpZExEERcXSj4DCT5dsEVq0qd0VRqZoq21hcqyqr1dxJrbZBOHWGWf94JPWQq+5lNDEBs2ZN32bOnKmBcmVOrPVGUrfSG6mdVBdOnWHWPw4QIyB/RXH++dt3R81X0dRqZK6V/K5a2V/uUmdSXTh1hln/eMrREVOrjWFycqpLa74NY+HCFDTKnnDLtEHMng2XXOKTuNmgKtsG4QAxYnrRqFsdYI47LvVSaiXgmFnvlQ0QsxptYMNl4cLiK4hONuouXeoAYDYTuA1iBOQHvm3alKp48tyoa2atcIAYckUD3yq9mdyoa2btcBXTkCsal7BlC8yfn2ZdMzNrla8ghlytcQl33+3kdmbWHgeIIVev8dnJ7cysHQ4QQ65M8j4ntzOzVjhADLlmJ/QxMyvLAWIENDOhj5lZWQ4QA67Z2dSc3M7MOsUBYoC1Mpuak9uZWac4F9MAK5N4z8ysWQM9YZCVU6th2Q3OZtYLDhADzLOpmVk/OUAMMDc4m1k/OUAMMDc4m1k/9SVZn6R1wBPAVuDpiFgsaQHweWARsA54c0Q82o/yDRLPvWBm/dLPK4jXRMShuZb0c4BrIuIg4JrsuZmZ9ckgVTEdD6zKHq8CTuhjWczMZrx+BYgAviHpeknLs2V7RcT92eMHgL36UzQzM4P+TRj0ioi4T9KzgKsl/Ty/MiJCUuEIviygLAdY6P6eZmZd05criIi4L7t/CPgK8FLgQUl7A2T3D9V47cqIWBwRi/fcc89eFdnMbMbpeYCQNE/SzpXHwO8DNwNXAMuyzZYBX+112czMbEo/qpj2Ar6iNHnBLOCyiLhK0n8Al0t6O3A38OY+lM3MzDI9DxAR8UvgkILlG4ElvS6PmZkVG6Rurj3R7PwKw2AU35OZ9V+/ejH1RWV+hc2b0/PK/AowvKOVR/E9mdlgmFHzQYzi/Aqj+J7MrLs8H0SBUZxfYRTfk5kNhhkVIEZxfoVRfE9mNhhmVIAYxfkVRvE9mdlgmFEBYhTnVxjF92Rmg2FGNVKbmZkbqc3MrE0OEGZmVsgBwszMCjlAmJlZIQcIMzMrNNS9mCRtIKUGb8UewMMdLE6nDXr5YPDL6PK1x+VrzyCXbzIiGs64NtQBoh2S1pbp5tUvg14+GPwyunztcfnaM+jlK8NVTGZmVsgBwszMCs3kALGy3wVoYNDLB4NfRpevPS5fewa9fA3N2DYIMzOrbyZfQZiZWR0OEGZmVmhGBQhJ45J+Kulfs+cHSPqRpDslfV7SnD6WbZ2kmyTdIGlttmyBpKsl3ZHd797H8u0m6YuSfi7pNklHDkr5JB2cfW6V2+OS3jMo5cvK+F5Jt0i6WdJnJe04YN+/s7Oy3SLpPdmyvn1+ki6W9JCkm3PLCsuj5B+yz/FGSYf1qXxvyj6/bZIWV21/bla+2yW9rtvl65QZFSCAs4Hbcs8/AvxdRBwIPAq8vS+lmvKaiDg013f6HOCaiDgIuCZ73i/nA1dFxHOBQ0if40CULyJuzz63Q4HDgc3AVwalfJL2Bd4NLI6IFwLjwMkMyPdP0guBdwIvJf1tXy/pQPr7+X0aOKZqWa3yHAsclN2WAxf0qXw3AycC380vlPR80t/7BdlrPi5pvAdlbF9EzIgbsB/pS/V7wL8CIo1ynJWtPxL4eh/Ltw7Yo2rZ7cDe2eO9gdv7VLZdgbvIOjUMWvmqyvT7wHWDVD5gX+BeYAEwK/v+vW5Qvn/Am4CLcs8/APx5vz8/YBFwc6PvG/BJ4JSi7XpZvtzya0k/BirPzwXOzT3/OnBkP/7Wzd5m0hXE35O+9Nuy5xPAYxHxdPZ8PekfuV8C+Iak6yUtz5btFRH3Z48fAPbqT9E4ANgAXJJV0V0oad4AlS/vZOCz2eOBKF9E3Ad8DLgHuB/4NXA9g/P9uxk4StKEpLnAccD+DMjnl1OrPJUAXNHv/+Vqg16+mmZEgJD0euChiLi+32Wp4xURcRjpcvldkl6ZXxnpp0e/+iTPAg4DLoiIlwBPUlXd0OfyAZDV4f8B8IXqdf0sX1ZXfjwp0O4DzGP76om+iYjbSNVd3wCuAm4AtlZt0/e/b96glWdUzYgAAbwc+ANJ64DPkaqZzgd2kzQr22Y/4L7+FO+ZX5lExEOk+vOXAg9K2hsgu3+oT8VbD6yPiB9lz79IChiDUr6KY4GfRMSD2fNBKd/RwF0RsSEingK+TPpODtL376KIODwiXklqD/k/DM7nV1GrPPeRrngq+vpZFhj08tU0IwJERJwbEftFxCJSFcS3ImIp8G3gpGyzZcBX+1E+SfMk7Vx5TKpHvxm4IitXX8sXEQ8A90o6OFu0BLiVASlfzilMVS/B4JTvHuBlkuZKElOf30B8/wAkPSu7X0hqaL2Mwfn8KmqV5wrgbVlvppcBv85VRQ2CK4CTJe0g6QBSY/qP+1ymcvrdCNLrG/Bq4F+zx88m/aHuJFVL7NCnMj0b+Fl2uwU4L1s+QWpYvwP4JrCgj5/bocBa4EbgX4DdB6x884CNwK65ZYNUvr8Cfk4K/J8BdhiU719Wvu+RgtbPgCX9/vxIgf5+4CnSFezba5WH1OHkn4FfADeRayDucfnemD3+LfAguU4HwHlZ+W4Hju3X37nZm1NtmJlZoRlRxWRmZs1zgDAzs0IOEGZmVsgBwszMCjlAmJlZIQcIG1mSTpAUkp5bYttTJe3TxrFerSxLcDs6tR+zTnCAsFF2CvDv2X0jp5LSYJhZxgHCRpKk+cArSAOYTq5a936luTd+JunDkk4CFgNrsvkkdlKan2OPbPvFkq7NHr9U0g+ypIXfz40ur1WOH0p6Qe75tdn+Gu5H0ock/Wnu+c2SFmWP3yrpx1l5P6k018m4pE9n290k6b2tfXpmyazGm5gNpeNJ81f8H0kbJR0eEddLOjZbd0REbJa0ICIekfTHwJ9GRGWyplr7/TlwVEQ8Lelo4G+A/1qnHJ8H3gx8MMsftHdErJW0S5P7eYak5wFvAV4eEU9J+jiwlDQKf99Ic04gabcy+zOrxQHCRtUppISMkBI0nkJKsX00cElEbAaIiEea3O+uwCpJB5Gyic5usP3lpCypHyQFii+2uJ+8JaSJkf4jC2Q7kRLXfQ14tqR/BP4tO65ZyxwgbORIWkDK2PsiSUGawS0k/VkTu3maqSrYHXPL/xr4dkS8MavuubbeTiLivuwK5sWkX/1nNLGffBny5RCwKiLOrX6BpENIkxGdQQpIp9crn1k9boOwUXQS8JmImIyIRRGxP2lGvKOAq4HTsolxKsEE4Alg59w+1pF+pcP0qp9dmUrVfGrJ8nyeNFnVrhFxYxP7WUdKq47SPMsHZMuvAU7KZWBdIGkyazMZi4gvAf+98lqzVjlA2Cg6hTSnRt6XSNNSXkVKv7xW0g1ApRH408AnKo3UpOyr50tay/TJcz4K/E9JP6X8FfgXSQ3llze5ny8BCyTdAvwxaY4GIuJWUgD4hqQbSUFvb9IsZddm72s1aapLs5Y5m6uZmRXyFYSZmRVygDAzs0IOEGZmVsgBwszMCjlAmJlZIQcIMzMr5ABhZmaF/i9JsN3oDrJeKgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "#Predict the response on the test data set\n",
        "\n",
        "print(\"The coefficients for each indicator feature in data set are\")\n",
        "\n",
        "Performance= pd.DataFrame({'Actual':Y_test, 'Predicted': Prediction_test})  \n",
        "\n",
        "print(Performance)\n",
        "\n",
        "plt.scatter(Y_test, Prediction_test,  color='blue')\n",
        "\n",
        "plt.title('Relation between true vs. predicted values')\n",
        "plt.xlabel('Actual values')\n",
        "plt.ylabel('Predicted values')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C7K54im0zmJQ"
      },
      "source": [
        "The output in the table details the actual and predicted values by the model. It is observed from the entries that the model learnt is very close to the actual values. The same is demonstrated by the figure. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7h4T0z2czmJQ"
      },
      "source": [
        "## 7. Evaluating the Performance of the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0ulNLx-lzmJQ",
        "outputId": "48b13c9d-c857-46ba-c6f6-3893c92ef87c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Mean absolute Error: 3.09\n",
            "\n",
            "\n",
            "Mean squared Error: 14.95\n",
            "\n",
            "\n",
            "R2 score: 88.32 %\n"
          ]
        }
      ],
      "source": [
        "# summarize the performance of the model using MAE, MSE and Rsquare\n",
        "\n",
        "print(\"Mean absolute Error:\", round(metrics.mean_absolute_error(Y_test, Prediction_test),2))\n",
        "print(\"\\n\")\n",
        "print(\"Mean squared Error:\", round(metrics.mean_squared_error(Y_test, Prediction_test),2))\n",
        "print(\"\\n\")\n",
        "print(\"R2 score:\", round(metrics.r2_score(Y_test, Prediction_test),4)*100,\"%\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3WbIVn67zmJR"
      },
      "source": [
        "The low MAE, MSE and high R2 score is 88.32% indicates a good model "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wbo2o0zJzmJR"
      },
      "source": [
        "## 8.  Overcoming  Model overfitting, Collinearity and Model complexity using Regularization\n",
        "\n",
        "In general a good model is the one that just not fit good to the sample of data set provided but, also to the any new sample of data set. Overfitting is a situation where models fits excellent on the training data set but, fails to produce the same result on the testing data set. Overfitting is a common issue in supervised learning and shall be avoided. Other most common issue in multiple linear regression is model complexity where, the weights learnt by the model are heavy. The aim is to produce less complex model for easy understanding and analysing than producing complex models difficult to interpret. The model complexity increases with presence of collinearity in features present in the data set. Collinearity indicates high correlation amnong indicator features. To produce model that does not overfit and is less complex in nature, the collinearity has to handled. \n",
        "\n",
        "The practical demonstration below details the effect of collinearity and overfitting issues and methods to overcome them. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qmHE9GuUzmJR"
      },
      "source": [
        "### 8.1 Generating Correlation matrix "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TkP_Yy0RzmJR",
        "outputId": "75d657f0-894f-4c73-a975-ea3f26ee5443"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style  type=\"text/css\" >\n",
              "    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col0 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col1 {\n",
              "            background-color:  #f4c5ad;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col2 {\n",
              "            background-color:  #e36b54;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col3 {\n",
              "            background-color:  #d75445;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col4 {\n",
              "            background-color:  #f29274;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col5 {\n",
              "            background-color:  #dbdcde;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col6 {\n",
              "            background-color:  #f7ac8e;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col7 {\n",
              "            background-color:  #f7b599;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col0 {\n",
              "            background-color:  #4f69d9;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col1 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col2 {\n",
              "            background-color:  #445acc;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col3 {\n",
              "            background-color:  #85a8fc;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col4 {\n",
              "            background-color:  #5977e3;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col5 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col6 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col7 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col0 {\n",
              "            background-color:  #e36c55;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col1 {\n",
              "            background-color:  #f1cdba;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col2 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col3 {\n",
              "            background-color:  #f6a385;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col4 {\n",
              "            background-color:  #ee8468;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col5 {\n",
              "            background-color:  #d6dce4;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col6 {\n",
              "            background-color:  #f7b599;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col7 {\n",
              "            background-color:  #f6bda2;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col0 {\n",
              "            background-color:  #da5a49;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col1 {\n",
              "            background-color:  #f6bda2;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col2 {\n",
              "            background-color:  #f7aa8c;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col3 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col4 {\n",
              "            background-color:  #f59c7d;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col5 {\n",
              "            background-color:  #cbd8ee;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col6 {\n",
              "            background-color:  #f5c2aa;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col7 {\n",
              "            background-color:  #f2c9b4;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col0 {\n",
              "            background-color:  #f5a081;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col1 {\n",
              "            background-color:  #efcfbf;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col2 {\n",
              "            background-color:  #f18f71;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col3 {\n",
              "            background-color:  #f5a081;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col4 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col5 {\n",
              "            background-color:  #c6d6f1;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col6 {\n",
              "            background-color:  #f5c0a7;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col7 {\n",
              "            background-color:  #f3c8b2;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col0 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col1 {\n",
              "            background-color:  #7a9df8;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col2 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col3 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col4 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col5 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col6 {\n",
              "            background-color:  #f6a586;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col7 {\n",
              "            background-color:  #f6a283;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col0 {\n",
              "            background-color:  #92b4fe;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col1 {\n",
              "            background-color:  #3b4cc0;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col2 {\n",
              "            background-color:  #85a8fc;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col3 {\n",
              "            background-color:  #7597f6;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col4 {\n",
              "            background-color:  #84a7fc;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col5 {\n",
              "            background-color:  #f6bda2;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col6 {\n",
              "            background-color:  #b40426;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col7 {\n",
              "            background-color:  #c32e31;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col0 {\n",
              "            background-color:  #81a4fb;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col1 {\n",
              "            background-color:  #4257c9;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col2 {\n",
              "            background-color:  #7a9df8;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col3 {\n",
              "            background-color:  #6a8bef;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col4 {\n",
              "            background-color:  #779af7;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col5 {\n",
              "            background-color:  #f7b79b;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col6 {\n",
              "            background-color:  #c32e31;\n",
              "        }    #T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col7 {\n",
              "            background-color:  #b40426;\n",
              "        }</style>  \n",
              "<table id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93\" > \n",
              "<thead>    <tr> \n",
              "        <th class=\"blank level0\" ></th> \n",
              "        <th class=\"col_heading level0 col0\" >RS</th> \n",
              "        <th class=\"col_heading level0 col1\" >RA</th> \n",
              "        <th class=\"col_heading level0 col2\" >OBP</th> \n",
              "        <th class=\"col_heading level0 col3\" >SLG</th> \n",
              "        <th class=\"col_heading level0 col4\" >BA</th> \n",
              "        <th class=\"col_heading level0 col5\" >Playoffs</th> \n",
              "        <th class=\"col_heading level0 col6\" >RD</th> \n",
              "        <th class=\"col_heading level0 col7\" >W</th> \n",
              "    </tr></thead> \n",
              "<tbody>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row0\" class=\"row_heading level0 row0\" >RS</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col0\" class=\"data row0 col0\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col1\" class=\"data row0 col1\" >0.415014</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col2\" class=\"data row0 col2\" >0.904909</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col3\" class=\"data row0 col3\" >0.926384</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col4\" class=\"data row0 col4\" >0.831625</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col5\" class=\"data row0 col5\" >0.371631</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col6\" class=\"data row0 col6\" >0.537539</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row0_col7\" class=\"data row0 col7\" >0.507382</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row1\" class=\"row_heading level0 row1\" >RA</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col0\" class=\"data row1 col0\" >0.415014</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col1\" class=\"data row1 col1\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col2\" class=\"data row1 col2\" >0.367105</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col3\" class=\"data row1 col3\" >0.459514</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col4\" class=\"data row1 col4\" >0.350411</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col5\" class=\"data row1 col5\" >-0.241213</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col6\" class=\"data row1 col6\" >-0.544105</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row1_col7\" class=\"data row1 col7\" >-0.507772</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row2\" class=\"row_heading level0 row2\" >OBP</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col0\" class=\"data row2 col0\" >0.904909</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col1\" class=\"data row2 col1\" >0.367105</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col2\" class=\"data row2 col2\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col3\" class=\"data row2 col3\" >0.806154</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col4\" class=\"data row2 col4\" >0.854055</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col5\" class=\"data row2 col5\" >0.34553</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col6\" class=\"data row2 col6\" >0.494251</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row2_col7\" class=\"data row2 col7\" >0.47408</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row3\" class=\"row_heading level0 row3\" >SLG</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col0\" class=\"data row3 col0\" >0.926384</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col1\" class=\"data row3 col1\" >0.459514</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col2\" class=\"data row3 col2\" >0.806154</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col3\" class=\"data row3 col3\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col4\" class=\"data row3 col4\" >0.814068</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col5\" class=\"data row3 col5\" >0.300191</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col6\" class=\"data row3 col6\" >0.428408</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row3_col7\" class=\"data row3 col7\" >0.405972</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row4\" class=\"row_heading level0 row4\" >BA</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col0\" class=\"data row4 col0\" >0.831625</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col1\" class=\"data row4 col1\" >0.350411</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col2\" class=\"data row4 col2\" >0.854055</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col3\" class=\"data row4 col3\" >0.814068</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col4\" class=\"data row4 col4\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col5\" class=\"data row4 col5\" >0.278772</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col6\" class=\"data row4 col6\" >0.442142</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row4_col7\" class=\"data row4 col7\" >0.416391</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row5\" class=\"row_heading level0 row5\" >Playoffs</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col0\" class=\"data row5 col0\" >0.371631</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col1\" class=\"data row5 col1\" >-0.241213</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col2\" class=\"data row5 col2\" >0.34553</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col3\" class=\"data row5 col3\" >0.300191</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col4\" class=\"data row5 col4\" >0.278772</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col5\" class=\"data row5 col5\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col6\" class=\"data row5 col6\" >0.566274</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row5_col7\" class=\"data row5 col7\" >0.588978</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row6\" class=\"row_heading level0 row6\" >RD</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col0\" class=\"data row6 col0\" >0.537539</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col1\" class=\"data row6 col1\" >-0.544105</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col2\" class=\"data row6 col2\" >0.494251</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col3\" class=\"data row6 col3\" >0.428408</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col4\" class=\"data row6 col4\" >0.442142</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col5\" class=\"data row6 col5\" >0.566274</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col6\" class=\"data row6 col6\" >1</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row6_col7\" class=\"data row6 col7\" >0.938515</td> \n",
              "    </tr>    <tr> \n",
              "        <th id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93level0_row7\" class=\"row_heading level0 row7\" >W</th> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col0\" class=\"data row7 col0\" >0.507382</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col1\" class=\"data row7 col1\" >-0.507772</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col2\" class=\"data row7 col2\" >0.47408</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col3\" class=\"data row7 col3\" >0.405972</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col4\" class=\"data row7 col4\" >0.416391</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col5\" class=\"data row7 col5\" >0.588978</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col6\" class=\"data row7 col6\" >0.938515</td> \n",
              "        <td id=\"T_9c2e2618_d865_11e9_ab8b_685b35930e93row7_col7\" class=\"data row7 col7\" >1</td> \n",
              "    </tr></tbody> \n",
              "</table> "
            ],
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x10c3480b8>"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# the corr() function generates pair wise correlation between features\n",
        "corr = dataset.corr()\n",
        "# following code is used to visually inspect the pair wise correlation\n",
        "corr.style.background_gradient(cmap='coolwarm')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W55vtlnuzmJS"
      },
      "source": [
        "As indicated by the results, RA, OBP. SLG. BA are very strongly correlated. In order to understand the effect of this correlation on the output of multiple linear regression modeling, we create three different models:\n",
        "\n",
        "Model 1: having features RS and OBP\n",
        "\n",
        "Model 2: having features RS, OBP and SLG\n",
        "\n",
        "Model 3: having features RS, OBP, SLG and BA\n",
        "\n",
        "Following section creates data sets for generation of these models. \n",
        "\n",
        "## 8.2 Creating data sets using Hold -out for Generating models in presence of high  correlated features|"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pUlqJ4I6zmJS"
      },
      "outputs": [],
      "source": [
        "# creating data set for model containing RS and OBP\n",
        "\n",
        "My_data_two = dataset.iloc[:,[0,2]] \n",
        "\n",
        "My_data_two_target=dataset.iloc[:,7]\n",
        "\n",
        "\n",
        "X_train_two, X_test_two, Y_train_two, Y_test_two = train_test_split(My_data_two, My_data_two_target, test_size=0.7, random_state=10)\n",
        "\n",
        "\n",
        "# creating data set for model containing RS,  OBP, SLG\n",
        "\n",
        "My_data_three= dataset.iloc[:,[0,2,3]] \n",
        "\n",
        "My_data_three_target=dataset.iloc[:,7]\n",
        "\n",
        "\n",
        "X_train_three, X_test_three, Y_train_three, Y_test_three = train_test_split(My_data_three, My_data_three_target, test_size=0.7, random_state=10)\n",
        "\n",
        "# creating data set for model containing RS,OBP, SLG, BA\n",
        "\n",
        "My_data_four = dataset.iloc[:,[0,2, 3,4]] \n",
        "\n",
        "My_data_four_target=dataset.iloc[:,7]\n",
        "\n",
        "\n",
        "X_train_four, X_test_four, Y_train_four, Y_test_four = train_test_split(My_data_four, My_data_four_target, test_size=0.7, random_state=10)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KucQpN23zmJT"
      },
      "source": [
        "## 8.3 Training  the three new models\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TdF6Rn_FzmJT"
      },
      "outputs": [],
      "source": [
        "\n",
        "#Train the model using X_train_two, Y_train_two associated with data set containing\n",
        "#features RS, OBP\n",
        "\n",
        "LR_model_two = linear_model.LinearRegression()\n",
        "LRfitted_two = LR_model_two.fit(X_train_two, Y_train_two)\n",
        "\n",
        "#Train the model using X_train_three, Y_train_three associated with data set containing\n",
        "#features RS, OBP, SLG\n",
        "\n",
        "LR_model_three = linear_model.LinearRegression()\n",
        "LRfitted_three = LR_model_three.fit(X_train_three, Y_train_three)\n",
        "\n",
        "#Train the model using X_train_four, Y_train_four associated with data set containing\n",
        "#features RS, OBP, SLG, BA\n",
        "\n",
        "LR_model_four = linear_model.LinearRegression()\n",
        "LRfitted_four = LR_model_four.fit(X_train_four, Y_train_four)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZgvEpdG4zmJU"
      },
      "source": [
        "## 8.4 Predicting training and test error for different models\n",
        "\n",
        "The training and test errors gives a good indication of overfitting. High training error and low test errors is indication to model overfitting. The code below evaluate model performance on training and test set. The errors are computed using the difference between actual and predicted values by the model. To do this, we need to predict the training and testing errors using predict() function on all the models generated so far :\n",
        "\n",
        "Model 1: having features RS and OBP\n",
        "\n",
        "Model 2: having features RS, OBP and SLG\n",
        "\n",
        "Model 3: having features RS, OBP, SLG and BA\n",
        "\n",
        "Model 4: original model containing all features\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "65ARmwx0zmJU"
      },
      "outputs": [],
      "source": [
        "# Predicting training and testing error on X_train_two, X_test_two \n",
        "# (model with features RS and OBP)\n",
        "\n",
        "Prediction_train_two = LRfitted_two.predict(X_train_two)\n",
        "Prediction_test_two = LRfitted_two.predict(X_test_two)\n",
        "\n",
        "\n",
        "# Predicting training and testing error on X_train_three, X_test_three \n",
        "# (model with features RS, OBP, SLG )\n",
        "\n",
        "Prediction_train_three = LRfitted_three.predict(X_train_three)\n",
        "Prediction_test_three = LRfitted_three.predict(X_test_three)\n",
        "\n",
        "# Predicting training and testing error on X_train_four, X_test_four \n",
        "# (model with features RS, OBP, SLG, BA )\n",
        "\n",
        "Prediction_train_four = LRfitted_four.predict(X_train_four)\n",
        "Prediction_test_four = LRfitted_four.predict(X_test_four)\n",
        "\n",
        "#Predicting training and testing error on X_train, X_test \n",
        "# (model with all features RS,RA, OBP, SLG, BA, Playoff, RD )\n",
        "\n",
        "Prediction_train_all = LRfitted.predict(X_train)\n",
        "Prediction_test_all = LRfitted.predict(X_test)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lCL3XmUjzmJU"
      },
      "source": [
        "## 8.5 Preparing summary of  Train, Test error, Sum of absolute weights for different models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rVcVPTZZzmJU",
        "outputId": "72062740-dae1-4e0b-db4b-206e83c0758a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "           Model  Train Errors  Test Errors         SAW Rsquare %\n",
            "0     RS and OBP          8.02         8.00   83.934237       ---\n",
            "1     RS,OBP,SLG          7.83         7.86  214.225402       ---\n",
            "2  RS,OBP,SLG,BA          7.80         7.86  383.060596       ---\n",
            "3   All features          3.09         3.09  145.769913     88.32\n"
          ]
        }
      ],
      "source": [
        "\n",
        "columns =['Model', 'Train Errors', 'Test Errors', 'SAW', 'Rsquare %']\n",
        "######################## Summary of Model one ###########################\n",
        "# Summary of training,test error and sum of absolute Weights(SAW) for model containing \n",
        "# features RS and OBP\n",
        "\n",
        "Reg_model1 = \"RS and OBP\"\n",
        "\n",
        "Reg_model1_values = [Reg_model1, \n",
        "                 round(metrics.mean_absolute_error(Y_train_two, Prediction_train_two),2),\n",
        "                round(metrics.mean_absolute_error(Y_test_two, Prediction_test_two),2),\n",
        "              np.absolute(LRfitted_two.coef_).sum() + np.absolute(LRfitted_two.intercept_),\n",
        "                 '---'   ]\n",
        "######################## Summary of Model two ###########################\n",
        "# Summary of training,test error and sum of absolute Weights(SAW) for model containing \n",
        "# features RS ,OBP, SLG\n",
        "\n",
        "Reg_model2 = \"RS,OBP,SLG\"\n",
        "\n",
        "Reg_model2_values = [Reg_model2, \n",
        "               round(metrics.mean_absolute_error(Y_train_three, Prediction_train_three),2),\n",
        "               round(metrics.mean_absolute_error(Y_test_three, Prediction_test_three),2),\n",
        "            np.absolute(LRfitted_three.coef_).sum() +np.absolute(LRfitted_three.intercept_),\n",
        "               \"---\"     ]\n",
        "######################## Summary of Model three  ###########################\n",
        "# Summary of training,test error and sum of absolute Weights(SAW) for model containing \n",
        "# features RS ,OBP, SLG, BA\n",
        "\n",
        "Reg_model3 = \"RS,OBP,SLG,BA\"\n",
        "\n",
        "Reg_model3_values = [Reg_model3, \n",
        "                round(metrics.mean_absolute_error(Y_train_four, Prediction_train_four),2),\n",
        "                round(metrics.mean_absolute_error(Y_test_four, Prediction_test_four),2),\n",
        "             np.absolute(LRfitted_four.coef_).sum() + np.absolute(LRfitted_four.intercept_),\n",
        "                 '---'   ]\n",
        "\n",
        "\n",
        "######################## Summary of Model four  ###########################\n",
        "# Summary of training,test error and sum of absolute Weights(SAW) for model containing \n",
        "# all features\n",
        "\n",
        "Reg_model4 = \"All features\"\n",
        "\n",
        "Reg_model4_values = [Reg_model4, \n",
        "                 round(metrics.mean_absolute_error(Y_train, Prediction_train),2),\n",
        "                round(metrics.mean_absolute_error(Y_test, Prediction_test),2),\n",
        "           np.absolute( LRfitted.coef_).sum() + np.absolute(LRfitted.intercept_),\n",
        "            round(metrics.r2_score(Y_test, Prediction_test),4)*100 ]\n",
        "\n",
        "Models_summary = pd.DataFrame([Reg_model1_values, Reg_model2_values, \n",
        "                               Reg_model3_values,Reg_model4_values\n",
        "                               ],\n",
        "                              columns= columns)\n",
        "print(Models_summary)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pLuzg8EJzmJV"
      },
      "source": [
        "As indicated by the train and test errors on all the models, overfitting does not seem to an issue. However, as the number of correlated features increases in the model, the complexity grows, as indicated by the column SAW(sum of absolute weights). \n",
        "\n",
        "The next part of this tutorial aims to produce low complex model by treating the correlated features present in the data set using Regularization. It is a technique that controls overfitting, produces less complex model with higher performance rates. There are three types of regularization:\n",
        "\n",
        "1. Ridge(L2) \n",
        "\n",
        "    - it produces low complex model\n",
        "    - reduces the collinearity\n",
        "    - not suitable for high dimensional data\n",
        "    \n",
        "2. Lasso (L1)\n",
        "    \n",
        "    - provides important features for deciding the outcome. \n",
        "    - reduces the collinearity\n",
        "    - produces low complex model\n",
        "    - suitable for high dimensional data\n",
        "    \n",
        "3. Elastic net\n",
        "\n",
        "    - combines features of both L1 and L2\n",
        "    \n",
        "    \n",
        "    "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zXhVSBwizmJV"
      },
      "source": [
        "## 8.5 Implementing Ridge\n",
        "\n",
        "Ridge will be implemented for on the original data set to make it less complex (by\n",
        "removing the collinearity that results in lowering the sum of absolute weights). The\n",
        "Ridge model is also evaluated for training and testing errors. The final results of Ridge application is added in the data frame containing previous results\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OWum_thrzmJV"
      },
      "outputs": [],
      "source": [
        "# Using Ridge function, create its object. Ridge is controlled by parameter alpha. It can \n",
        "#be set by the user to tune results. \n",
        "\n",
        "ridge = linear_model.Ridge(alpha =1)\n",
        "\n",
        "# Fitting ridge on data set containing all features\n",
        "ridge.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "Ridge_pred_train = ridge.predict(X_train)\n",
        "Ridge_pred_test= ridge.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "\n",
        "Reg_model_Ridge = \"Ridge on All features\"\n",
        "Ridge_values = [Reg_model_Ridge,\n",
        "        round(metrics.mean_absolute_error(Y_train, Ridge_pred_train),2),\n",
        "        round(metrics.mean_absolute_error(Y_test, Ridge_pred_test),2),\n",
        "        np.absolute(ridge.coef_).sum() +np.absolute(ridge.intercept_ ),\n",
        "         round(metrics.r2_score(Y_test, Ridge_pred_test),4)*100           ]\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H2mM9wE4zmJW"
      },
      "source": [
        "## 8.6 Implementing Lasso Regression\n",
        "\n",
        "The following code implements LASSO on the original data set to make it less complex (by\n",
        "removing the collinearity that results in lowering the sum of absolute weights) andto find important features for the decision variable. The LASSO model is also evaluated for training and testing errors. The final results of LASSO application is added in the data frame containing previous results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KmGrBWkuzmJW",
        "outputId": "351c2eeb-8191-4b2a-9378-0a58faefbab7"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:475: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 3.7025394530592166, tolerance: 3.6041718518518517\n",
            "  positive)\n"
          ]
        }
      ],
      "source": [
        "# Loading library\n",
        "\n",
        "\n",
        "# Using lasso function, create its object. lasso is controlled by parameter alpha. It can \n",
        "#be set by the user to tune results. \n",
        "\n",
        "lasso = linear_model.Lasso(alpha =0.2)\n",
        "\n",
        "# Fitting lasso on data set containing all features\n",
        "lasso.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "lasso_pred_train = lasso.predict(X_train)\n",
        "lasso_pred_test= lasso.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "Reg_model_Lasso = \"Lasso on All features\"\n",
        "Lasso_values = [Reg_model_Lasso,\n",
        "        round(metrics.mean_absolute_error(Y_train, lasso_pred_train),2),\n",
        "        round(metrics.mean_absolute_error(Y_test, lasso_pred_test),2),\n",
        "        np.absolute(lasso.coef_).sum() +np.absolute(lasso.intercept_ ),\n",
        "         round(metrics.r2_score(Y_test, lasso_pred_test),4)*100           ]\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GEv1PZUVzmJW"
      },
      "source": [
        "### 8.6.1 Analysing important features as resulted by LASSO regularization\n",
        "\n",
        "The key aim is to discover how many variables Lasso picked out of total list of features present in the data set ? For non relevant features, LASSO makes the coefficient equal to \n",
        "zero.  The following code find all those features where the coefficients(weights) of the \n",
        "LASSO model for features were discovered to be zero. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O8duq3WYzmJX",
        "outputId": "63e4bc43-729c-4e7a-acb2-afc9a3d462b4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "Lasso picked 4 features and eliminated the other 3 variables\n",
            "\n",
            "\n",
            "List of Relevant features: \n",
            "  ['OBP', 'SLG', 'BA']\n",
            "\n",
            "\n",
            "List of Relevant features: \n",
            "  ['RS', 'RA', 'Playoffs', 'RD']\n"
          ]
        }
      ],
      "source": [
        "# Performance of Lasso \n",
        "print(\"\\n\")\n",
        "print(\"Lasso picked \" + str(sum(lasso.coef_!= 0)) \n",
        "      + \" features and eliminated the other \" +  \n",
        "      str(sum(lasso.coef_ == 0)) + \" variables\")\n",
        "\n",
        "print(\"\\n\")\n",
        "# Printing relevant and nonrelevant features as discovered by Lasso \n",
        "Relevant_features =[]\n",
        "Nonrelevant_features =[]\n",
        "for i in range(len(dataset.columns)-1):\n",
        "    if(lasso.coef_[i]==0):\n",
        "        Relevant_features.append(dataset.columns[i])\n",
        "    else:\n",
        "        Nonrelevant_features.append(dataset.columns[i])\n",
        "    \n",
        "print(\"List of Relevant features: \\n \", Relevant_features)\n",
        "print(\"\\n\")\n",
        "print(\"List of Relevant features: \\n \", Nonrelevant_features)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A9_eWot6zmJX"
      },
      "source": [
        "LASSO discovers features RS, RA, Playoff and RD as only relevant features. It means that\n",
        "only these features are enough to be used out of total available for model building. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a7Tf701azmJX"
      },
      "source": [
        "## 8.7 Implementing ElasticNet Regression\n",
        "\n",
        "The following code implements Elasticnet on the original data set. The ElasticNet  model is also evaluated for training and testing errors. The final results of ElasticNet application is added in the data frame containing previous results\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iM5_Sm-0zmJY"
      },
      "outputs": [],
      "source": [
        "\n",
        "# Using elasticnet function, create its object. elasticnet is controlled by parameter alpha. It can \n",
        "#be set by the user to tune results. \n",
        "\n",
        "elasticnet = linear_model.ElasticNet(alpha =0.2)\n",
        "\n",
        "# Fitting ridge on data set containing all features\n",
        "elasticnet.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "elasticnet_pred_train = elasticnet.predict(X_train)\n",
        "elasticnet_pred_test= elasticnet.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "Reg_model_elasticnet = \"Elasticnet on All features\"\n",
        "elasticnet_values = [Reg_model_elasticnet,\n",
        "        round(metrics.mean_absolute_error(Y_train, elasticnet_pred_train),2),\n",
        "        round(metrics.mean_absolute_error(Y_test, elasticnet_pred_test),2),\n",
        "        np.absolute(elasticnet.coef_).sum() +np.absolute(elasticnet.intercept_ ),\n",
        "         round(metrics.r2_score(Y_test, elasticnet_pred_test),4)*100           ]\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7GKelAaezmJY"
      },
      "source": [
        "## 8.8 Evaluating results of all models\n",
        "\n",
        "In this section we add, results of Ridge, LASSO and ElasticNet to the Pandas DataFrame created earlier for comparison of results. \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rRX4jvuWzmJY",
        "outputId": "0c6072ef-5009-4c3a-e26b-0ab05377a57e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "                        Model  Train Errors  Test Errors         SAW Rsquare %\n",
            "0                  RS and OBP          8.02         8.00   83.934237       ---\n",
            "1                  RS,OBP,SLG          7.83         7.86  214.225402       ---\n",
            "2               RS,OBP,SLG,BA          7.80         7.86  383.060596       ---\n",
            "3                All features          3.09         3.09  145.769913     88.32\n",
            "4       Ridge on All features          3.09         3.09   86.457191     88.34\n",
            "5       Lasso on All features          3.14         3.07   84.124461      88.5\n",
            "6  Elasticnet on All features          3.16         3.07   83.793031     88.47\n"
          ]
        }
      ],
      "source": [
        "# putting summary of results in Pandas dataframe and printing it\n",
        "columns =['Model', 'Train Errors', 'Test Errors', 'SAW', 'Rsquare %']\n",
        "regularization = pd.DataFrame([ Ridge_values, Lasso_values,elasticnet_values ],\n",
        "                              columns= columns)\n",
        "\n",
        "\n",
        "results = [Models_summary, regularization]\n",
        "All_models_summary= pd.concat(results, ignore_index=True)                           \n",
        "print(All_models_summary)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0QBSSCDLzmJY"
      },
      "source": [
        "The regularization methods have significantly decreased the sum of absolute weights by keeping low training and test errors and high rsquare value."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OtwW_zuQzmJY"
      },
      "source": [
        "## 8.9 Generating Regression equations of all models and comparing their weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-f2nWxZ5zmJZ",
        "outputId": "22c9b7fe-9d2d-4e9e-a95f-156d07c420e6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "The LR model on RS and OBP \n",
            "\n",
            "W = 22.33 +  RS X 0.06 + OBP X 61.55\n",
            "\n",
            "\n",
            "The LR model on RS,OBP, SLG \n",
            "\n",
            "W = 55.3 +  RS X 0.11 + OBP X 16.06 + SLG X -142.75\n",
            "\n",
            "\n",
            "The LR model on RS, OBP, SLG, BA \n",
            "\n",
            "W = 50.52 +  RS X 0.11 + OBP X -44.44 + SLG X -164.48 + BA X 123.51\n",
            "\n",
            "\n",
            "The LR model on RS, RA, OBP, SLG, BA, Playoff, RD \n",
            "\n",
            "W = 70.86 + RS X 0.02 + RA X -0.04 + OBP X 41.83 + SLG X 21.09 + BA X -8.17 + Playoff X 3.69 + RD X 0.06\n",
            "\n",
            "\n",
            "The Ridge model on the dataset \n",
            "\n",
            "W = 81.54 + RS X 0.03 + RA X -0.03 + OBP X 0.37 + SLG X 0.66 + BA X 0.2 + Playoff X 3.56 + RD X 0.07\n",
            "\n",
            "\n",
            "The Lasso model on the data set\n",
            "\n",
            "W = 81.54 + RS X 0.07 + RA X -0.07 + OBP X 0.0 + SLG X 0.0 + BA X 0.0 + Playoff X 1.91 + RD X 0.04\n",
            "\n",
            "\n",
            "The elasticnet model on the data set\n",
            "\n",
            "W = 82.13 + RS X 0.08 + RA X -0.08 + OBP X 0.0 + SLG X 0.0 + BA X 0.0 + Playoff X 1.48 + RD X 0.02\n"
          ]
        }
      ],
      "source": [
        "############ model on features RS, OBP #############\n",
        "\n",
        "LR_model_one_features_names = list(['RS', 'OBP'])\n",
        "\n",
        "LR_model_one_table_coeff= pd.DataFrame({'Feature_name':LR_model_one_features_names ,\n",
        "                                        'Coefficients': LRfitted_two.coef_})  \n",
        "\n",
        "# printing the Linear regression model\n",
        "print(\"\\n\")\n",
        "print(\"The LR model on RS and OBP \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(LRfitted_two.intercept_,2), \"+\",\n",
        "      \" RS\" \" X\", round(LRfitted_two.coef_[0],2),\n",
        "     \"+ OBP\" \" X\"  ,round(LRfitted_two.coef_[1],2))\n",
        "     \n",
        "print(\"\\n\")\n",
        "\n",
        "\n",
        "############ model on features RS, OBP, SLG #############\n",
        "\n",
        "LR_model_two_features_names = list(['RS', 'OBP', 'SLG'])\n",
        "\n",
        "LR_model_two_table_coeff= pd.DataFrame({'Feature_name':LR_model_two_features_names ,\n",
        "                                        'Coefficients': LRfitted_three.coef_})  \n",
        "\n",
        "\n",
        "# printing the Linear regression model\n",
        "\n",
        "print(\"The LR model on RS,OBP, SLG \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(LRfitted_three.intercept_,2), \"+\",\n",
        "      \" RS\" \" X\", round(LRfitted_three.coef_[0],2),\n",
        "     \"+ OBP\" \" X\"  ,round(LRfitted_three.coef_[1],2),  \n",
        "      \"+ SLG\" \" X\"  ,round(LRfitted_three.coef_[2],2)\n",
        "     )\n",
        "print(\"\\n\")\n",
        "\n",
        "\n",
        "############ model on features RS, OBP, SLG, BA #############\n",
        "\n",
        "\n",
        "\n",
        "LR_model_three_features_names = list(['RS', 'OBP', 'SLG', 'RA'])\n",
        "\n",
        "LR_model_three_table_coeff= pd.DataFrame({'Feature_name':LR_model_three_features_names ,\n",
        "                                        'Coefficients': LRfitted_four.coef_})  \n",
        "\n",
        "\n",
        "# printing the Linear regression model\n",
        "\n",
        "print(\"The LR model on RS, OBP, SLG, BA \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(LRfitted_four.intercept_,2), \"+\", \n",
        "      \" RS\" \" X\", round(LRfitted_four.coef_[0],2), \n",
        "     \"+ OBP\" \" X\"  ,round(LRfitted_four.coef_[1],2),  \n",
        "      \"+ SLG\" \" X\"  ,round(LRfitted_four.coef_[2],2),\n",
        "     \"+ BA\" \" X\", round(LRfitted_four.coef_[3],2)\n",
        "      )\n",
        "print(\"\\n\")\n",
        "\n",
        "\n",
        "\n",
        "############ model on features RS, OBP, SLG, BA, Playoff, RD #############\n",
        "\n",
        "\n",
        "\n",
        "LR_model_features_names = list(['RS','RA','OBP', 'SLG', 'BA', 'Playoff', 'RD'])\n",
        "\n",
        "LR_model_table_coeff= pd.DataFrame({'Feature_name':LR_model_features_names ,\n",
        "                                        'Coefficients': LRfitted.coef_})  \n",
        "\n",
        "\n",
        "# printing the Linear regression model\n",
        "\n",
        "print(\"The LR model on RS, RA, OBP, SLG, BA, Playoff, RD \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(LRfitted.intercept_,2), \n",
        "      \"+ RS\" \" X\", round(LRfitted.coef_[0],2), \n",
        "      \"+ RA\" \" X\"  ,round(LRfitted.coef_[1],2),\n",
        "      \"+ OBP\" \" X\"  ,round(LRfitted.coef_[2],2), \n",
        "      \"+ SLG\" \" X\"  ,round(LRfitted.coef_[3],2),\n",
        "       \"+ BA\" \" X\", round(LRfitted.coef_[4],2),\n",
        "      \"+ Playoff\" \" X\", round(LRfitted.coef_[5],2),\n",
        "      \"+ RD\" \" X\", round(LRfitted.coef_[6],2),\n",
        "      \n",
        "\n",
        "      )\n",
        "print(\"\\n\")\n",
        "\n",
        "\n",
        "############ Ridge model on features RS, RA OBP, SLG, BA, Playoff, RD #############\n",
        "\n",
        "\n",
        "Ridge_Feature_names = list(My_data.columns.values)\n",
        "\n",
        "\n",
        "Ridge_table_coeff= pd.DataFrame({'Feature_name':Ridge_Feature_names , \n",
        "                                 'Coefficients': ridge.coef_})  \n",
        "\n",
        "# printing the Ridge Linear regression model\n",
        "print(\"The Ridge model on the dataset \\n\")\n",
        "\n",
        "print(\"W =\"  ,round(ridge.intercept_,2), \n",
        "      \"+ RS\" \" X\", round(ridge.coef_[0],2), \n",
        "      \"+ RA\" \" X\"  ,round(ridge.coef_[1],2),\n",
        "      \"+ OBP\" \" X\"  ,round(ridge.coef_[2],2), \n",
        "      \"+ SLG\" \" X\"  ,round(ridge.coef_[3],2),\n",
        "       \"+ BA\" \" X\", round(ridge.coef_[4],2),\n",
        "      \"+ Playoff\" \" X\", round(ridge.coef_[5],2),\n",
        "      \"+ RD\" \" X\", round(ridge.coef_[6],2),\n",
        "      \n",
        "\n",
        "      )\n",
        "print(\"\\n\")\n",
        "\n",
        "\n",
        "\n",
        "############ Lasso model on features RS, RA OBP, SLG, BA, Playoff, RD #############\n",
        "\n",
        "\n",
        "Lasso_Feature_names = list(My_data.columns.values)\n",
        "\n",
        "Lasso_table_coeff= pd.DataFrame({'Feature_name':Lasso_Feature_names , \n",
        "                                 'Coefficients': lasso.coef_})  \n",
        "\n",
        "# printing the Lasso Linear regression model\n",
        "print(\"The Lasso model on the data set\\n\")\n",
        "print(\"W =\"  ,round(ridge.intercept_,2), \n",
        "      \"+ RS\" \" X\", round(lasso.coef_[0],2), \n",
        "      \"+ RA\" \" X\"  ,round(lasso.coef_[1],2),\n",
        "      \"+ OBP\" \" X\"  ,round(lasso.coef_[2],2), \n",
        "      \"+ SLG\" \" X\"  ,round(lasso.coef_[3],2),\n",
        "       \"+ BA\" \" X\", round(lasso.coef_[4],2),\n",
        "      \"+ Playoff\" \" X\", round(lasso.coef_[5],2),\n",
        "      \"+ RD\" \" X\", round(lasso.coef_[6],2),\n",
        "      \n",
        "\n",
        "      )\n",
        "\n",
        "\n",
        "############ Elasticnet model on features RS, RA OBP, SLG, BA, Playoff, RD #############\n",
        "print(\"\\n\")\n",
        "\n",
        "Elasticnet_Feature_names = list(My_data.columns.values)\n",
        "\n",
        "Elasticnet_table_coeff= pd.DataFrame({'Feature_name':Elasticnet_Feature_names , \n",
        "                                 'Coefficients': elasticnet.coef_})  \n",
        "\n",
        "# printing the Elasticnet Linear regression model\n",
        "print(\"The elasticnet model on the data set\\n\")\n",
        "\n",
        "print(\"W =\"  ,round(elasticnet.intercept_,2), \n",
        "      \"+ RS\" \" X\", round(elasticnet.coef_[0],2), \n",
        "      \"+ RA\" \" X\"  ,round(elasticnet.coef_[1],2),\n",
        "      \"+ OBP\" \" X\"  ,round(elasticnet.coef_[2],2), \n",
        "      \"+ SLG\" \" X\"  ,round(elasticnet.coef_[3],2),\n",
        "       \"+ BA\" \" X\", round(elasticnet.coef_[4],2),\n",
        "      \"+ Playoff\" \" X\", round(elasticnet.coef_[5],2),\n",
        "      \"+ RD\" \" X\", round(elasticnet.coef_[6],2),\n",
        "      \n",
        "\n",
        "      )\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zHLlK3sRzmJZ"
      },
      "source": [
        "## 8.10 Visualization of Results of all models on significance of features\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3BGTUVAjzmJZ",
        "outputId": "2095bbf7-011a-41af-e4ab-a7338de5ff7b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x10c861c88>"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm4AAAJOCAYAAAAOBIslAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XuYZGV5tv3zkmHrKKh0kO2MIAFRccBRlE8NkRhRQdSIQIwBTSQm+BqVaCSahNeoURPzuc3ni4mixgBGRdloFI0b8grIAMNOVEAhqGyGvciAMtzfH7Uaiqa7p5np2jwz5+846qBqPWut566arpur1lrVnapCkiRJ4+8hoy5AkiRJc2NwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU3qJLkyye/MYb3FSSrJgmHUJWk0knw0yV/PMl5JHjvMmtZWkmOS/Nsc1/1Wkj8edE16cAxuGriZAlGSfZLck+T2JL9I8sMkrxxFjZLWP11vWtn1oGuTHJdk4eR4Vb2mqv5ulDVKUxncNGo/r6qFwMOBNwAfS7LLiGuStP44oOtBS4A9gKNHXI80K4ObxkL1fBm4Cdh9unX6TlG+MsnVSW5O8pokT0lyYZJbkny4b/2HJHlbkquSXJ/kU0k27xt/RTd2Y5K3TpnrIUnekuSKbvyzSR45qOcvabSq6lrgq/QCHADdEbh39D1+U5Jrkvw8yav6t0/yqCSnJLktyTlJ3pHkv/vGd01yepKburMLL5uplu4U5TuSfLc7GnhKt//P9O1/cd/6e3fLbu3+u3ff2GOSfLs7q3E6sOWUuZ7WzXNLkguS7LMmr5+Gx+CmsdAFpRfSayqXr2b1vYCdgYOB9wNvBX4HeDzwsiS/1a13eHf7bWBHYCHw4W6+3YD/D3gFsA3wKGC7vjn+F/Ai4Le68ZuBj6zFU5Q0xpJsBzyPGfpPkv2AvwCeQ6//TL384yPAL4FHA4d1t8ltHwqcDvw78BvAIcA/d31oJofQ60/bAjsBZwKfAB4JXAr8bbfvRwKnAR+k18f+CTgtyaO6/fw7cC693vp3U+rattv2Hd1+/wL4fJKJWerSiBncNGrbJLkFWAmcBLyxqs5fzTZ/V1V3VtXX6DXK46vq+qr6GXAGvdMdAC8H/qmqflxVt9M7BXJI96WClwKnVtV3quou4K+Be/rmeA3w1qr6aTd+DPBSv5AgrXO+mOQXwNXA9XSBaBovAz5RVRdX1S/p9QQAkmwA/B7wt1V1R1V9H/hk37b7A1dW1Seq6u6ux30eOGiWuj5RVVdU1a3AV4ArqurrVXU38B/c1+deAFxWVZ/u9n088APggCQ7AE8B/rqq7qqq7wCn9M3xB8CXq+rLVXVPVZ0OLAOeP/tLplEyuGnUfl5VW9C7xu2DwLPnsM11ffdXTvN48uLibYCr+sauAhYAW3VjV08OdI34xr51FwEndacPbqH3CXdVt62kdceLquphwD7Arkw5ldjnfj2D+/eWCXq9pX+8//4iYK/JftL1lJfTOzo3kzXtc5O1bduN3dz1t+nqXgQcNKWuZwBbz1KXRszgprHQHdX6S+CJSV40T7v9Ob3GNGkH4G56DfAaYPvJgSSb0TvNMOlq4HlVtUXfbZPuqJ6kdUxVfRs4DvjHGVa5X8+g108mraDXW/ovt+hf92rg21P6ycKq+tO1r/wBfW6ytp91NT+iO1U7Xd1XA5+eUtdDq+rd81CXBsTgpmHZMMkmfbcHnHKsql8B7wP+Zp7mPB54Q3dx7kLgXcCJ3amGzwH7J3lGko2At3P/98NHgXcmWQSQZCLJgfNUl6Tx9H7gOUmeNM3YZ4HDk+zWfdC795RqVa0CvgAck2SzJLsCf9i37anAb3ZfiNqwuz0lyePmoeYvd/v+/SQLkhwM7EbvUpCr6J36/N9JNkryDOCAvm3/jd4p1ecm2aDrzft01/tpTBncNCxfpnd4f/J2zAzrfRzYIckBM4w/GB8HPg18B/gJcCe9Lx1QVZcAR9K7cPcael8++Gnfth8ATga+1l3/cha9L0VIWkdV1QrgU0zz4bGqvkIv2P0XvS8w/NeUVV4LbA5cS6/vHA/c1W37C+B36X3h4OfdOu8BNp6Hmm+kdw3dUfQu93gzsH9V3dCt8vv0etdN9MLmp/q2vRo4EPgrekcNrwbehNlgrKWqRl2DJEnrlCTvAR5dVYetdmXpQTBVS5K0lrrf07Z7ep4K/BG9b8pL88pfbSBJ0tp7GL3To9vQ+wLU+4AvjbQirZM8VSpJktQIT5VKkiQ1Yp09VbrlllvW4sWLR12GpCE599xzb6iqdeJP9di/pPXPXHvYOhvcFi9ezLJly0ZdhqQhSTL1t8c3y/4lrX/m2sM8VSpJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjVhnf4+bJK1vfvqWM+73eLt3P3NElUgaFI+4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDViLIJbklVJlie5OMkpSbboli9OsjLJ+UkuTfK9JIePuFxJupf9S9IwjUVwA1ZW1ZKqegJwE3Bk39gVVbVHVT0OOAR4fZJXjqRKSXog+5ekoRmX4NbvTGDb6Qaq6sfAG4HXDbUiSZob+5ekgRqr4JZkA2Bf4ORZVjsP2HWG7Y9IsizJshUrVgyiREmalv1L0jCMS3DbNMly4FpgK+D0WdbNTANVdWxVLa2qpRMTE/NdoyRNx/4laWjGJbitrKolwCJ6je3IWdbdA7h0KFVJ0urZvyQNzbgENwCq6g56138cleQBf0c1yWLgH4EPDbcySZqd/UvSMIzdH5mvqvOTXAgcCpwB7JTkfGAT4BfAB6vquBGWKEnTsn9JGrSxCG5VtXDK4wP6Hm465HIkac7sX5KGaaxOlUqSJGlmBjdJkqRGGNwkSZIaMRbXuEmS1t52737mqEuQNGAecZMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIasWDUBWhm7zt4/7Xex1EnnjoPlUhqwXz0DEnzY1D///WImyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjVhvckqxKsjzJxUn+I8lm3fLb57uYJP+Q5JLuvxNJzk5yfpJnzvdcktYP9jBJ65K5HHFbWVVLquoJwK+A1wywniOA3avqTcC+wEVVtUdVnTHAOSWt2+xhktYZD/ZU6RnAY/sXJFmY5BtJzktyUZIDu+VvT/L6vvXemeTP0/MP3affi5Ic3I2fDCwEzk3yl8B7gQO7T8oPTXJc3zZvWJsnLWm9ZQ+T1LQ5/wLeJAuA5wH/OWXoTuDFVXVbki2Bs7oG9nHgC8D7kzwEOAR4KvASYAnwJGBL4Jwk36mqFya5vaqWdPNdByytqtcmeTKwbfeJmSRbzFDjEfQ+8bLDDjvM9alJWg+Mew+zf0mai7kccds0yXJgGfA/wL9OGQ/wriQXAl8HtgW2qqorgRuT7AH8LnB+Vd0IPAM4vqpWVdV1wLeBp6ymhh8DOyb5UJL9gNumW6mqjq2qpVW1dGJiYg5PTdJ6oIkeZv+SNBdzOeK2cvIT5AxeDkwAT66qXye5EtikG/sX4HDg0fQ+va6Rqro5yZOA59K7PuVlwKvWdH+S1iv2MEnrjPn4dSCbA9d3De+3gUV9YycB+9H7NPrVbtkZwMFJNkgyATwL+N5sE3SnLx5SVZ8H3gbsOQ91SxLYwyQ1ZD7+yPxngFOSXETvVMQPJgeq6ldJvgncUlWrusUnAU8HLgAKeHNVXbuaObYFPtFdZwJw9DzULUlgD5PUkNUGt6paONvyqrqBXhN7gK5JPQ04qG+7At7U3Wacq6qOA47r7l+An1AlrQF7mKR1ycD+ckKS3YDLgW9U1WWDmkeSBsEeJmkczcep0mlV1feBHQe1f0kaJHuYpHHk3yqVJElqxMCOuGntHXXiqaMuQVJD7BnSus8jbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNWDDqAsbBR17zX6MuYVpHfvTZoy5BUkMebC+zx0jt8YibJElSIwxukiRJjTC4SZIkNcLgJkmS1IixCW5JViVZnuSCJOcl2XvK+OuT3Jlk81HVKEnTsX9JGpaxCW7AyqpaUlVPAo4G/n7K+KHAOcBLhl6ZJM3O/iVpKMYpuPV7OHDz5IMkOwELgbfRa4CSNK7sX5IGZpx+j9umSZYDmwBbA/2/YOgQ4ATgDGCXJFtV1XVTd5DkCOAIgB122GHwFUtSj/1L0lCM0xG3yVMNuwL7AZ9Kkm7sUOCEqroH+Dxw0HQ7qKpjq2ppVS2dmJgYTtWSZP+SNCTjdMTtXlV1ZpItgYkkWwE7A6d3fXAj4CfAh0dYoiRNy/4laZDG6YjbvZLsCmwA3Ejv0+oxVbW4u20DbJNk0UiLlKRp2L8kDdI4HXGbvEYEIMBhVbUqySHA86esexK960beM8wCJWkG9i9JQzE2wa2qNphh+Y7TLHvj4CuSpLmxf0kalrE8VSpJkqQHMrhJkiQ1wuAmSZLUiLG5xm2Ujvzos1e/kiSNOXuZtO7ziJskSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIxaMugDN7NJdHzfqEtZ7j/vBpaMuQZqzVnqG7ytpzXnETZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRIwluSd6a5JIkFyZZnmSvJN9KsnSadZ/ajV2W5LwkpyV54ijqliT7l6RRGvqvA0nydGB/YM+quivJlsBGM6y7FfBZ4Per6rvdsmcAOwEXDalkSQLsX5JGbxS/x21r4Iaqugugqm4ASDLduq8FPjnZ9Lr1/3sYRUrSNOxfkkZqFKdKvwZsn+RHSf45yW/Nsu7jgfPmuuMkRyRZlmTZihUr1rpQSZrC/iVppIYe3KrqduDJwBHACuDEJIfPZdskZye5NMkHZtj3sVW1tKqWTkxMzFvNkgT2L0mjN5I/eVVVq4BvAd9KchFw2AyrXgLsCXyp226vJC+ld42JJA2d/UvSKA39iFuSXZLs3LdoCXDVDKt/BDg8yd59yzYbWHGSNAv7l6RRG8URt4XAh5JsAdwNXE7vtMPngNOS/Lpb78yqOijJwcB7kmwLXA/cALx9BHVLkv1L0kgNPbhV1bnA3tMM7TPD+mcBs10ALElDYf+SNGr+5QRJkqRGGNwkSZIaYXCTJElqxEh+HYjm5nE/uHTUJUhqiD1DWvd5xE2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqxIJRF6CZPfGTTxx1CdJIXXTYRaMuoSmD7Bn+W0jjwSNukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0YanBLsl2SLyW5LMkVST6QZKMk+yS5NcnyJBcm+XqS3+i2OTzJim7s+0lePcyaJWmSPUzSqA0tuCUJ8AXgi1W1M/CbwELgnd0qZ1TVkqraHTgHOLJv8xOragmwD/CuJFsNq25JAnuYpPEwzCNuzwburKpPAFTVKuANwKuAzSZX6prjw4Cbp+6gqq4HrgAWDaNgSepjD5M0csP8BbyPB87tX1BVtyX5H+CxwDOTLAceBfwS+KupO0iyI7AjcPl0EyQ5AjgCYIcddpjX4iWt9wbaw+xfkuZinL6cMHmaYXvgE8B7+8YO7hri8cCfVNVN0+2gqo6tqqVVtXRiYmIIJUvSvdaqh9m/JM3FMIPb94En9y9I8nBgBx746fNk4Fl9j0/sGuJeVXXSYMuUpGnZwySN3DCD2zeAzZL8IUCSDYD3AccBd0xZ9xn0rgORpHFhD5M0ckMLblVVwIuBg5JcBvwIuJP7rgN5Zvd1+QuAVwBHDas2SVode5ikcTDMLydQVVcDB0wz9C1g8xm2OY7eJ1pJGil7mKRRG6cvJ0iSJGkWBjdJkqRGGNwkSZIaMdRr3PTgXHTYRaMuQVJD7BnSus8jbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNWDDqAiRJa+GYzfvu3zq6OiQNhUfcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEaMTXBLsirJ8iQXJzklyRZTxl+f5M4km8+0D0kaBfuXpGEZm+AGrKyqJVX1BOAm4Mgp44cC5wAvGXplkjQ7+5ekoRin4NbvTGDbyQdJdgIWAm+j1wAlaVzZvyQNzNgFtyQbAPsCJ/ctPgQ4ATgD2CXJVjNse0SSZUmWrVixYvDFSlIf+5ekQRun4LZpkuXAtcBWwOl9Y4cCJ1TVPcDngYOm20FVHVtVS6tq6cTExMALlqSO/UvSUIxTcFtZVUuARUDorhFJ8kRgZ+D0JFfS+/Tq6QZJ48T+JWkoxim4AVBVdwCvA45KsoBekzumqhZ3t22AbZIsGmmhkjSF/UvSoI1dcAOoqvOBC+k1vUOAk6asclK3XJLGiv1L0iCNzR+Zr6qFUx4f0N399DTrvnEoRUnSHNi/JA3LWB5xkyRJ0gMZ3CRJkhphcJMkSWrE2FzjJklaA8fcOuoKJA2RR9wkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwk6R1xOK3nMbit5w26jIkDZDBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRYxPckqxKsjzJxUlOSbJFt/whST7YLb8oyTlJHjPqeiWpnz1M0jCMTXADVlbVkqp6AnATcGS3/GBgG2D3qnoi8GLglhHVKEkzsYdJGrgFoy5gBmcCu3f3twauqap7AKrqpyOrSpLmxh4maSDG6YgbAEk2APYFTu4WfRY4oDsF8b4ke8yy7RFJliVZtmLFimGUK0n3s6Y9zP4laS7GKbhtmmQ5cC2wFXA63PvpdBfgaOAe4BtJ9p1uB1V1bFUtraqlExMTQypbkoC17GH2L0lzMU7BbWVVLQEWAeG+60Ooqruq6itV9SbgXcCLRlSjJM3EHiZp4MYpuAFQVXcArwOOSrIgyZ5JtoHet7PoXTdy1ShrlKSZ2MMkDdJYfjmhqs5PciFwKLAC+FiSjbvh7wEfHllxkrQa9jBJgzI2wa2qFk55fEDfw/8ccjmS9KDYwyQNw9idKpUkSdL0DG6SJEmNGJtTpZKktXPlu18w6hIkDZhH3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRqSqRl3DQCRZAVzVPdwSuGGE5awp6x4u6x6u+a57UVVNzOP+RmZK/3owRv2z4PzO7/xrbk49bJ0Nbv2SLKuqpaOu48Gy7uGy7uFqte5xNurX1Pmd3/kHP7+nSiVJkhphcJMkSWrE+hLcjh11AWvIuofLuoer1brH2ahfU+d3fucfsPXiGjdJkqR1wfpyxE2SJKl5BjdJkqRGrLPBLclBSS5Jck+SpVPGjk5yeZIfJnnuqGqcSZL9utouT/KWUdczmyQfT3J9kov7lj0yyelJLuv++4hR1jhVku2TfDPJ97ufkT/vlo913QBJNknyvSQXdLX/7275Y5Kc3f3MnJhko1HXOlWSDZKcn+TU7vHY1zyOVtcfkmzcvZ6Xd6/v4iHPf3iSFUmWd7c/nuf5H9BzpownyQe7+i5MsueQ598nya19z/9v5nn+afvXlHUG9hrMcf6BvQYz9cAp6wzsPTDH+Qf6HqCq1skb8DhgF+BbwNK+5bsBFwAbA48BrgA2GHW9ffVt0NW0I7BRV+tuo65rlnqfBewJXNy37L3AW7r7bwHeM+o6p9S8NbBnd/9hwI+6n4uxrrurK8DC7v6GwNnA04DPAod0yz8K/Omoa52m9jcC/w6c2j0e+5rH7TaX/gD8GfDR7v4hwIlDnv9w4MMDfA0e0HOmjD8f+Er3XnkacPaQ599n8md8QM9/2v41rNdgjvMP7DWYqQdOWWeQ74G5zD/Q98A6e8Stqi6tqh9OM3QgcEJV3VVVPwEuB5463Opm9VTg8qr6cVX9CjiBXs1jqaq+A9w0ZfGBwCe7+58EXjTUolajqq6pqvO6+78ALgW2ZczrBqie27uHG3a3Ap4NfK5bPna1J9kOeAHwL93jMOY1j6m59If+n+PPAft2r/ew5h+oGXpOvwOBT3XvlbOALZJsPcT5B2qW/tVvYK/BHOcfmFl6YL+BvQfmOP9ArbPBbRbbAlf3Pf4pQ/yhm4Nxr28utqqqa7r71wJbjbKY2XSH0Peg96mpibq7U47LgeuB0+kdAbmlqu7uVhnHn5n3A28G7ukeP4rxr3kczaU/3LtO9/reSu/1Htb8AL/XnaL7XJLt52nuuRqHHvr07lTaV5I8flCTTOlf/YbyGswyPwzwNZjaA6tqxuc/gPfAXOaHAb4Hmg5uSb6e5OJpbmN7hGp9U73jxmP5O2eSLAQ+D7y+qm7rHxvnuqtqVVUtAbajdwRk1xGXNKsk+wPXV9W5o65FQ3EKsLiqdqf3weKTq1l/XXMevb85+STgQ8AXBzHJbP1rGFYz/0Bfg6k9MMkT5nP/8zD/QN8DTQe3qvqdqnrCNLcvzbLZz4D+9Ltdt2xcjHt9c3Hd5GH57r/Xj7ieB0iyIb2m85mq+kK3eOzr7ldVtwDfBJ5O71TIgm5o3H5m/h/ghUmupHdq7dnABxjvmsfVXPrDvet0r+/mwI3Dmr+qbqyqu7qH/wI8eZ7mnquR9tCqum3yVFpVfRnYMMmW8znHDP2r30Bfg9XNP4zXoNv3ZA/cb8rQIN8Dq51/0O+BpoPbGjoZOKT71sljgJ2B7424pn7nADun9427jehdWHnyiGt6sE4GDuvuHwbMFqSHrrvW4V+BS6vqn/qGxrpugCQTSbbo7m8KPIfeNSbfBF7arTZWtVfV0VW1XVUtpvfz/F9V9XLGuOYxNpf+0P9z/FJ6r/d8HT1e7fxTrqV6Ib2fz2E6GfjD7puVTwNu7bsEYuCSPHryeqokT6X3/9l5Cw2z9K9+A3sN5jL/IF+DGXrgD6asNrD3wFzmH/h7YD6+4TCON+DF9M7r3wVcB3y1b+yt9K4L+iHwvFHXOk3tz6f3TZ0rgLeOup7V1Ho8cA3w6+71/iN61xJ8A7gM+DrwyFHXOaXmZ9A7DXohsLy7PX/c6+5q3x04v6v9YuBvuuU70vsAcjnwH8DGo651hvr34b5vlTZR87jdpusPwNuBF3b3N+lez8u713fHIc//98Al9L5x+k1g13mef7qe8xrgNd14gI909V1E328VGNL8r+17/mcBe8/z/DP1r6G8BnOcf2CvwSw9cCjvgTnOP9D3gH/ySpIkqRHr46lSSZKkJhncJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMepCTHJXnHHNe9MsnvDLomSdNLcniS/x7Afl+e5Gvzvd9xluRbSf54jutWkscOuqb1kcFNa8VgImnUuj60MsntfbcPz+P+F3dBZMHksqr6TFX97nzNMcc65vyhUeuuBatfRZKksXdAVX191EVIg+YRNw1EkkckOTXJiiQ3d/e36xs/PMmPk/wiyU+SvLxb/tgk305ya5IbkpzYt83eSc7pxs5Jsvcs81+Z5E1JLkzyyyT/mmSrJF/p5vx6kkf0rf/CJJckuaU7HfC4vrE9kpzXbXcisMmUufZPsrzb9rtJdp+nl1HSPEvygSRXJ7ktyblJntk39tQky7qx65L8Uzf0ne6/t3RH854+9RRskscnOT3JTd22fzXD/Mcl+UiS07qecnaSnfrGd+3bzw+TvKxbfgTwcuDNXQ2nzLD/SvJnSS7r9v93SXbqetNtST6bZKO+9V+d5PJuvpOTbNM39pwkP+h67oeBTJnrVUku7Xr8V5MsmuM/g9ZGVXnztsY34Ergd6ZZ/ijg94DNgIcB/wF8sRt7KHAbsEv3eGvg8d3944G30vtQsQnwjG75I4GbgVfQO1J8aPf4UbPUdRawFbAtcD1wHrBHt9//Av62W/c3gV8CzwE2BN4MXA5s1N2uAt7Qjb0U+DXwjm7bPbp97wVsABzWzb3xbK+PN2/e5u822/sMOBz4777Hf9D1pwXAUcC1wCbd2JnAK7r7C4GndfcXAwUsmG6/XY+7ptvfJt3jvWao5zjgRuCpXQ2fAU7oxh4KXA28shvbA7gB2K1v23es5rUo4EvAw4HHA3cB3wB2BDYHvg8c1q377G7/ewIbAx8CvtONbQn8out5G3Y98G7gj7vxA7s++biu1rcB351Sx2NH/bOxLt484qaBqKobq+rzVXVHVf0CeCfwW32r3AM8IcmmVXVNVV3SLf81sAjYpqrurKrJT7QvAC6rqk9X1d1VdTzwA+CAWcr4UFVdV1U/A84Azq6q86vqTuAkek0R4GDgtKo6vap+DfwjsCmwN/A0ek3r/VX166r6HHBO3xxHAP+nqs6uqlVV9Ul6jfJpD/pFk7Q2vtgd9Z68vXq6larq37r+dHdVvY9eYNmlG/5Za00oAAAaXklEQVQ18NgkW1bV7VV11hzn3h+4tqre1/WtX1TV2bOsf1JVfa+q7qYX3Jb07efKqvpEV9/5wOeBg+ZYx6T3VtVtXV+9GPhaVf24qm4FvsJ9ve/lwMer6ryqugs4Gnh6ksXA84FLqupzXV98P72QO+k1wN9X1aXd83gXsMSjboNncNNAJNksyf9JclWS2+idatgiyQZV9Ut6Yek1wDXdKYNdu03fTO9w/Pe6U5ev6pZvQ+/IV7+r6B1Nm8l1ffdXTvN44XT7rqp76H3q3bYb+1lV7yNk37yTFgFH9f8PA9i+207S8Lyoqrbou31supWS/EV3eu/W7v26Ob2jSwB/RO8I/A+6yzH2n+Pc2wNXPIha+wPQHdzXixYBe03pJy8HHv0g9g1r3vtup3c0cLL3Xd03Vv2Pu1o/0FfnTfR692w9WfPA4KZBOYrep9i9qurhwLO65QGoqq9W1XPonSb9AfCxbvm1VfXqqtoG+BPgn9P7SvnP6TWKfjsAP5uHWu+37ySh14h/Ru/0x7bdsv55J10NvHPK/zA2644IShoj3fVsbwZeBjyiqrYAbuW+vnRZVR0K/AbwHuBzSR5K77TfbK6mdypybV0NfHtKP1lYVX/aja+ujgdrau97KL3TyJO9b/u+sfQ/7mr9kym1blpV353nGjWFwU3zYcMkm/TdFtC7xmMlvYt5Hwn87eTK3ZcEDuyaxF3A7fROnZLkoNz3JYab6TWqe4AvA7+Z5PeTLEhyMLAbcOo81P9Z4AVJ9k2yIb3QeRfwXXrXvNwNvC7JhkleQu/alEkfA16TZK/0PDTJC5I8bB7qkjS/Hkbv/bwCWJDkb+hdCwZAkj9IMtEddb+lW3xPt/49zBzOTgW2TvL6JBsneViSvdagvlPp9blXdP1mwyRPyX1flrpulhrWxPHAK5MsSbIxvdOdZ1fVlcBpwOOTvKTr6a/j/kf+PgocneTxAEk2T/JgT+lqDRjcNB++TC+kTd6OoXc9xKb0Lnw9C/jPvvUfAryR3qe9m+hd+zb5ifIpwNlJbgdOBv68uzbjRnrXfxxF71D+m4H9q+qGtS2+qn5I74LlD3X1HkDvVwv8qqp+BbyE3oXIN9E7xfuFvm2XAa8GPkwvaF7erStpuE7J/X+P20nTrPNVer3oR/ROEd7J/U//7Qdc0vWfDwCHVNXKqrqD3nW6/7c7NXi/a1i763ifQ693XAtcBvz2g30C3X5+FziEXn+8lt6Rv427Vf4V2K2r4YsPdv/TzPd14K/pXUd3DbBTNzddbz0IeDe9nrsz8H/7tj2pq+2E7nKYi4HnrW1NWr3c/9IdSZIkjSuPuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNWGf/yPyWW25ZixcvHnUZkobk3HPPvaGqJkZdx3ywf0nrn7n2sHU2uC1evJhly5aNugxJQ5Jk6l/WaJb9S1r/zLWHeapUkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhqxzv4et/n007eccb/H2737mSOqRJLWXn9Ps59JbfGImyRJUiMMbpIkSY0wuEmSJDXC4CZJktSIsQhuSVYlWZ7k4iSnJNmiW744ycok5ye5NMn3khw+4nIl6V72L0nDNBbBDVhZVUuq6gnATcCRfWNXVNUeVfU44BDg9UleOZIqJemB7F+ShmZcglu/M4Ftpxuoqh8DbwReN9SKJGlu7F+SBmqsgluSDYB9gZNnWe08YNcZtj8iybIky1asWDGIEiVpWvYvScMwLsFt0yTLgWuBrYDTZ1k3Mw1U1bFVtbSqlk5MTMx3jZI0HfuXpKEZl+C2sqqWAIvoNbYjZ1l3D+DSoVQlSatn/5I0NOMS3ACoqjvoXf9xVJIH/DmuJIuBfwQ+NNzKJGl29i9JwzB2f6u0qs5PciFwKHAGsFOS84FNgF8AH6yq40ZYoiRNy/4ladDGIrhV1cIpjw/oe7jpkMuRpDmzf0kaprE6VSpJkqSZGdwkSZIaYXCTJElqxFhc4zbutnv3M0ddgiTNG3ua1C6PuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1YsGoC2jB+w7ef9QlSOulo048ddQlrJPsadLgDap/ecRNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqRGrDW5JViVZnuTiJP+RZLNu+e3zXUySf0hySfffiSRnJzk/yTPney5J6wd7mKR1yVyOuK2sqiVV9QTgV8BrBljPEcDuVfUmYF/goqrao6rOGOCcktZt9jBJ64wHe6r0DOCx/QuSLEzyjSTnJbkoyYHd8rcneX3feu9M8ufp+Yfu0+9FSQ7uxk8GFgLnJvlL4L3Agd0n5YcmOa5vmzeszZOWtN6yh0lq2px/AW+SBcDzgP+cMnQn8OKqui3JlsBZXQP7OPAF4P1JHgIcAjwVeAmwBHgSsCVwTpLvVNULk9xeVUu6+a4DllbVa5M8Gdi2+8RMki1mqPEIep942WGHHeb61CStB8a9h9m/JM3FXI64bZpkObAM+B/gX6eMB3hXkguBrwPbAltV1ZXAjUn2AH4XOL+qbgSeARxfVauq6jrg28BTVlPDj4Edk3woyX7AbdOtVFXHVtXSqlo6MTExh6cmaT3QRA+zf0mai7kccVs5+QlyBi8HJoAnV9Wvk1wJbNKN/QtwOPBoep9e10hV3ZzkScBz6V2f8jLgVWu6P0nrFXuYpHXGfPw6kM2B67uG99vAor6xk4D96H0a/Wq37Azg4CQbJJkAngV8b7YJutMXD6mqzwNvA/ach7olCexhkhoyH39k/jPAKUkuoncq4geTA1X1qyTfBG6pqlXd4pOApwMXAAW8uaquXc0c2wKf6K4zATh6HuqWJLCHSWrIaoNbVS2cbXlV3UCviT1A16SeBhzUt10Bb+puM85VVccBx3X3L8BPqJLWgD1M0rpkYH85IcluwOXAN6rqskHNI0mDYA+TNI7m41TptKrq+8COg9q/JA2SPUzSOBpYcFuXHHXiqaMuQZLmjT1Napd/ZF6SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhoxNsEtyaoky5NckOS8JHtPGX99kjuTbD6qGiVpOvYvScMyNsENWFlVS6rqScDRwN9PGT8UOAd4ydArk6TZ2b8kDcU4Bbd+DwdunnyQZCdgIfA2eg1QksaV/UvSwCwYdQF9Nk2yHNgE2Bp4dt/YIcAJwBnALkm2qqrrpu4gyRHAEQA77LDD4CuWpB77l6ShGKcjbpOnGnYF9gM+lSTd2KHACVV1D/B54KDpdlBVx1bV0qpaOjExMZyqJcn+JWlIxumI272q6swkWwITSbYCdgZO7/rgRsBPgA+PsERJmpb9S9IgjdMRt3sl2RXYALiR3qfVY6pqcXfbBtgmyaKRFilJ07B/SRqkcTriNnmNCECAw6pqVZJDgOdPWfcketeNvGeYBUrSDOxfkoZibIJbVW0ww/Idp1n2xsFXJElzY/+SNCxjeapUkiRJD2RwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJasRIgluStya5JMmFSZYn2SvJt5IsnWbdp3ZjlyU5L8lpSZ44irolyf4laZQWDHvCJE8H9gf2rKq7kmwJbDTDulsBnwV+v6q+2y17BrATcNGQSpYkwP4lafSGHtyArYEbquougKq6ASDJdOu+FvjkZNPr1v/vYRQpSdOwf0kaqVGcKv0asH2SHyX55yS/Ncu6jwfOm+uOkxyRZFmSZStWrFjrQiVpCvuXpJEaenCrqtuBJwNHACuAE5McPpdtk5yd5NIkH5hh38dW1dKqWjoxMTFvNUsS2L8kjd4oTpVSVauAbwHfSnIRcNgMq14C7Al8qdturyQvpXeNiSQNnf1L0igN/Yhbkl2S7Ny3aAlw1QyrfwQ4PMnefcs2G1hxkjQL+5ekURvFEbeFwIeSbAHcDVxO77TD54DTkvy6W+/MqjooycHAe5JsC1wP3AC8fQR1S5L9S9JIDT24VdW5wN7TDO0zw/pnAbNdACxJQ2H/kjRq/uUESZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJasRQg1uS7ZJ8KcllSa5I8oEkGyXZJ8mtSZYnuTDJ15P8RrfN4UlWdGPfT/LqYdYsSZPsYZJGbWjBLUmALwBfrKqdgd8EFgLv7FY5o6qWVNXuwDnAkX2bn1hVS4B9gHcl2WpYdUsS2MMkjYdhHnF7NnBnVX0CoKpWAW8AXgVsNrlS1xwfBtw8dQdVdT1wBbBoGAVLUh97mKSRWzDEuR4PnNu/oKpuS/I/wGOBZyZZDjwK+CXwV1N3kGRHYEfg8ukmSHIEcATADjvsMK/FS1rvDbSH2b8kzcU4fTlh8jTD9sAngPf2jR3cNcTjgT+pqpum20FVHVtVS6tq6cTExBBKlqR7rVUPs39JmothBrfvA0/uX5Dk4cAOPPDT58nAs/oen9g1xL2q6qTBlilJ07KHSRq5YQa3bwCbJflDgCQbAO8DjgPumLLuM+hdByJJ48IeJmnkhhbcqqqAFwMHJbkM+BFwJ/ddB/LM7uvyFwCvAI4aVm2StDr2MEnjYJhfTqCqrgYOmGboW8DmM2xzHL1PtJI0UvYwSaM2Tl9OkCRJ0iwMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUCIObJElSIwxukiRJjTC4SZIkNcLgJkmS1AiDmyRJUiMMbpIkSY0wuEmSJDXC4CZJktQIg5skSVIjDG6SJEmNMLhJkiQ1wuAmSZLUiAWjLmAsHLP5asZvHU4dkvRgra5/TbuNPU1qlUfcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhoxNsEtyaoky5NcnOSUJFtMGX99kjuTrMFXqCRpcOxfkoZlbIIbsLKqllTVE4CbgCOnjB8KnAO8ZOiVSdLs7F+ShmKcglu/M4FtJx8k2QlYCLyNXgOUpHFl/5I0MGMX3JJsAOwLnNy3+BDgBOAMYJckW82w7RFJliVZtmLFisEXK0l97F+SBm2cgtumSZYD1wJbAaf3jR0KnFBV9wCfBw6abgdVdWxVLa2qpRMTEwMvWJI69i9JQzFOwW1lVS0BFgGhu0YkyROBnYHTk1xJ79OrpxskjRP7l6ShGKfgBkBV3QG8DjgqyQJ6Te6Yqlrc3bYBtkmyaKSFStIU9i9JgzZ2wQ2gqs4HLqTX9A4BTpqyykndckkaK/YvSYO0YNQFTKqqhVMeH9Dd/fQ0675xKEVJ0hzYvyQNy1gecZMkSdIDGdwkSZIaYXCTJElqxNhc4zZSx9w66gokac3Yv6T1ikfcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIaYXCTJElqhMFNkiSpEQY3SZKkRhjcJEmSGmFwkyRJaoTBTZIkqREGN0mSpEYsGHUBLVj8ltNmHLvy3S8YYiWStPZm62lT2eOk8eIRN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGjE1wS7IqyfIkFyc5JckW3fKHJPlgt/yiJOckecyo65WkfvYwScMwNsENWFlVS6rqCcBNwJHd8oOBbYDdq+qJwIuBW0ZUoyTNxB4maeDG9fe4nQns3t3fGrimqu4BqKqfjqwqSZobe5ikgRinI24AJNkA2Bc4uVv0WeCA7hTE+5LsMcu2RyRZlmTZihUrhlGuJN3PmvYw+5ekuRin4LZpkuXAtcBWwOlw76fTXYCjgXuAbyTZd7odVNWxVbW0qpZOTEwMqWxJAtayh9m/JM3FOAW3lVW1BFgEhPuuD6Gq7qqqr1TVm4B3AS8aUY2SNBN7mKSBG6fgBkBV3QG8DjgqyYIkeybZBnrfzqJ33chVo6xRkmZiD5M0SGP55YSqOj/JhcChwArgY0k27oa/B3x4ZMVJ0mrYwyQNytgEt6paOOXxAX0P/3PI5UjSg2IPkzQMY3eqVJIkSdMzuEmSJDXC4CZJktSIsbnGbZxd+e4XjLoESZo39jSpXR5xkyRJaoTBTZIkqREGN0mSpEYY3CRJkhphcJMkSWqEwU2SJKkRBjdJkqRGGNwkSZIakaoadQ0DkWQFcNU87W5L4IZ52tc4W1+eJ6w/z3V9eZ4Au1TVw0ZdxHyY5/41nXH7uRi3esCa5sqa5mYuNS2qqonV7Wid/csJc3nyc5VkWVUtna/9jav15XnC+vNc15fnCb3nOuoa5st89q/pjNvPxbjVA9Y0V9Y0N/NZk6dKJUmSGmFwkyRJaoTBbW6OHXUBQ7K+PE9Yf57r+vI8Yf16rmtr3F6rcasHrGmurGlu5q2mdfbLCZIkSesaj7hJkiQ1wuAmSZLUCINbnyT7JflhksuTvGWa8Y2TnNiNn51k8fCrXHtzeJ6HJ1mRZHl3++NR1Lm2knw8yfVJLp5hPEk+2L0OFybZc9g1zoc5PM99ktza9+/5N8OucT4k2T7JN5N8P8klSf58mnXWiX/TNbU2PSzJ0d3yHyZ57hBremP3b3phkm8kWdQ3tqrv5/bkIdY0Yw9McliSy7rbYUOs6f/tq+dHSW7pG5v312lt+ucAX6PV1fTyrpaLknw3yZP6xq7sli/PPP7qoLXpv6v7N59RVXnrXee3AXAFsCOwEXABsNuUdf4M+Gh3/xDgxFHXPaDneTjw4VHXOg/P9VnAnsDFM4w/H/gKEOBpwNmjrnlAz3Mf4NRR1zkPz3NrYM/u/sOAH03zs7tO/Juu4euzxj0M2K1bf2PgMd1+NhhSTb8NbNbd/9P+vgrcPqLXadoeCDwS+HH330d09x8xjJqmrP+/gI8P+HVao/45qNdojjXtPTkX8Lz+9z9wJbDlCF6nafvvg/037795xO0+TwUur6ofV9WvgBOAA6escyDwye7+54B9k2SINc6HuTzPdUJVfQe4aZZVDgQ+VT1nAVsk2Xo41c2fOTzPdUJVXVNV53X3fwFcCmw7ZbV14t90Da1NDzsQOKGq7qqqnwCXd/sbeE1V9c2quqN7eBaw3TzMu1Y1zeK5wOlVdVNV3QycDuw3gpoOBY6fh3lntBb9c1Cv0WprqqrvdnPCcH6W1qb/rvHPocHtPtsCV/c9/ikP/J/CvetU1d3ArcCjhlLd/JnL8wT4ve6Q8+eSbD+c0oZurq/FuuDpSS5I8pXk/2/v7kGkusIwjv8fyEZBJQgSUoSEXRAEISCKBLMQEkTQYmNhsYUKZiFaWKTfRtKYKl2KgCmDRRKVDQh+sKbzM8EPEhOJdosoiERsRMJrcc7GyzDXWWfm3rt39/nBMDNn7g7vfc/cd8/cew6jjU0HM6h8iW8TcLnjpeXUp50GqWFV5e1133eKdBZn3kpJ1yRdkrR7CPG8TkzdamDjecqXkkeB2UJzFXnqpSzmxXIMdn6WAjgr6TdJX9QcS7f623eeluxPXtlAfgGOR8QzSQdJ39A/bTgm69/vpN/AeyppF3AKWN9wTH2TtBr4GfgyIp40HY8Nh6S9wBbg40Lz+xExJ2kMmJV0KyLu1hDOYq6Bk8BPEfFfoa2pPC1Kkj4hDdzGC83jOUdvA+ck/ZXPllVt6PXXZ9xemgOKZ5bezW1dt5H0BvAW8KiW6Ian535GxKOIeJafHgM21xRb3RbS560XEU8i4ml+fBoYkbSu4bD6ImmENGj7ISJOdNlkWfRpiUFqWFV5W9D7StoOTAMThdpDRMzl+3vAr6SzrJXH9Ioa2Gieskk6LpNWlKdeymJu9BiU9AGpzz6LiP//Pxdy9BA4yXCmAvT0ivrbd548cHvpKrBe0qikN0kHR+fqnBlgfoXMHmA28izDFum5nx1zgiZIc4mWohlgf14d9SHwb0TcbzqoYZP0zvxcTElbScd9275wkPfhe+B2RHxTstmy6NMSg9SwGWBSadXpKOmMwJU6YpK0CfiONGh7WGhfK2lFfrwO+Aj4s6aYymrgGWBHjm0tsCO3VR5TjmsDacL/xUJbVXnqpexYqypHPUl6DzgB7IuIO4X2VZLWzD/OMXVdBVpBTGX1d0F93tUgqymW2o20SuYOaaXHdG77ilRQAFYCP5Im7l4BxpqOuaL9PAr8QVrlcgHY0HTMfe7nceA+8Jw0f2AKOAQcyq8L+Dbn4RawpemYK9rPw4X+vARsazrmPvdznDRP5SZwPd92LcU+HSBHfdcw0hmvu8DfwM4aYzoPPCj06Uxu35b78Ea+n6oxptIaCHye8/cPcKCumPLzI8DXHX9XSZ4GqZ8V5qhXTMeAx4XP0rXcPpbzcyP363SNMZXW3259vpCbf/LKzMzMrCV8qdTMzMysJTxwMzMzM2sJD9zMzMzMWsIDNzMzM7OW8MDNzMzMrCU8cDMzMzNrCQ/czMzMzFriBSSq0vyl6PbyAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 720x720 with 4 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "Features_importance_model = pd.Series(LRfitted.coef_, index= X_train.columns)\n",
        "Features_importance_Ridge = pd.Series(ridge.coef_, index= X_train.columns)\n",
        "Features_importance_Lasso = pd.Series(lasso.coef_, index= X_train.columns)\n",
        "Features_importance_Elasticnet =pd.Series(elasticnet.coef_, index= X_train.columns)\n",
        "\n",
        "fig, axs = plt.subplots(2, 2, figsize=(10,10))\n",
        "Features_importance_model.plot(ax=axs[0,0], kind='barh', title ='LR model' )\n",
        "\n",
        "Features_importance_Ridge.plot(ax=axs[0,1], kind='barh', title ='Ridge model')\n",
        "Features_importance_Lasso.plot(ax=axs[1,0], kind='barh', title ='Lasso model')\n",
        "Features_importance_Elasticnet.plot(ax=axs[1,1], kind='barh', title ='Elastic net model')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BRsM84cczmJa"
      },
      "source": [
        "As discussed earlier, LASSO model not only helps in making the model less complex but, it can also be used to find important features. The most important features for the outcome variable W are Playoff, RD, RA and RS as discovered by LASSO and Elasticnet models"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sjo3HrkvzmJa"
      },
      "source": [
        "# 9. Hyperparameter Selection via Cross-Validation\n",
        "\n",
        "The output of regularization techniques depends on the value given to the  paramter alpha. The good choice of value of alpha can be discovered by cross-validation technique. \n",
        "\n",
        "The following codes find best alpha for Ridge, LASSO and Elasticnet using 10 cross validation and alpha value ranging from 0.2 to 1.0. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7kBIgguOzmJa"
      },
      "source": [
        "## 9.1 Tuning Hyperparameter parameter of Ridge regression using Cross-validation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8toPvZu8zmJa",
        "outputId": "3c400a1c-43a0-49e4-c129-a528c623103e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The best alpha value discovered: 1.0\n"
          ]
        }
      ],
      "source": [
        "# Using RidgeCV function, create its object. RidgeCV is controlled by parameters cv and\n",
        "# alphas. Paramter cv defines number of cross validation cycles whereas, range of \n",
        "# different alpha values can be listed using alphas parameter. \n",
        "\n",
        "ridge_cv = linear_model.RidgeCV(cv =10, alphas =[0.2, 0.4,0.5,0.6,0.8,1.0])\n",
        "\n",
        "# Fitting ridge on data set containing all features\n",
        "ridge_cv.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "Ridge_cv_pred_train = ridge_cv.predict(X_train)\n",
        "Ridge_cv_pred_test= ridge_cv.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "\n",
        "Ridge_cv_model = \"RidgeCV on All features\"\n",
        "Ridge_cv_values = [Ridge_cv_model,\n",
        "        round(metrics.mean_absolute_error(Y_train, Ridge_cv_pred_train),2),\n",
        "        round(metrics.mean_absolute_error(Y_test, Ridge_cv_pred_test),2),\n",
        "        np.absolute(ridge_cv.coef_).sum() +np.absolute(ridge_cv.intercept_ ),\n",
        "         round(metrics.r2_score(Y_test,Ridge_cv_pred_test),4)*100           ]\n",
        "\n",
        "print(\"The best alpha value discovered:\" ,ridge_cv.alpha_)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8gpuLTeqzmJa"
      },
      "source": [
        "## 9.2  Tuning Hyperparameter parameter of Lasso regression using Cross-validation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hV-1IbAkzmJb",
        "outputId": "8ecc4f06-34e6-49d2-d169-ecaeb831c0c7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The best alpha value discovered: 0.2\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:475: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 3.7025394530592166, tolerance: 3.6041718518518517\n",
            "  positive)\n"
          ]
        }
      ],
      "source": [
        "# Using LassoCV function, create its object. LassoCV is controlled by parameters cv and\n",
        "# alphas. Paramter cv defines number of cross validation cycles whereas, range of \n",
        "# different alpha values can be listed using alphas parameter. \n",
        "\n",
        "lasso_cv = linear_model.LassoCV(cv =10, alphas =[0.2, 0.4,0.5,0.6,0.8,1.0])\n",
        "\n",
        "# Fitting Lasso on data set containing all features\n",
        "lasso_cv.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "Lasso_cv_pred_train = lasso_cv.predict(X_train)\n",
        "Lasso_cv_pred_test= lasso_cv.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "\n",
        "Lasso_cv_model = \"LassoCV on All features\"\n",
        "Lasso_cv_values = [Lasso_cv_model,\n",
        "        round(metrics.mean_absolute_error(Y_train, Lasso_cv_pred_train ),2),\n",
        "        round(metrics.mean_absolute_error(Y_test, Lasso_cv_pred_test),2),\n",
        "        np.absolute(lasso_cv .coef_).sum() +np.absolute(lasso_cv .intercept_ ),\n",
        "         round(metrics.r2_score(Y_test,Lasso_cv_pred_test),4)*100           ]\n",
        "\n",
        "print(\"The best alpha value discovered:\" ,lasso_cv.alpha_)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fA0bjrd5zmJb"
      },
      "source": [
        "## 9.3  Tuning Hyperparameter parameter of Elastic Net regression using Cross-validation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_INhyRuHzmJb",
        "outputId": "06dcd96c-0e89-4fe8-c5ec-5fef3e78a5a9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The best alpha value discovered: 0.2\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.847479465553079, tolerance: 3.2065629629629613\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 5.441754216434219, tolerance: 3.3032740740740745\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 5.13031660622255, tolerance: 3.3372205761316867\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 5.02641342611787, tolerance: 3.2948666666666666\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.825070821885674, tolerance: 3.212266666666667\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.973728348577424, tolerance: 3.3039292181069966\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.513637758877849, tolerance: 3.1444666666666667\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.921344221041181, tolerance: 3.2076518518518524\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 4.618709951369965, tolerance: 3.1432740740740743\n",
            "  tol, rng, random, positive)\n",
            "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/linear_model/coordinate_descent.py:471: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 5.184608235023006, tolerance: 3.266484773662552\n",
            "  tol, rng, random, positive)\n"
          ]
        }
      ],
      "source": [
        "# Using ElasticNetCV function, create its object. ElasticNetCV is controlled by parameters\n",
        "# cv and alphas. Paramter cv defines number of cross validation cycles whereas, range of \n",
        "# different alpha values can be listed using alphas parameter. \n",
        "\n",
        "elasticnet_cv = linear_model.ElasticNetCV(cv =10, alphas =[0.2, 0.4,0.5,0.6,0.8,1.0])\n",
        "\n",
        "# Fitting Elastic net CV on data set containing all features\n",
        "elasticnet_cv.fit(X_train, Y_train)\n",
        "\n",
        "# Getting prediction on train and test sets\n",
        "Elasticnet_cv_pred_train = elasticnet_cv.predict(X_train)\n",
        "Elasticnet_cv_pred_test= elasticnet_cv.predict(X_test)\n",
        "\n",
        "# preparing summary of training, test errors, sum of absolute Weights(SAW) and rsquare\n",
        "\n",
        "Elasticnet_cv_model = \"Elasticnet_cv on All features\"\n",
        "Elasticnet_cv_values = [Elasticnet_cv_model,\n",
        "        round(metrics.mean_absolute_error(Y_train, Elasticnet_cv_pred_train ),2),\n",
        "        round(metrics.mean_absolute_error(Y_test_five, Elasticnet_cv_pred_test),2),\n",
        "        np.absolute(elasticnet_cv .coef_).sum() +np.absolute(elasticnet_cv.intercept_ ),\n",
        "         round(metrics.r2_score(Y_test,Elasticnet_cv_pred_test),4)*100           ]\n",
        "\n",
        "print(\"The best alpha value discovered:\" ,elasticnet_cv.alpha_)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s67NMJo1zmJb"
      },
      "source": [
        "# Summary\n",
        "\n",
        "This script demonstrates  fitting Multiple linear regression models to a given dataset. Regression concepts such as  model overfitting, collinearlity, model complexity are discussed.A practical understanding on  regularization methods, called ridge, lasso regression and Eleastic net are also presented."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}