09_Naive_Bayes.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "09 Naive Bayes.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyM3U2Y4vfFjpAOYAdgH8sxa",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sandipanpaul21/ML-Code-in-Python/blob/master/09_Naive_Bayes.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cdIDWT9HLD00",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Naive Bayes \n",
        "\n",
        "# Introduction\n",
        "# Naive Bayes is the most straightforward and fast classification algorithm, \n",
        "# It is suitable for a large chunk of data. \n",
        "# It uses Bayes theorem of probability for prediction of unknown class.\n",
        "# Works for Multi Class Classification also (Target Columns >= 2 and more)\n",
        "# Logistic Regression only works for Binary Classification (Target Columns = 2)\n",
        "\n",
        "# Assumption\n",
        "# It assumes that the effect of a particular feature in a class is independent of other features. \n",
        "# For example, a loan applicant is desirable or not depending on his/her income, previous loan \n",
        "# and transaction history, age, and location. \n",
        "# Even if these features are interdependent, these features are still considered independently. \n",
        "# This assumption simplifies computation, and that's why it is considered as naive. \n",
        "# This assumption is called class conditional independence.\n",
        "\n",
        "# Zero Probability Problem\n",
        "# Suppose there is no tuple for a risky loan in the dataset, \n",
        "# in this scenario, the posterior probability will be zero, \n",
        "# and the model is unable to make a prediction. \n",
        "# This problem is known as Zero Probability because the occurrence of the particular class is zero.\n",
        "# The solution for such an issue is the Laplacian correction or Laplace Transformation. \n",
        "# Laplacian correction is one of the smoothing techniques. \n",
        "# Here, you can assume that the dataset is large enough \n",
        "# that adding one row of each class will not make a difference in the estimated probability. \n",
        "# This will overcome the issue of probability values to zero.\n",
        "# For Example: Suppose that for the class loan risky, there are 1000 training tuples in the database. \n",
        "# In this database, \n",
        "# income column has 0 tuples for low income, \n",
        "# 990 tuples for medium income, \n",
        "# and 10 tuples for high income. \n",
        "# The probabilities of these events, \n",
        "# without the Laplacian correction, are 0, 0.990 (from 990/1000), and 0.010 (from 10/1000)\n",
        "# Now, apply Laplacian correction on the given dataset. \n",
        "# Let's add 1 more tuple for each income-value pair. The probabilities of these events:\n",
        "# P(Low Income) = 1/1003 = 0.001, P(Medium Income) = 991/1003 = 0.988 & P(High Income) = 11/1003 = 0.011\n",
        "\n",
        "# Advantages\n",
        "# It is not only a simple approach but also a fast and accurate method for prediction.\n",
        "# It can be used with multiple class prediction problems.\n",
        "# It also performs well in the case of text analytics problems.\n",
        "# When the assumption of independence holds, a Naive Bayes classifier performs better \n",
        "# compared to other models like logistic regression.\n",
        "\n",
        "# Disadvantages\n",
        "# The assumption of independent features. \n",
        "# In practice, it is almost impossible that model will get a set of predictors which are entirely independent.\n",
        "# If there is no training tuple of a particular class, this causes zero posterior probability. \n",
        "# In this case, the model is unable to make predictions. \n",
        "# This problem is known as Zero Probability/Frequency Problem."
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aaJy5SriCZDL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "c70acea9-1687-472f-d6c8-f4096582c19e"
      },
      "source": [
        "#  Libraries\n",
        "\n",
        "from sklearn import datasets\n",
        "import pandas as pd\n",
        "import statsmodels.api as sm\n",
        "# Import Gaussian Naive Bayes model\n",
        "from sklearn.naive_bayes import GaussianNB\n",
        "from sklearn import metrics\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import accuracy_score,recall_score,precision_score,f1_score\n",
        "from sklearn.metrics import confusion_matrix, classification_report, roc_auc_score\n",
        "import warnings\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "import numpy as np\n",
        "from sklearn.metrics import classification_report"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
            "  import pandas.util.testing as tm\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "A-2LJd1SCiKS",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "4086f6da-ed75-4a1d-c637-d4fa1bba57fb"
      },
      "source": [
        "# IRIS Dataset\n",
        "iriss = datasets.load_iris()\n",
        "iris = pd.DataFrame(iriss.data)\n",
        "iris.columns = iriss.feature_names\n",
        "iris['species'] = iriss.target\n",
        "iris.columns = iris.columns.str.replace(\" \",\"\")\n",
        "iris.columns = iris.columns.str.replace(\"(\",\"\")\n",
        "iris.columns = iris.columns.str.replace(\")\",\"\")\n",
        "iris.head()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sepallengthcm</th>\n",
              "      <th>sepalwidthcm</th>\n",
              "      <th>petallengthcm</th>\n",
              "      <th>petalwidthcm</th>\n",
              "      <th>species</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>5.1</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4.9</td>\n",
              "      <td>3.0</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4.7</td>\n",
              "      <td>3.2</td>\n",
              "      <td>1.3</td>\n",
              "      <td>0.2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.6</td>\n",
              "      <td>3.1</td>\n",
              "      <td>1.5</td>\n",
              "      <td>0.2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5.0</td>\n",
              "      <td>3.6</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   sepallengthcm  sepalwidthcm  petallengthcm  petalwidthcm  species\n",
              "0            5.1           3.5            1.4           0.2        0\n",
              "1            4.9           3.0            1.4           0.2        0\n",
              "2            4.7           3.2            1.3           0.2        0\n",
              "3            4.6           3.1            1.5           0.2        0\n",
              "4            5.0           3.6            1.4           0.2        0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BSad3e-YCsDd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 89
        },
        "outputId": "c4500905-f6d4-4124-ba94-ab9c34fda109"
      },
      "source": [
        "# Target Column Distribution\n",
        "iris['species'].value_counts()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2    50\n",
              "1    50\n",
              "0    50\n",
              "Name: species, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5GqxQzKCCypn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        },
        "outputId": "ea4ace37-806f-49fe-bfcc-6751d8a6eb9a"
      },
      "source": [
        "# Distribution (mean) of Independent Columns respect to Dependent Column\n",
        "iris.groupby('species').mean().round(2)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sepallengthcm</th>\n",
              "      <th>sepalwidthcm</th>\n",
              "      <th>petallengthcm</th>\n",
              "      <th>petalwidthcm</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>species</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>5.01</td>\n",
              "      <td>3.43</td>\n",
              "      <td>1.46</td>\n",
              "      <td>0.25</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>5.94</td>\n",
              "      <td>2.77</td>\n",
              "      <td>4.26</td>\n",
              "      <td>1.33</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>6.59</td>\n",
              "      <td>2.97</td>\n",
              "      <td>5.55</td>\n",
              "      <td>2.03</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "         sepallengthcm  sepalwidthcm  petallengthcm  petalwidthcm\n",
              "species                                                          \n",
              "0                 5.01          3.43           1.46          0.25\n",
              "1                 5.94          2.77           4.26          1.33\n",
              "2                 6.59          2.97           5.55          2.03"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WsRAmcb1C3aE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "def39857-bf7f-4eab-e392-805c8fcde470"
      },
      "source": [
        "# Independent Variables\n",
        "Independent_Variable_Base_Set = iris[iris.columns[0:4]]\n",
        "Independent_Variable_Base_Set.head()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sepallengthcm</th>\n",
              "      <th>sepalwidthcm</th>\n",
              "      <th>petallengthcm</th>\n",
              "      <th>petalwidthcm</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>5.1</td>\n",
              "      <td>3.5</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4.9</td>\n",
              "      <td>3.0</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>4.7</td>\n",
              "      <td>3.2</td>\n",
              "      <td>1.3</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4.6</td>\n",
              "      <td>3.1</td>\n",
              "      <td>1.5</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5.0</td>\n",
              "      <td>3.6</td>\n",
              "      <td>1.4</td>\n",
              "      <td>0.2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   sepallengthcm  sepalwidthcm  petallengthcm  petalwidthcm\n",
              "0            5.1           3.5            1.4           0.2\n",
              "1            4.9           3.0            1.4           0.2\n",
              "2            4.7           3.2            1.3           0.2\n",
              "3            4.6           3.1            1.5           0.2\n",
              "4            5.0           3.6            1.4           0.2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X25sAaAwC_mr",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "6365e889-3c6e-478c-b529-6a2032e64ce6"
      },
      "source": [
        "# Dependent Variable\n",
        "Dependent_Variable = iris[iris.columns[-1:iris.columns.size]]\n",
        "Dependent_Variable.head()"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>species</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   species\n",
              "0        0\n",
              "1        0\n",
              "2        0\n",
              "3        0\n",
              "4        0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9vnglX0dDDW4",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 359
        },
        "outputId": "704e4116-8fd2-4cd6-f0c3-87ea88d23d4a"
      },
      "source": [
        "# Split the Dataset\n",
        "\n",
        "# Model 1\n",
        "# Lets start with selecting one variable\n",
        "Independent_Variable_Set_v1 = iris[iris.columns[0:1]]\n",
        "X_train, X_test, y_train, y_test = train_test_split(Independent_Variable_Set_v1,Dependent_Variable,test_size = 0.3,random_state = 21)\n",
        "\n",
        "# Create a Gaussian Classifier\n",
        "model = GaussianNB()\n",
        "\n",
        "# Train the model using the training sets\n",
        "result = model.fit(X_train,y_train)\n",
        "\n",
        "# Model Prediction\n",
        "print(\"Sample Prediction of Model 1\")\n",
        "pred = result.predict(X_test)\n",
        "model_prediction = pd.DataFrame(pred.round(2),columns = ['Prediction'])\n",
        "print(model_prediction['Prediction'].head())\n",
        "\n",
        "# Test Set Target Column Distribution\n",
        "print(\"\\nTest Set Distribution\")\n",
        "print(y_test['species'].value_counts())\n",
        "\n",
        "# Predicted Set Target Column Distribution\n",
        "print(\"\\nPredicted Set Distribution\")\n",
        "print(model_prediction['Prediction'].value_counts())"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sample Prediction of Model 1\n",
            "0    1\n",
            "1    0\n",
            "2    0\n",
            "3    0\n",
            "4    1\n",
            "Name: Prediction, dtype: int64\n",
            "\n",
            "Test Set Distribution\n",
            "1    16\n",
            "2    15\n",
            "0    14\n",
            "Name: species, dtype: int64\n",
            "\n",
            "Predicted Set Distribution\n",
            "1    19\n",
            "0    14\n",
            "2    12\n",
            "Name: Prediction, dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L9WM2W7fDmfj",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "outputId": "c305d816-08bd-470f-f9af-f12d3cdbe12e"
      },
      "source": [
        "# Model Performance\n",
        "\n",
        "# Take the average of the f1-score for each class: that's the avg / total result above. \n",
        "# It's also called macro averaging.\n",
        "\n",
        "# Compute the f1-score using the global count of true positives / false negatives, etc. \n",
        "# (Sum the number of true positives / false negatives for each class). Aka micro averaging.\n",
        "\n",
        "y_pred = model_prediction[['Prediction']]\n",
        "model_1_accuracy = accuracy_score(y_test,y_pred).round(2)\n",
        "print(\"Model 1 Performance\")\n",
        "print(\"\\nModel 1, Accuracy :\",model_1_accuracy)\n",
        "model_1_precision = precision_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 1, Precision :\",model_1_precision)\n",
        "model_1_recall = recall_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 1, Recall :\",model_1_recall)\n",
        "model_1_fscore = f1_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 1, F1 Score :\",model_1_fscore)\n",
        "print(\"\\nConfusion Matrix, Model 1\")\n",
        "model_1_cm = confusion_matrix(y_test,y_pred)\n",
        "print(model_1_cm)\n",
        "print(\"\\nClassification Report, Model 1\")\n",
        "model_1_cr = classification_report(y_test, y_pred)\n",
        "print(model_1_cr)\n",
        "\n",
        "print(\"Inference : Good Fit, can we make it better ?\")"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model 1 Performance\n",
            "\n",
            "Model 1, Accuracy : 0.67\n",
            "Model 1, Precision : 0.67\n",
            "Model 1, Recall : 0.67\n",
            "Model 1, F1 Score : 0.67\n",
            "\n",
            "Confusion Matrix, Model 1\n",
            "[[13  1  0]\n",
            " [ 1 10  5]\n",
            " [ 0  8  7]]\n",
            "\n",
            "Classification Report, Model 1\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.93      0.93      0.93        14\n",
            "           1       0.53      0.62      0.57        16\n",
            "           2       0.58      0.47      0.52        15\n",
            "\n",
            "    accuracy                           0.67        45\n",
            "   macro avg       0.68      0.67      0.67        45\n",
            "weighted avg       0.67      0.67      0.66        45\n",
            "\n",
            "Inference : Good Fit, can we make it better ?\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6KhsgLWzREB3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 359
        },
        "outputId": "d11e443c-a823-42f8-c30f-27818615ee57"
      },
      "source": [
        "# Model 2\n",
        "# Lets build model with all variables\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(Independent_Variable_Base_Set,Dependent_Variable,test_size = 0.3,random_state = 21)\n",
        "# Train the model using the training sets\n",
        "result = model.fit(X_train,y_train)\n",
        "\n",
        "# Model Prediction\n",
        "print(\"Sample Prediction of Model 2\")\n",
        "pred = result.predict(X_test)\n",
        "model_prediction = pd.DataFrame(pred.round(2),columns = ['Prediction'])\n",
        "print(model_prediction['Prediction'].head())\n",
        "\n",
        "# Test Set Target Column Distribution\n",
        "print(\"\\nTest Set Distribution\")\n",
        "print(y_test['species'].value_counts())\n",
        "\n",
        "# Predicted Set Target Column Distribution\n",
        "print(\"\\nPredicted Set Distribution\")\n",
        "print(model_prediction['Prediction'].value_counts())"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Sample Prediction of Model 2\n",
            "0    1\n",
            "1    0\n",
            "2    0\n",
            "3    0\n",
            "4    1\n",
            "Name: Prediction, dtype: int64\n",
            "\n",
            "Test Set Distribution\n",
            "1    16\n",
            "2    15\n",
            "0    14\n",
            "Name: species, dtype: int64\n",
            "\n",
            "Predicted Set Distribution\n",
            "2    17\n",
            "1    14\n",
            "0    14\n",
            "Name: Prediction, dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CK1T-iljE1dK",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "outputId": "4fe84110-683a-4fa4-ac8a-308b35264403"
      },
      "source": [
        "# Model Performance\n",
        "\n",
        "# Take the average of the f1-score for each class: that's the avg / total result above. \n",
        "# It's also called macro averaging.\n",
        "\n",
        "# Compute the f1-score using the global count of true positives / false negatives, etc. \n",
        "# (Sum the number of true positives / false negatives for each class). Aka micro averaging.\n",
        "\n",
        "y_pred = model_prediction[['Prediction']]\n",
        "model_2_accuracy = accuracy_score(y_test,y_pred).round(2)\n",
        "print(\"Model 2 Performance\")\n",
        "print(\"\\nModel 2, Accuracy :\",model_2_accuracy)\n",
        "model_2_precision = precision_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 2, Precision :\",model_2_precision)\n",
        "model_2_recall = recall_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 2, Recall :\",model_2_recall)\n",
        "model_2_fscore = f1_score(y_test,y_pred, average=\"micro\").round(2)\n",
        "print(\"Model 2, F1 Score :\",model_2_fscore)\n",
        "print(\"\\nConfusion Matrix, Model 2\")\n",
        "model_2_cm = confusion_matrix(y_test,y_pred)\n",
        "print(model_2_cm)\n",
        "print(\"\\nClassification Report, Model 2\")\n",
        "model_2_cr = classification_report(y_test, y_pred)\n",
        "print(model_2_cr)\n",
        "\n",
        "print(\"Inference : Model 2 is OVERFIT, so choose Model 1\")"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model 2 Performance\n",
            "\n",
            "Model 2, Accuracy : 0.96\n",
            "Model 2, Precision : 0.96\n",
            "Model 2, Recall : 0.96\n",
            "Model 2, F1 Score : 0.96\n",
            "\n",
            "Confusion Matrix, Model 2\n",
            "[[14  0  0]\n",
            " [ 0 14  2]\n",
            " [ 0  0 15]]\n",
            "\n",
            "Classification Report, Model 2\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        14\n",
            "           1       1.00      0.88      0.93        16\n",
            "           2       0.88      1.00      0.94        15\n",
            "\n",
            "    accuracy                           0.96        45\n",
            "   macro avg       0.96      0.96      0.96        45\n",
            "weighted avg       0.96      0.96      0.96        45\n",
            "\n",
            "Inference : Model 2 is OVERFIT, so choose Model 1\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}