Probabilistic_Graphical_Models_in_one_notebook.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Probabilistic Graphical Models in one notebook",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y3aMFbIMafwg"
      },
      "source": [
        "# Probabilistic Graphical Models in one notebook\n",
        "> A quick and to the point walk through \n",
        "\n",
        "\n",
        "## Theoretical\n",
        "[Wiki](https://en.wikipedia.org/wiki/Graphical_model#:~:text=A%20graphical%20model%20or%20probabilistic,Bayesian%20statistics%E2%80%94and%20machine%20learning.)\n",
        "\n",
        "[Blog post](https://medium.com/@jonathan_hui/machine-learning-graphical-model-b68b0c27a749)\n",
        "\n",
        "Bert Huang's videos are amazing \n",
        " - [Undirected Graphical Models](https://www.youtube.com/watch?v=iBQkZdPHlCs)\n",
        " - [Probabilistic Graphical Models and Bayesian Networks](https://www.youtube.com/watch?v=zCWRTKnOYYg)\n",
        "\n",
        "\n",
        "## Practical \n",
        "[Video - Probabilistic Graphical Models in Python](https://www.youtube.com/watch?v=DEHqIxX1Kq4), code starts @ 16:00 | [Github Repo](https://github.com/AileenNielsen/PyGotham_2016_Probabilistic_Graphical_Models)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0q35dAGraf_x",
        "outputId": "6762fede-4aef-4b1d-ae44-4abb15e01884",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        }
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.metrics import confusion_matrix, accuracy_score, auc, roc_curve, roc_auc_score\n",
        "\n",
        "from IPython.core.interactiveshell import InteractiveShell # multiline output, so `x` does `print(x)`\n",
        "InteractiveShell.ast_node_interactivity = \"all\""
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
            "  import pandas.util.testing as tm\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ddmajdSvagQe"
      },
      "source": [
        "[pomegranate Docs](https://pomegranate.readthedocs.io/en/latest/faq.html)\n",
        "\n",
        "[pgmpy Docs](https://github.com/pgmpy/pgmpy#installation) | [Notebooks](https://github.com/pgmpy/pgmpy_notebook/tree/master/notebooks)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3fgSB8zIagW8"
      },
      "source": [
        "# ! pip install pgmpy"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Asr_qcjqagaP",
        "outputId": "90f573e9-37ed-4dbd-e7a0-dfd85a9b9b73",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        }
      },
      "source": [
        "# sample code https://github.com/IBM/predict-fraud-using-auto-ai/blob/master/notebook/Fraud_Detect.ipynb\n",
        "df = pd.read_csv(\"https://raw.githubusercontent.com/IBM/predict-fraud-using-auto-ai/master/data/fraud_dataset.csv\")\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Gender</th>\n",
              "      <th>Married</th>\n",
              "      <th>Dependents</th>\n",
              "      <th>Education</th>\n",
              "      <th>Self_Employed</th>\n",
              "      <th>ApplicantIncome</th>\n",
              "      <th>CoapplicantIncome</th>\n",
              "      <th>LoanAmount</th>\n",
              "      <th>Loan_Term</th>\n",
              "      <th>Credit_History_Available</th>\n",
              "      <th>Housing</th>\n",
              "      <th>Locality</th>\n",
              "      <th>Fraud_Risk</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>5849</td>\n",
              "      <td>0</td>\n",
              "      <td>146</td>\n",
              "      <td>360</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>4583</td>\n",
              "      <td>1508</td>\n",
              "      <td>128</td>\n",
              "      <td>360</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3000</td>\n",
              "      <td>0</td>\n",
              "      <td>66</td>\n",
              "      <td>360</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2583</td>\n",
              "      <td>2358</td>\n",
              "      <td>120</td>\n",
              "      <td>360</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>6000</td>\n",
              "      <td>0</td>\n",
              "      <td>141</td>\n",
              "      <td>360</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Gender  Married  Dependents  ...  Housing  Locality  Fraud_Risk\n",
              "0       1        0           0  ...        1         1           0\n",
              "1       1        1           1  ...        1         3           1\n",
              "2       1        1           0  ...        1         1           1\n",
              "3       1        1           0  ...        1         1           1\n",
              "4       1        0           0  ...        1         1           0\n",
              "\n",
              "[5 rows x 13 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FSM67nMJ9dQm"
      },
      "source": [
        "count_fraud = len(df[df['Fraud_Risk']==0])\n",
        "count_non_fraud = len(df[df['Fraud_Risk']==1])\n",
        "pct_of_non_fraud = count_non_fraud/(count_non_fraud +count_fraud)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WBQkk9hXkWh9",
        "outputId": "d998a85a-8ef4-44ab-eca9-2dd00b63aaff",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 372
        }
      },
      "source": [
        "print(list(df.columns))\n",
        "\n",
        "print(\"percentage of non Fraud Risk is\", round(pct_of_non_fraud*100,2))\n",
        "print(\"percentage of Fraud Risk is\", round((1-pct_of_non_fraud)*100,2))\n",
        "\n",
        "sns.countplot(x='Fraud_Risk',data=df, palette='hls')\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'ApplicantIncome', 'CoapplicantIncome', 'LoanAmount', 'Loan_Term', 'Credit_History_Available', 'Housing', 'Locality', 'Fraud_Risk']\n",
            "percentage of non Fraud Risk is 57.68\n",
            "percentage of Fraud Risk is 42.32\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f7e7605d240>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAELCAYAAAA2mZrgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAPhklEQVR4nO3de6ylVX3G8e8jI2JblcucUpwZHKwTGzSI9pRQNbZl0haoBUqRqLWMdpKpCa1abVpa/7A2baJJWwq2RSeiDqZeqEoZlbSSUWtsQT0URC4apgRkpuCMgHjBS6C//rHXWWyGM8zm8p59nPP9JDtnvWutvfdvkjPzzFr7fd+dqkKSJIAnTLsASdLSYShIkjpDQZLUGQqSpM5QkCR1hoIkqRs0FJLckuQrSa5JMtf6Dk1yeZKb2s9DWn+SnJ9ke5Jrk7xgyNokSQ+1GCuFX6mqY6tqth2fA2yrqnXAtnYMcBKwrj02ARcsQm2SpDHT2D46FdjS2luA08b6L6qRK4GDkxwxhfokadlaMfDrF/CpJAW8q6o2A4dX1e1t/A7g8NZeBdw29twdre929mLlypW1du3ax71oSdqfXXXVVd+sqpmFxoYOhRdX1c4kPw1cnuSr44NVVS0wJpZkE6PtJY488kjm5uYev2olaRlIcuvexgbdPqqqne3nLuAS4DjgG/PbQu3nrjZ9J7Bm7OmrW9+er7m5qmaranZmZsGgkyQ9SoOFQpKfTPKU+Tbwa8B1wFZgQ5u2Abi0tbcCZ7WzkI4H7hnbZpIkLYIht48OBy5JMv8+H6iqf0vyJeDiJBuBW4Ez2/zLgJOB7cC9wGsGrE2StIDBQqGqbgaet0D/ncD6BfoLOHuoeiRJ++YVzZKkzlCQJHWGgiSpMxQkSZ2hIEnqhr6iWdKj9Nr/8mp9PdQ7Xzi770mPgSsFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWDh0KSA5JcneQT7fioJF9Isj3Jh5Mc2Pqf1I63t/G1Q9cmSXqwxVgpvB64cez47cC5VfUs4G5gY+vfCNzd+s9t8yRJi2jQUEiyGvgN4N3tOMAJwEfalC3Aaa19ajumja9v8yVJi2TolcLfA38C/F87Pgz4VlXd1453AKtaexVwG0Abv6fNf5Akm5LMJZnbvXv3kLVL0rIzWCgkeSmwq6quejxft6o2V9VsVc3OzMw8ni8tScveigFf+0XAKUlOBg4CngqcBxycZEVbDawGdrb5O4E1wI4kK4CnAXcOWJ8kaQ+DrRSq6s+qanVVrQVeDny6qn4H+AxwRpu2Abi0tbe2Y9r4p6uqhqpPkvRQ07hO4U+BNybZzugzgwtb/4XAYa3/jcA5U6hNkpa1IbePuqr6LPDZ1r4ZOG6BOT8AXrYY9UiSFuYVzZKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLULcqX7Cxlc6977bRL0BI0e/47p12CNBWuFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSd1goZDkoCRfTPLlJNcneWvrPyrJF5JsT/LhJAe2/ie14+1tfO1QtUmSFjbkSuGHwAlV9TzgWODEJMcDbwfOrapnAXcDG9v8jcDdrf/cNk+StIgGC4Ua+W47fGJ7FHAC8JHWvwU4rbVPbce08fVJMlR9kqSHGvQzhSQHJLkG2AVcDvwP8K2quq9N2QGsau1VwG0Abfwe4LAh65MkPdigoVBV91fVscBq4Djg5x7raybZlGQuydzu3bsfc42SpAcsytlHVfUt4DPALwIHJ1nRhlYDO1t7J7AGoI0/DbhzgdfaXFWzVTU7MzMzeO2StJwMefbRTJKDW/vJwK8CNzIKhzPatA3Apa29tR3Txj9dVTVUfZKkh1qx7ymP2hHAliQHMAqfi6vqE0luAD6U5K+Aq4EL2/wLgfcn2Q7cBbx8wNokSQsYLBSq6lrg+Qv038zo84U9+38AvGyoeiRJ++YVzZKkzlCQJHWGgiSpMxQkSZ2hIEnqJgqFJNsm6ZMk/Xh72FNSkxwE/ASwMskhwPwN6p7KA/cskiTtJ/Z1ncLvA28Ang5cxQOh8G3gHwasS5I0BQ8bClV1HnBekj+sqncsUk2SpCmZ6IrmqnpHkhcCa8efU1UXDVSXJGkKJgqFJO8Hfha4Bri/dRdgKEjSfmTSex/NAkd711JJ2r9Nep3CdcDPDFmIJGn6Jl0prARuSPJF4IfznVV1yiBVSZKmYtJQ+Ishi5AkLQ2Tnn30H0MXIkmavknPPvoOo7ONAA4Engh8r6qeOlRhkqTFN+lK4Snz7SQBTgWOH6ooSdJ0POK7pNbIvwK/PkA9kqQpmnT76PSxwycwum7hB4NUJEmamknPPvrNsfZ9wC2MtpAkSfuRST9TeM3QhUiSpm/SL9lZneSSJLva46NJVg9dnCRpcU36QfN7ga2Mvlfh6cDHW58kaT8yaSjMVNV7q+q+9ngfMDNgXZKkKZg0FO5M8qokB7THq4A7hyxMkrT4Jg2F3wPOBO4AbgfOAF49UE2SpCmZ9JTUvwQ2VNXdAEkOBf6GUVhIkvYTk64UjpkPBICqugt4/jAlSZKmZdJQeEKSQ+YP2kph0lWGJOnHxKT/sP8tcEWSf2nHLwP+epiSJEnTMukVzRclmQNOaF2nV9UNw5UlSZqGibeAWggYBJK0H3vEt86WJO2/BguFJGuSfCbJDUmuT/L61n9oksuT3NR+HtL6k+T8JNuTXJvkBUPVJkla2JArhfuAN1XV0Yy+pe3sJEcD5wDbqmodsK0dA5wErGuPTcAFA9YmSVrAYKFQVbdX1X+39neAG4FVjL6HYUubtgU4rbVPBS5q3+x2JXBwkiOGqk+S9FCL8plCkrWMLnb7AnB4Vd3ehu4ADm/tVcBtY0/b0fokSYtk8FBI8lPAR4E3VNW3x8eqqoB6hK+3Kclckrndu3c/jpVKkgYNhSRPZBQI/1xVH2vd35jfFmo/d7X+ncCasaevbn0PUlWbq2q2qmZnZrx7tyQ9noY8+yjAhcCNVfV3Y0NbgQ2tvQG4dKz/rHYW0vHAPWPbTJKkRTDk/YteBPwu8JUk17S+PwfeBlycZCNwK6NbcgNcBpwMbAfuBfxeaElaZIOFQlV9HshehtcvML+As4eqR5K0b17RLEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkrrBQiHJe5LsSnLdWN+hSS5PclP7eUjrT5Lzk2xPcm2SFwxVlyRp74ZcKbwPOHGPvnOAbVW1DtjWjgFOAta1xybgggHrkiTtxWChUFWfA+7ao/tUYEtrbwFOG+u/qEauBA5OcsRQtUmSFrbYnykcXlW3t/YdwOGtvQq4bWzejtb3EEk2JZlLMrd79+7hKpWkZWhqHzRXVQH1KJ63uapmq2p2ZmZmgMokafla7FD4xvy2UPu5q/XvBNaMzVvd+iRJi2ixQ2ErsKG1NwCXjvWf1c5COh64Z2ybSZK0SFYM9cJJPgj8MrAyyQ7gLcDbgIuTbARuBc5s0y8DTga2A/cCrxmqLknS3g0WClX1ir0MrV9gbgFnD1WLJGkyXtEsSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlCQJHWGgiSpMxQkSZ2hIEnqDAVJUmcoSJI6Q0GS1BkKkqTOUJAkdYaCJKkzFCRJnaEgSeoMBUlSZyhIkjpDQZLUGQqSpM5QkCR1hoIkqTMUJEmdoSBJ6gwFSVK3pEIhyYlJvpZke5Jzpl2PJC03SyYUkhwA/CNwEnA08IokR0+3KklaXpZMKADHAdur6uaq+hHwIeDUKdckScvKUgqFVcBtY8c7Wp8kaZGsmHYBj1SSTcCmdvjdJF+bZj37mZXAN6ddxJLwjndNuwI9mL+bzeP0m/mMvQ0spVDYCawZO17d+h6kqjYDmxerqOUkyVxVzU67DmlP/m4unqW0ffQlYF2So5IcCLwc2DrlmiRpWVkyK4Wqui/JHwD/DhwAvKeqrp9yWZK0rCyZUACoqsuAy6ZdxzLmtpyWKn83F0mqato1SJKWiKX0mYIkacoMBXl7ES1ZSd6TZFeS66Zdy3JhKCxz3l5ES9z7gBOnXcRyYijI24toyaqqzwF3TbuO5cRQkLcXkdQZCpKkzlDQRLcXkbQ8GAry9iKSOkNhmauq+4D524vcCFzs7UW0VCT5IHAF8OwkO5JsnHZN+zuvaJYkda4UJEmdoSBJ6gwFSVJnKEiSOkNBktQZCpKkzlDQspPk/iTXjD3WDvAetyRZOUEN1yX5eJKDW//Tk3zkYZ631ttIa0iGgpaj71fVsWOPW+YHMrIYfy/ma3guo7uAng1QVf9bVWcswvtLCzIUtOy1/31/LclFwHXAmiQXJJlLcn2St47N7SuAJLNJPtvahyX5VJv/biCPoIQraHemHV8JJHlOki+2FcW1SdbtUfczk1yd5Bcey59fGmcoaDl68tjW0SWtbx3wT1X1nKq6FXhzVc0CxwC/lOSYfbzmW4DPV9VzgEuAIycppH3J0XoWvt/Ua4HzqupYYJbRbc3nn/ds4KPAq6vqS5O8lzSJFdMuQJqC77d/aIHR/86BW6vqyrE5ZybZxOjvyBGMvpXu2od5zZcApwNU1SeT3L2PGp6c5BpGK4QbgcsXmHMF8OYkq4GPVdVNSQBmgEuB06vqhn28j/SIuFKQRr4330hyFPDHwPqqOgb4JHBQG76PB/7eHMSjNx9Mz2C01XT2nhOq6gPAKcD3gcuSnNCG7gG+Drz4Mby/tCBDQXqopzIKiXuSHM7o+6vn3QL8fGv/9lj/54BXAiQ5CThkkjeqqnuB1wFvSvKglXuSZwI3V9X5jFYG81tYPwJ+CzgrySsn/2NJ+2YoSHuoqi8DVwNfBT4A/OfY8FuB85LMAffv0f+SJNcz2kb6+iN4v6sZbU29Yo+hM4Hr2jbTc4GLxp7zPeClwB8lOWXS95L2xVtnS5I6VwqSpM6zj6SBJDkM2LbA0PqqunOx65Em4faRJKlz+0iS1BkKkqTOUJAkdYaCJKkzFCRJ3f8DMmXT0qN2WEcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RDPZQyz1kWrY",
        "outputId": "5c904cfd-0584-41cd-8298-9ae0b715a2e0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 400
        }
      },
      "source": [
        "df.groupby('Fraud_Risk').mean()\n",
        "\n",
        "df.corr()['Fraud_Risk'] # Matrix to Series"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Gender</th>\n",
              "      <th>Married</th>\n",
              "      <th>Dependents</th>\n",
              "      <th>Education</th>\n",
              "      <th>Self_Employed</th>\n",
              "      <th>ApplicantIncome</th>\n",
              "      <th>CoapplicantIncome</th>\n",
              "      <th>LoanAmount</th>\n",
              "      <th>Loan_Term</th>\n",
              "      <th>Credit_History_Available</th>\n",
              "      <th>Housing</th>\n",
              "      <th>Locality</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>Fraud_Risk</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.574286</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.308571</td>\n",
              "      <td>0.808571</td>\n",
              "      <td>0.177143</td>\n",
              "      <td>4779.974286</td>\n",
              "      <td>1092.642857</td>\n",
              "      <td>125.611429</td>\n",
              "      <td>346.685714</td>\n",
              "      <td>0.974286</td>\n",
              "      <td>0.700000</td>\n",
              "      <td>2.005714</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.851153</td>\n",
              "      <td>0.834382</td>\n",
              "      <td>0.905660</td>\n",
              "      <td>0.777778</td>\n",
              "      <td>0.865828</td>\n",
              "      <td>5530.683438</td>\n",
              "      <td>1774.714885</td>\n",
              "      <td>152.104822</td>\n",
              "      <td>331.849057</td>\n",
              "      <td>0.819706</td>\n",
              "      <td>0.620545</td>\n",
              "      <td>1.958071</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              Gender   Married  ...   Housing  Locality\n",
              "Fraud_Risk                      ...                    \n",
              "0           0.574286  0.000000  ...  0.700000  2.005714\n",
              "1           0.851153  0.834382  ...  0.620545  1.958071\n",
              "\n",
              "[2 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Gender                      0.309570\n",
              "Married                     0.825064\n",
              "Dependents                  0.315421\n",
              "Education                  -0.037406\n",
              "Self_Employed               0.688170\n",
              "ApplicantIncome             0.066347\n",
              "CoapplicantIncome           0.120304\n",
              "LoanAmount                  0.164087\n",
              "Loan_Term                  -0.097339\n",
              "Credit_History_Available   -0.239512\n",
              "Housing                    -0.082534\n",
              "Locality                   -0.030530\n",
              "Fraud_Risk                  1.000000\n",
              "Name: Fraud_Risk, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ej_QpxeZkW0U"
      },
      "source": [
        "X = df[df.columns[0:12]]\n",
        "y = df[df.columns[12:]]\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y.values.ravel(), test_size=0.3, random_state=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AoYtUBZ5kWxQ",
        "outputId": "f18cdc9f-f463-4dee-dcf2-0a072aca85fc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "from lightgbm import LGBMClassifier\n",
        "\n",
        "model = LGBMClassifier(metric='binary_logloss', objective='binary', n_estimators=5000,learning_rate=0.0001, min_child_weight=0.0005)\n",
        "_ = model.fit(X_train, y_train)\n",
        "print(model.score(X_test,y_test))\n",
        "y_hat = model.predict(X_test)\n",
        "\n",
        "model = LGBMClassifier(metric='binary_logloss', objective='binary', n_estimators=50,learning_rate=0.01, min_child_weight=0.9)\n",
        "_ = model.fit(X_train, y_train)\n",
        "print(model.score(X_test,y_test))\n",
        "y_hat = model.predict(X_test)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.9317269076305221\n",
            "0.9317269076305221\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R9MbRT5JkWu8",
        "outputId": "dc811ac9-a1fb-466d-d835-1ce3c4838b8c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        }
      },
      "source": [
        "struct = pd.Series(model.feature_importances_, index=X.columns).nlargest(len(X.columns))\n",
        "struct.plot(kind='barh', figsize=(18, 6))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7f7e74bb03c8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABHoAAAFlCAYAAACUd4+cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde5xddX3v/9ebBMPVoJXSFC9DMWi5BhioNyxY9GebWlGxQDkClhr1KGpbW9PWU60eaypt5SC2FDkWVCoUFUvFI/ebAsIEQgIIohJbIxWomoIoQvj8/tjf6O44k5ncZs+svJ6Pxzxmre/3u77rs/esx86DN9+1dqoKSZIkSZIkzXxbDboASZIkSZIkbRoGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHTF70AVo+nrKU55SQ0NDgy5DkiRJkiT1Wbp06QNVtfNYfQY9GtfQ0BAjIyODLkOSJEmSJPVJ8s3x+rx1S5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjZg+6gJkmyUNVtcMUnWsf4ONt9+nA6vbzQFUdPhU1SJIkSZKkmcOgZxqrqhXAAoAkZwGfq6pPTebYJLOqas1mLE+SJEmSJE0z3rq1CSRZkOSGJMuTXJDkSa39dUluSnJrkk8n2a61n5Xk1CTXJflGkiPX83wvSXJ9kpuTnJ9kh9a+MslfJbkZeHXbf3+SZUlGkhyQ5OIkX0/yhnHmXtTGjtx///0b+c5IkiRJkqSpZNCzaXwMeEdV7QusAN7V2j9TVQdV1X7AV4AT+46ZB7wA+E1gyWRPlOQpwDuBw6vqAGAE+IO+If9ZVQdU1blt/9+qagFwLXAWcCTwHOAvxpq/qs6oquGqGt55550nW5YkSZIkSZoGvHVrIyWZC+xUVVe3prOB89v23kn+N7ATsANwcd+hn62qx4E7kuyyHqd8DrAn8KUkAE8Aru/rP2/U+Avb7xXADlX1IPBgkkeS7FRV31+Pc0uSJEmSpGnMoGfzOgs4oqpuTXICcGhf3yN921mPOQNcWlXHjNP/g1H7a8/z+KhzPo5/f0mSJEmSOsVbtzZSVa0GvpfkkNb0GmDt6p4dgXuTbA0cu4lOeQPw/CTPBEiyfZI9NtHckiRJkiRpBnNFx/rbLsm3+vb/FjgeOL09bPkbwGtb3/8Cvgzc337vuLEnr6r72+qgTyaZ05rfCXx1Y+eWJEmSJEkzW6pq0DVomhoeHq6RkZFBlyFJkiRJkvokWVpVw2P1eeuWJEmSJElSR3jr1jSRZB/g46OaH6mqXxlEPZIkSZIkaeYx6JkmqmoFsGDQdUiSJEmSpJnLW7ckSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjpi9qAL0PS1YtVqhhZfNOG4lUsWTkE1kiRJkiRpIq7okSRJkiRJ6giDHkmSJEmSpI4w6NnMkjw0av+EJKdt4nN8PslOm3JOSZIkSZI08/iMng6oqt8YdA2SJEmSJGnwXNEzQEmGklyRZHmSy5M8vbWfleTIvnEPtd/zklyTZFmS25Ic0tpXJnlKm+8rST6S5PYklyTZto05qJ1nWZKTk9w2iNcsSZIkSZI2H4OezW/bFq4sS7IMeE9f34eAs6tqX+Ac4NQJ5vod4OKqWgDsBywbY8x84MNVtRfwfeBVrf0fgde3Y9eMd4Iki5KMJBlZ8/Dqybw+SZIkSZI0TRj0bH4/rKoFa3+AP+/rey7wT23748ALJpjrJuC1Sd4N7FNVD44x5p6qWhsALQWG2vN7dqyq61v7P41xHABVdUZVDVfV8Kzt5k5QjiRJkiRJmk4Meqanx2h/myRbAU8AqKprgBcCq4Czkhw3xrGP9G2vwecwSZIkSZK0xTDoGazrgKPb9rHAtW17JXBg2/4tYGuAJM8AvlNVHwHOBA6YzEmq6vvAg0l+pTUdva7xkiRJkiRpZnK1x2CdBPxjkj8C7gde29o/AvxLkluBLwA/aO2HAn+U5FHgIWCsFT3jORH4SJLHgasBH8AjSZIkSVLHpKoGXYOmQJIdqmrtt3ctBuZV1VvXdcycefNr3vGnTDj3yiULN02RkiRJkiRpQkmWVtXwWH2u6NlyLEzyJ/T+5t8EThhsOZIkSZIkaVNzRY/GNTw8XCMjI4MuQ5IkSZIk9VnXih4fxixJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRswddgKavFatWM7T4og0+fuWShZuwGkmSJEmSNBFX9EiSJEmSJHWEQY8kSZIkSVJHGPRshCRrkizr+1k8xphDk3xuE5/30CTP69t/Q5LjNuU5JEmSJEnSzOMzejbOD6tqwQDOeyjwEHAdQFWdPoAaJEmSJEnSNOOKns0gyUuT3JnkZuCVfe3vTvL2vv3bkgy17eOSLE9ya5KPt7aXJflykluSXJZklzb+DcDvt1VEh/TPm2RBkhvaXBckeVJrvyrJXyW5MclXkxwyRW+HJEmSJEmaIgY9G2fbUbduHZVkG+AjwMuAA4FfmGiSJHsB7wReVFX7AW9tXV8EnlNV+wPnAn9cVSuB04EPVtWCqrp21HQfA95RVfsCK4B39fXNrqqDgbeNapckSZIkSR3grVsb52du3UqyALinqu5u+58AFk0wz4uA86vqAYCq+m5rfypwXpJ5wBOAe9Y1SZK5wE5VdXVrOhs4v2/IZ9rvpcDQOHMsWlvvrCfuPEHZkiRJkiRpOnFFz9R6jP/+nm8zwfgPAadV1T7A6ycxfiKPtN9rGCfkq6ozqmq4qoZnbTd3I08nSZIkSZKmkkHPpncnMJRk97Z/TF/fSuAAgCQHALu19iuAVyf5udb35NY+F1jVto/vm+dBYMfRJ66q1cD3+p6/8xrg6tHjJEmSJElSNxn0bJzRz+hZUlU/onfr00XtYcz39Y3/NPDkJLcDbwa+ClBVtwPvA65Ocivwt238u4HzkywFHuib51+BV6x9GPOomo4HTk6yHFgAvGdTvmBJkiRJkjR9paoGXYOmqTnz5te840/Z4ONXLlm4CauRJEmSJEkASZZW1fBYfa7okSRJkiRJ6gi/dUvj2mfXuYy4KkeSJEmSpBnDFT2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdMXvQBWj6WrFqNUOLLxpoDSuXLBzo+SVJkiRJmklc0SNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEZ0PepL8WZLbkyxPsizJr6xj7FlJjmzbh7TjliXZdoyxQ0l+2PrX/hy3iWp+aFPMs475f/I6JUmSJElSd3T6YcxJngv8JnBAVT2S5CnAEyZ5+LHA+6vqE+sY8/WqWrCxdUqSJEmSJG0KXV/RMw94oKoeAaiqB6rq20kOTHJ1kqVJLk4yr/+gJL8H/Dbw3iTnrO9JkzyU5OS2IuiyJAcnuSrJN5L8VhtzQpJ/ae13J3nXGPOkzXNbkhVJjmrtH0tyRN+4c5K8PMmsNv6mtoLp9X3znJbkriSXAT+/vq9JkiRJkiRNf10Pei4Bnpbkq0n+LsmvJtka+BBwZFUdCHwUeF//QVV1JnAh8EdVdew65t991K1bh7T27YErqmov4EHgfwMvBl4BvKfv+IOBVwH7Aq9OMjxq/lcCC4D9gMOBk1so9X+BEwCSzAWeB1wEnAisrqqDgIOA1yXZrZ33WcCewHFt/JiSLEoykmRkzcOr1/HSJUmSJEnSdNPpW7eq6qEkBwKHAIcB59ELXfYGLk0CMAu4dwNPMd6tWz8GvtC2VwCPVNWjSVYAQ33jLq2q/wRI8hngBcBIX/8LgE9W1RrgO0muBg6qqgtbcLUzvaDo01X1WJKXAPv2PX9nLjAfeGHfPN9OcsV4L6iqzgDOAJgzb35N/q2QJEmSJEmD1umgB6CFG1cBV7Wg5U3A7VX13M142keram1I8jiw9taxx5P0v+ejg5T1CVY+BvwP4Gjgta0twElVdXH/wCS/sR7zSpIkSZKkGarTt24leVaS+X1NC4CvADu3BzWTZOskew2kQHhxkie3b/U6AvjSqP5rgaPas3d2prcy58bWdxbwNoCquqO1XQy8sd2eRpI9kmwPXNM3zzx6q5skSZIkSVLHdH1Fzw7Ah5LsBDwGfA1YRO/WpFPb821mA6cAt2/A/LsnWda3/9GqOnU9jr8R+DTwVOATVTUyqv8C4LnArfRW+/xxVf0HQFV9J8lXgM/2jT+T3q1hN6d3X9r99AKkC4AXAXcA/wZcvx41SpIkSZKkGSI/vcNIUynJCcBwVb15A4/fjt7zfw6oqs3y1OQ58+bXvONP2RxTT9rKJQsHen5JkiRJkqabJEuravQXOgEdv3Wrq5IcTu8WtA9trpBHkiRJkiTNPK7omUCSfYCPj2p+pKp+ZRD1TKXh4eEaGRl9N5kkSZIkSRqkda3o6fozejZaVa2g9xBnSZIkSZKkac1btyRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOmL2oAvQ9LVi1WqGFl806DI2u5VLFg66BEmSJEmSNglX9EiSJEmSJHWEQY8kSZIkSVJHGPRIkiRJkiR1hEHPKEnWJFmW5PYktyb5wyQDe5+SrEzylA089ogke27qmiRJkiRJ0vRk0POzflhVC6pqL+DFwK8D7xpwTRvqCMCgR5IkSZKkLYRBzzpU1X3AIuDN6ZmV5OQkNyVZnuT1AEkOTXJNkouS3JXk9LWrgJK8JMn1SW5Ocn6SHVr7yiR/0dpXJHl2a/+5JJe0FUVnAllbT5L/keTGtuLoH5LMau0PJXlfW4F0Q5JdkjwP+C3g5DZ+9yRvSXJHq/3cKX0zJUmSJEnSZmfQM4Gq+gYwC/h54ERgdVUdBBwEvC7Jbm3owcBJ9FbQ7A68st1y9U7g8Ko6ABgB/qBv+gda+98Db29t7wK+2FYUXQA8HSDJLwNHAc+vqgXAGuDYdsz2wA1VtR9wDfC6qroOuBD4o7ZC6evAYmD/qtoXeMNYrzfJoiQjSUbWPLx6A981SZIkSZI0CLMHXcAM8xJg3yRHtv25wHzgx8CNLRQiySeBFwA/ohf8fCkJwBOA6/vm+0z7vRR4Zdt+4drtqrooyfda+68BBwI3tbm2Be5rfT8GPtc314vHqX85cE6SzwKfHWtAVZ0BnAEwZ978GmceSZIkSZI0DRn0TCDJL9FbPXMfvduoTqqqi0eNORQYHYpUG39pVR0zzvSPtN9rmPhvEeDsqvqTMfoeraq151/XXAvpBUkvA/4syT5V9dgE55UkSZIkSTOEt26tQ5KdgdOB01qQcjHwxiRbt/49kmzfhh+cZLf2bJ6jgC8CNwDPT/LMNn77JHtMcNprgN9p438deFJrvxw4MsnPt74nJ3nGBHM9COzYxm8FPK2qrgTeQW810g6TeR8kSZIkSdLM4Iqen7VtkmXA1sBjwMeBv219ZwJDwM3p3T91P71vtgK4CTgNeCZwJXBBVT2e5ATgk0nmtHHvBL66jvP/RRt/O3Ad8G8AVXVHkncCl7TQ5lHgTcA31zHXucBHkrwFOBr4v0nm0lsddGpVfX8S74ckSZIkSZoh8tM7frSh2q1bb6+q3xx0LZvSnHnza97xpwy6jM1u5ZKFgy5BkiRJkqRJS7K0qobH6vPWLUmSJEmSpI5wRY/GNTw8XCMjI4MuQ5IkSZIk9XFFjyRJkiRJ0hbAoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSNmD7oATV8rVq1maPFFgy5jRlu5ZOGgS5AkSZIkbUFc0SNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9UyjJQ5t77iS/mORTbXtBkt/YXOeUJEmSJEnTi0FPx1TVt6vqyLa7ADDokSRJkiRpC2HQM2Bt1c0NSZYnuSDJk1r7M5NcluTWJDcn2T3JDkkub/srkrx8jPmGktyW5AnAe4CjkixLclSSu5Ps3MZtleRra/clSZIkSdLMZ9AzeB8D3lFV+wIrgHe19nOAD1fVfsDzgHuBHwGvqKoDgMOAv0mSsSatqh8Dfw6cV1ULquo84BPAsW3I4cCtVXV//3FJFiUZSTKy5uHVm/SFSpIkSZKkzcugZ4CSzAV2qqqrW9PZwAuT7AjsWlUXAFTVj6rqYSDAXyZZDlwG7Arssh6n/ChwXNv+XeAfRw+oqjOqariqhmdtN3eDXpckSZIkSRqM2YMuQOvlWGBn4MCqejTJSmCbyR5cVf+e5DtJXgQczE9X90iSJEmSpA5wRc8AVdVq4HtJDmlNrwGurqoHgW8lOQIgyZwk2wFzgftayHMY8IwJTvEgsOOotjPp3cJ1flWt2VSvRZIkSZIkDZ5Bz9TaLsm3+n7+ADgeOLndjrWA3gOUoRf6vKW1Xwf8Ar3n9gwnWUHvFqw7JzjflcCeax/G3NouBHZgjNu2JEmSJEnSzOatW1OoqsYL1p4zxti7gReNMfa548y9Q/u9Eti7bX8XOGjU0P3oPYR5opBIkiRJkiTNMAY9W5Aki4E34rN5JEmSJEnqpFTVoGvQNDU8PFwjIyODLkOSJEmSJPVJsrSqhsfq8xk9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHTF70AVo+lqxajVDiy8adBmdsXLJwkGXIEmSJEnqOFf0SJIkSZIkdYRBjyRJkiRJUkdMm6AnyS8kOTfJ15MsTfL5JHtMwXnfneTtbfs9SQ7fxPO/Lcl2ffsrkzxlU55DkiRJkiQJpknQkyTABcBVVbV7VR0I/Amwy1TWUVV/XlWXbeJp3wZsN+EoSZIkSZKkjTQtgh7gMODRqjp9bUNV3Qp8McnJSW5LsiLJUQBJdkhyeZKbW/vLW/tQkjuTnJPkK0k+tXY1TVtJ84E2/sYkzxxdRJKzkhzZtg9Kcl2SW9v4Hdv817bz3pzkeW3soUmuaudbe/4keQvwi8CVSa4cda6hVuNHktye5JIk27a+Zya5rJ375iS7t/nGei8OTXJ1kn9J8o0kS5Ic22pekWT3Nm7nJJ9OclP7ef6m/iNKkiRJkqTBmi5Bz97A0jHaXwksAPYDDgdOTjIP+BHwiqo6gF5I9DdtVRDAs4C/q6pfBv4L+J99862uqn2A04BTxismyROA84C3VtXac/8QuA94cTvvUcCpfYftT2/1zp7ALwHPr6pTgW8Dh1XVYWOcaj7w4araC/g+8KrWfk5r3w94HnDvOt4LWtsbgF8GXgPsUVUHA2cCJ7Ux/wf4YFUd1M5z5jivfVGSkSQjax5ePd5bJEmSJEmSpqHpEvSM5wXAJ6tqTVV9B7gaOAgI8JdJlgOXAbvy09u8/r2qvtS2P9HmWOuTfb+fu47zPgu4t6puAqiq/6qqx4CtgY8kWQGcTy/UWevGqvpWVT0OLAOGJvH67qmqZW17KTCUZEdg16q6oJ37R1X18DreC4CbqureqnoE+DpwSWtf0VfH4cBpSZYBFwJPTLLD6IKq6oyqGq6q4VnbzZ3ES5AkSZIkSdPF7EEX0NwOHLke448FdgYOrKpHk6wEtml9NWpsTWJ7sn4f+A69FTRb0VtZtNYjfdtrmNx7O/qYbTegptHzPN63/3hfHVsBz6mq/polSZIkSVKHTJcVPVcAc5IsWtuQZF96tzMdlWRWkp2BFwI3AnOB+1rIcxjwjL65np5k7Wqd3wG+2Nd3VN/v69dRz13AvCQHtVp2TDK7nffetmrnNcCsSby2B4EdJzEOgKp6EPhWkiPauee05wxdy9jvxWRdwk9v4yLJgvU4VpIkSZIkzQDTIuipqgJeARzevl79duD9wD8By4Fb6YVBf1xV/0HvGTbD7Raq44A7+6a7C3hTkq8ATwL+vq/vSe12r7fSW50zXj0/phcGfSjJrcCl9FYM/R1wfGt7NvCDSby8M4AvjH4Y8wReA7yl1Xod8Av0vpVsrPdist5C7z1bnuQOes/0kSRJkiRJHZJextINSYaAz1XV3mP0rQSGq+qBKS5rxpozb37NO37cZ1ZrPa1csnDQJUiSJEmSOiDJ0qoaHqtvWqzokSRJkiRJ0sbr1IoebVrDw8M1MjIy6DIkSZIkSVIfV/RIkiRJkiRtAQx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCoEeSJEmSJKkjDHokSZIkSZI6wqBHkiRJkiSpI2YPugBNXytWrWZo8UWDLkN9Vi5ZOOgSJEmSJEnTmCt6JEmSJEmSOsKgR5IkSZIkqSMMegYgyS5J/inJN5IsTXJ9kldsgnkPTfK5TVGjJEmSJEmaeQx6pliSAJ8FrqmqX6qqA4GjgacOoBaf0SRJkiRJUocY9Ey9FwE/rqrT1zZU1Ter6kNJZiU5OclNSZYneT38ZKXOVUk+leTOJOe0wIgkL21tNwOvXDtnku2TfDTJjUluSfLy1n5CkguTXAFcPqWvXJIkSZIkbVau6Jh6ewE3j9N3IrC6qg5KMgf4UpJLWt/+7dhvA18Cnp9kBPgIvfDoa8B5fXP9GXBFVf1ukp2AG5Nc1voOAPatqu+OLiDJImARwKwn7rwRL1OSJEmSJE01g54BS/Jh4AXAj4FvAvsmObJ1zwXmt74bq+pb7ZhlwBDwEHBPVd3d2j9BC2mAlwC/leTtbX8b4Olt+9KxQh6AqjoDOANgzrz5tYlepiRJkiRJmgIGPVPvduBVa3eq6k1JngKMAP8GnFRVF/cfkORQ4JG+pjVM/LcL8KqqumvUXL8C/GCDq5ckSZIkSdOWz+iZelcA2yR5Y1/bdu33xcAbk2wNkGSPJNuvY647gaEku7f9Y/r6LgZO6nuWz/6bpHpJkiRJkjRtGfRMsaoq4AjgV5Pck+RG4GzgHcCZwB3AzUluA/6Bdazcqaof0btV66L2MOb7+rrfC2wNLE9ye9uXJEmSJEkdll7uIP2sOfPm17zjTxl0GeqzcsnCQZcgSZIkSRqwJEuranisPlf0SJIkSZIkdYQPY9a49tl1LiOuIJEkSZIkacZwRY8kSZIkSVJHGPRIkiRJkiR1hEGPJEmSJElSRxj0SJIkSZIkdYRBjyRJkiRJUkcY9EiSJEmSJHWEQY8kSZIkSVJHGPRIkiRJkiR1hEGPJEmSJElSRxj0SJIkSZIkdYRBjyRJkiRJUkcY9EiSJEmSJHWEQY8kSZIkSVJHzB50AZq+VqxazdDiiwZdhrZwK5csHHQJkiRJkjRjuKJHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMmFfQk+YUk5yb5epKlST6fZI8NOWGSs5Ic2bbPTLJn2/7TSRz70Kj9E5Kc1rbfkOS4dRx7aJLnbUjN6yvJsiTnbsTxw0lObds/eY3rGP/uJG8fo30oyW0bWockSZIkSZpZJnwYc5IAFwBnV9XRrW0/YBfgq21/dlU9tr4nr6rf69v9U+Av13eOvrlOn2DIocBDwHWTnXNDXleSXwZmAYck2b6qfrA+xwNU1Qgwsr7HSZIkSZKkLdtkVvQcBjzaH6RU1a3ArCTXJrkQuCPJrCQnJ7kpyfIkr4deUJTktCR3JbkM+Pm18yS5qq1eWQJs21bCnLMhL6R/VUuStyS5o9VxbpIh4A3A77dzHNJWu1zRxlye5Ont2LOSnJ7ky8AHktydZOfWt1WSr63dH8cxwMeBS4CXt+NuSLLXGK/74CTXJ7klyXVJntX6D03yuTFe48uSfLmNvyzJLn3d+7W57k7yujGOHfPvI0mSJEmSumMyX6++N7B0nL4DgL2r6p4ki4DVVXVQkjnAl5JcAuwPPAvYk94qoDuAj/ZPUlWLk7y5qhZMUMu2SZb17T8ZuHCMcYuB3arqkSQ7VdX3k5wOPFRVfw2Q5F/prVI6O8nvAqcCR7Tjnwo8r6rWJFkNHAucAhwO3FpV96+jxqOAFwPPBk4C/gk4D/ht4F1J5gHzqmokyROBQ6rqsSSH01vR9Kp1zP1F4DlVVUl+D/hj4A9b377Ac4DtgVuSjP5e9BMZ4+9TVff0D2p/x0UAs564rjxLkiRJkiRNN5MJetblxr6g4CXAvmufvwPMBeYDLwQ+WVVrgG8nuWIjzvfD/jAoyQnA8BjjlgPnJPks8Nlx5nou8Mq2/XHgA31957d6oRdK/Qu9oOd3gX8cr7gkw8ADVfVvSVYBH03yZOCf6a3weRe9wOdT7ZC5wNlJ5gMFbD3e3M1TgfNaWPQEoD+k+Zeq+iHwwyRXAgcD/aHYeH+f/xb0VNUZwBkAc+bNrwnqkSRJkiRJ08hkbt26HThwnL7+588EOKmqFrSf3arqko2ucMMsBD5Mb8XRTUnWN9D6yeuqqn8Hvs85g2AAABIQSURBVJPkRfTCk/+3juOOAZ6dZCXwdeCJwKuqahXwn0n2pbfi57w2/r3AlVW1N/AyYJsJ6voQcFpV7QO8ftT40aHM6P3p9PeRJEmSJEmbwWSCniuAOe2WHgBaYHHIqHEXA29MsnUbs0eS7YFrgKPaM2Lm0Xvmz1geXXvsxkiyFfC0qroSeAe9lSs7AA8CO/YNvQ44um0fC1y7jmnPBD7Bf1/pM9Z5fxvYp6qGqmqI3jN6jmlDzqN3q9Xcqlre2uYCq9r2CZN4ef3jjx/V9/Ik2yT5OXoPnr5pVP94fx9JkiRJktQREwY9VVXAK4DD0/t69duB9wP/MWromfSev3Nz+0rvf6B3a9gFwN2t72PA9eOc6gxg+YY+jLnPLOATSVYAtwCnVtX3gX8FXrH2Ycz0np/z2iTLgdcAb13HnBfSC4vGvW2LXvC1qqq+3dd2DbBnC7g+RS9Y+ue+/g8A709yC5O7je7dwPlJlgIPjOpbDlwJ3AC8d1QdMP7fR5IkSZIkdUR6OY7WpT1754NVNXoVU6fNmTe/5h1/yqDL0BZu5ZKFgy5BkiRJkqaVJEuraqxnFruiYyJJFgNvpHd71xZln13nMuJ/ZEuSJEmSNGNMu6CnPWPm8jG6fq2q/nOq66mqJcCS/rYkfwa8etTQ86vqfVNWmCRJkiRJ0ijTLuhpYc6CCQcOUAt0DHUkSZIkSdK0Mplv3ZIkSZIkSdIMYNAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR0xe9AFaPpasWo1Q4svGnQZ2sKtXLJw0CVIkiRJ0ozhih5JkiRJkqSOMOiRJEmSJEnqCIOeKZSkknyib392kvuTfG4j5/3FJJ9az2POSnLkxpxXkiRJkiRNLwY9U+sHwN5Jtm37LwZWrc8ESWaP3q+qb1eVoY0kSZIkSVs4g56p93lg7dNljwE+ubYjycFJrk9yS5LrkjyrtZ+Q5MIkVwCXj7E/lOS2NnZWkpOT3JRkeZLXt/YkOS3JXUkuA35+Kl+0JEmSJEna/Ax6pt65wNFJtgH2Bb7c13cncEhV7Q/8OfCXfX0HAEdW1a+Os7/WicDqqjoIOAh4XZLdgFcAzwL2BI4DnrdpX5YkSZIkSRo0v159ilXV8iRD9FbzfH5U91zg7CTzgQK27uu7tKq+u479tV4C7Nv3/J25wHzghcAnq2oN8O22GuhnJFkELAKY9cSd1+elSZIkSZKkAXNFz2BcCPw1fbdtNe8FrqyqvYGXAdv09f1g1NjR+2sFOKmqFrSf3arqkskWVlVnVNVwVQ3P2m7uZA+TJEmSJEnTgEHPYHwU+IuqWjGqfS4/fTjzCRs498XAG5NsDZBkjyTbA9cAR7Vn+MwDDtvA+SVJkiRJ0jRl0DMAVfWtqjp1jK4PAO9PcgsbflvdmcAdwM3tAc3/0Oa6ALi79X0MuH4D55ckSZIkSdNUqmrQNWiamjNvfs07/pRBl6Et3MolCyceJEmSJElbkCRLq2p4rD5X9EiSJEmSJHWEQY8kSZIkSVJH+PXqGtc+u85lxNtmJEmSJEmaMVzRI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHWHQI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRBj2SJEmSJEkdYdAjSZIkSZLUEQY9kiRJkiRJHTF70AVo+lqxajVDiy8adBmSpBlq5ZKFgy5BkiRpi+OKHkmSJEmSpI4w6JEkSZIkSeoIgx5JkiRJkqSOMOhpkjw0Bed4W5IfJZm7uc81QR1/OsjzS5IkSZKkzcOgZ2odA9wEvHLAdRj0SJIkSZLUQQY965BkQZIbkixPckGSJ7X21yW5KcmtST6dZLvWflaSU5Ncl+QbSY7sm2t3YAfgnfQCn7XtJyT5bJJLk6xM8uYkf5DklnbuJ09Qy1VJhtv2U5Ks7Jv3M0m+kOTuJB9o7UuAbZMsS3LOFLyNkiRJkiRpihj0rNvHgHdU1b7ACuBdrf0zVXVQVe0HfAU4se+YecALgN8ElvS1Hw2cC1wLPCvJLn19e9Nb5XMQ8D7g4araH7geOG6CWtZlAXAUsA9wVJKnVdVi4IdVtaCqjh19QJJFSUaSjKx5ePUkTiFJkiRJkqYLg55xtOfo7FRVV7ems4EXtu29k1ybZAVwLLBX36GfrarHq+oOoD/MOQY4t6oeBz4NvLqv78qqerCq7gdWA//a2lcAQxPUsi6XV9XqqvoRcAfwjIkOqKozqmq4qoZnbTfQRwlJkiRJkqT1NHvQBcxQZwFHVNWtSU4ADu3re6RvOwBJ9gHmA5cmAXgCcA9w2hjHPN63/zgT/40e46eB3Taj+vrnXTOJuSRJkiRJ0gzmip5xVNVq4HtJDmlNrwHWrqjZEbg3ydb0VvRM5Bjg3VU11H5+EfjFJBOusJlELSuBA9v2kUzOo612SZIkSZLUIa7w+Kntknyrb/9vgeOB09vDlr8BvLb1/S/gy8D97feOE8x9NPAbo9ouaO3fmWR949Xy18A/J1kEXDTJuc4Alie5eazn9EiSJEmSpJkpVTXoGjRNzZk3v+Ydf8qgy5AkzVArlywcdAmSJEmdlGRpVQ2P1eetW5IkSZIkSR3hrVsa1z67zmXE/xsrSZIkSdKM4YoeSZIkSZKkjjDokSRJkiRJ6giDHkmSJEmSpI4w6JEkSZIkSeoIgx5JkiRJkqSOMOiRJEmSJEnqCIMeSZIkSZKkjjDokSRJkiRJ6giDHkmSJEmSpI4w6JEkSZIkSeoIgx5JkiRJkqSOMOiRJEmSJEnqiNmDLkDT14pVqxlafNGgy5AkSZIkaZNbuWThoEvYLFzRI0mSJEmS1BEGPZIkSZIkSR1h0CNJkiRJktQRMzroSXJEkkry7I2Y46wkR7btM5PsuekqhCR/Omr/oU05vyRJkiRJ0lozOugBjgG+2H5vtKr6vaq6Y1PM1edPJx4iSZIkSZK08WZs0JNkB+AFwInA0a3t0CTXJLkoyV1JTk+yVet7KMkHk9ye5PIkO48x51VJhtv2S5PcnOTWJJe3toOTXJ/kliTXJXlWaz8hyWeSfCHJ3Uk+0NqXANsmWZbknFHnOrSd71NJ7kxyTpK0voPa/LcmuTHJjkm2SfKPSVa08x/Wd+7PJrk0ycokb07yB23MDUme3Mbt3upbmuTajVkFJUmSJEmSpqcZG/QALwe+UFVfBf4zyYGt/WDgJGBPYHfgla19e2CkqvYCrgbeNd7ELQT6CPCqqtoPeHXruhM4pKr2B/4c+Mu+wxYARwH7AEcleVpVLQZ+WFULqurYMU61P/C2VusvAc9P8gTgPOCt7dyHAz8E3gRUVe1DbwXT2Um2afPs3V7nQcD7gIdbjdcDx7UxZwAnVdWBwNuBvxvntS9KMpJkZM3Dq8d7iyRJkiRJ0jQ0e9AFbIRjgP/Tts9t+58DbqyqbwAk+SS9VT+fAh6nF6AAfAL4zDrmfg5wTVXdA1BV323tc+kFLPOBArbuO+byqlrdznsH8Azg3yd4DTdW1bfaMcuAIWA1cG9V3dTO/V+t/wXAh1rbnUm+CezR5rmyqh4EHkyyGvjX1r4C2LetfnoecH5bNAQwZ6yCquoMeqEQc+bNrwnqlyRJkiRJ08iMDHra7UgvAvZJUsAsesHLRe13v/HCig0JMd5LL1R5RZIh4Kq+vkf6ttcwufd2Q46ZaJ7H+/Yfb3NuBXy/qhZs4PySJEmSJGkGmKm3bh0JfLyqnlFVQ1X1NOAe4BDg4CS7tWfzHEXvYc3Qe61Htu3f6Wsfyw3AC5PsBj8JlqC3omdV2z5hkrU+mmTriYf9xF3AvCQHtXPvmGQ2cC1wbGvbA3h6GzuhtironiSvbscnyX7rUZMkSZIkSZoBZmrQcwxwwai2T7f2m4DTgK/QC3/WjvsBvRDoNnqrgd4z3uRVdT+wCPhMklv56S1fHwDen+QWJr/65gxg+eiHMa/j3D+mF1B9qJ37UmAbes/U2SrJilbPCVX1yPgz/YxjgRPbnLfTe8aRJEmSJEnqkFR15zEsSQ4F3l5VvzlG30NVtcPUVzVzzZk3v+Ydf8qgy5AkSZIkaZNbuWThoEvYYEmWVtXwWH0zdUWPJEmSJEmSRunUih5tWsPDwzUyMjLoMiRJkiRJUh9X9EiSJEmSJG0BDHokSZIkSZI6wqBHkiRJkiSpIwx6JEmSJEmSOsKgR5IkSZIkqSMMeiRJkiRJkjrCr1fXuJI8CNw16Dq0xXsK8MCgi9AWz+tQ04HXoaYLr0VNB16Hmg4GeR0+o6p2Hqtj9lRXohnlrqoaHnQR2rIlGfE61KB5HWo68DrUdOG1qOnA61DTwXS9Dr11S5IkSZIkqSMMeiRJkiRJkjrCoEfrcsagC5DwOtT04HWo6cDrUNOF16KmA69DTQfT8jr0YcySJEmSJEkd4YoeSZIkSZKkjjDo0ZiSvDTJXUm+lmTxoOvRliPJyiQrkixLMtLanpzk0iR3t99PGnSd6pYkH01yX5Lb+trGvO7Sc2r7fFye5IDBVa4uGec6fHeSVe0zcVmS3+jr+5N2Hd6V5P8bTNXqmiRPS3JlkjuS3J7kra3dz0RNmXVch34masok2SbJjUlubdfhX7T23ZJ8uV1v5yV5Qmuf0/a/1vqHBlW7QY9+RpJZwIeBXwf2BI5Jsudgq9IW5rCqWtD3VYWLgcuraj5weduXNqWzgJeOahvvuvt1YH77WQT8/RTVqO47i5+9DgE+2D4TF1TV5wHav8tHA3u1Y/6u/fstbazHgD+sqj2B5wBvatebn4maSuNdh+BnoqbOI8CLqmo/YAHw0iTPAf6K3nX4TOB7wIlt/InA91r7B9u4gTDo0VgOBr5WVd+oqh8D5wIvH3BN2rK9HDi7bZ8NHDHAWtRBVXUN8N1RzeNddy8HPlY9NwA7JZk3NZWqy8a5DsfzcuDcqnqkqu4Bvkbv329po1TVvVV1c9t+EPgKsCt+JmoKreM6HI+fidrk2ufaQ2136/ZTwIuAT7X20Z+Haz8nPwX8WpJMUbn/jUGPxrIr8O99+99i3R+s0qZUwCVJliZZ1Np2qap72/Z/ALsMpjRtYca77vyM1FR7c7sl5qN9t656HWqza7cd7A98GT8TNSCjrkPwM1FTKMmsJMuA+4BLga8D36+qx9qQ/mvtJ9dh618N/NzUVtxj0CNpunlBVR1Abyn4m5K8sL+zel8V6NcFakp53WmA/h7Ynd6S8XuBvxlsOdpSJNkB+DTwtqr6r/4+PxM1Vca4Dv1M1JSqqjVVtQB4Kr1VYs8ecEmTYtCjsawCnta3/9TWJm12VbWq/b4PuOD/b+/+fWSKogCOfw9DIiIURJQKvZJotqHQSTayCjaiUPAXaDQKlX9ASET8yBTERoTG9rZQsKtRkGhWIqFR4SjuHSabmWjWe8md76d5L+++4hQnJzMn755LKajro8/A6/VzfxFqhkzLO2ukOpOZ6/VH5i/gJn+3IpiH+m8iYhvlz/W9zHxUH1sT1alJeWhNVF8y8yuwDBylbFEd1KXxXPuTh3V9N/Cl41ABGz2abAU4VKeJb6cMNlvqOSbNgIjYGRG7RvfACeAtJf8W62uLwJN+ItSMmZZ3S8C5etLMEeDb2HYGaVNtmHVyilIToeThQj3h4yBlEO6rruNTe+o8iVvAu8y8MbZkTVRnpuWhNVFdioh9EbGn3u8AjlPmRS0D8/W1jfVwVCfngZf1C8jODf79imZNZv6IiMvAC2ArcDszV3sOS7NhP/C4ziwbAPcz83lErADDiLgAfARO9xijGhQRD4A5YG9EfAKuAteZnHfPgJOUQY/fgfOdB6wmTcnDuYg4TNkm8wG4CJCZqxExBNYop9NcysyffcSt5hwDzgJv6lwKgCtYE9WtaXl4xpqoDh0A7tQT3LYAw8x8GhFrwMOIuAa8pjQlqde7EfGecrjCQh9BA0RPDSZJkiRJkiRtMrduSZIkSZIkNcJGjyRJkiRJUiNs9EiSJEmSJDXCRo8kSZIkSVIjbPRIkiRJkiQ1wkaPJEmSJElSI2z0SJIkSZIkNcJGjyRJkiRJUiN+A56ja8ovs8fMAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1296x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aXJrw_kPkWo3",
        "outputId": "5d2e1c7b-c787-4cc4-d66d-5f28c9575b55",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# for the next part, 2**(n*(n-1)) graphs need to be searched. choose N to be <= 5. 6 if you have a new threadripper :) \n",
        "whitelist = np.argsort(model.feature_importances_)[-4:] # last N of the argsort, which is the index\n",
        "whitelist"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 9, 7, 5])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dmmKL8OFkWmp",
        "outputId": "14407de2-31de-4b3e-9adf-42c0f02daca3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "print(list(df.columns))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Gender', 'Married', 'Dependents', 'Education', 'Self_Employed', 'ApplicantIncome', 'CoapplicantIncome', 'LoanAmount', 'Loan_Term', 'Credit_History_Available', 'Housing', 'Locality', 'Fraud_Risk']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3O2vOxn-agUY"
      },
      "source": [
        "df_top = df.iloc[:, whitelist]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1ffkzP3J-Rv4"
      },
      "source": [
        "# from pgmpy.factors.discrete import TabularCPD\n",
        "# from pgmpy.models import BayesianModel\n",
        "# from pgmpy.estimators import ExhaustiveSearch, MaximumLikelihoodEstimator, BayesianEstimator\n",
        "\n",
        "# bm = BayesianModel()\n",
        "# # help(ExhaustiveSearch)\n",
        "\n",
        "# s = ExhaustiveSearch(pd.DataFrame(df_top))\n",
        "# all_edges = [dag.edges() for dag in s.all_dags()]\n",
        "# all_edges[-1]\n",
        "\n",
        "# # help(MaximumLikelihoodEstimator)\n",
        "\n",
        "# bm = BayesianModel(all_edges[-1])\n",
        "# MaximumLikelihoodEstimator(bm,df_top).estimate_cpd('ApplicantIncome')\n",
        "\n",
        "\n",
        "# https://pgmpy.org/_modules/pgmpy/models/BayesianModel.html#BayesianModel\n",
        "# https://github.com/pgmpy/pgmpy_notebook/blob/master/notebooks/9.%20Learning%20Bayesian%20Networks%20from%20Data.ipynb\n",
        "# https://www.kaggle.com/surendrakarna/customer-fraud"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X8Bq8KuhEJfL"
      },
      "source": [
        "# X_train, X_test, y_train, y_test = train_test_split(X, y.values.ravel(), test_size=0.3, random_state=0)\n",
        "# _ = model.fit(X_train, y_train)\n",
        "# print(model.score(X_test,y_test))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FQUk7ds2jGoj"
      },
      "source": [
        "```\n",
        "class BayesClassifier(pomegranate.bayes.BayesModel)\n",
        " |  A Bayes classifier, a more general form of a naive Bayes classifier.\n",
        " |  \n",
        " |  A Bayes classifier, like a naive Bayes classifier, uses Bayes' rule in\n",
        " |  order to calculate the posterior probability of the classes, which are\n",
        " |  used for the predictions. However, a naive Bayes classifier assumes that\n",
        " |  each of the features are independent of each other and so can be modelled\n",
        " |  as independent distributions. A generalization of that, the Bayes\n",
        " |  classifier, allows for an arbitrary covariance between the features. This\n",
        " |  allows for more complicated components to be used, up to and including\n",
        " |  even HMMs to form a classifier over sequences, or mixtures to form a\n",
        " |  classifier with complex emissions.\n",
        "```\n",
        "\n",
        "\n",
        "        from_samples(...) from builtins.type \n",
        "                Create a Bayes classifier directly from the given dataset.\n",
        "\n",
        "               This will initialize the distributions using maximum likelihood estimates\n",
        "               derived by partitioning the dataset using the label vector. If any labels\n",
        "               are missing, the model will be trained using EM in a semi-supervised\n",
        "               setting.\n",
        "       \n",
        "               A homogeneous model can be defined by passing in a single distribution\n",
        "               callable as the first parameter and specifying the number of components,\n",
        "               while a heterogeneous model can be defined by passing in a list of\n",
        "               callables of the appropriate type.\n",
        "       \n",
        "               A Bayes classifier is a superset of the naive Bayes classifier in that\n",
        "               the math is identical, but the distributions used do not have to be\n",
        "               independent for each feature. Simply put, one can create a multivariate\n",
        "               Gaussian Bayes classifier with a full covariance matrix, but a Gaussian\n",
        "               naive Bayes would require a diagonal covariance matrix.\n",
        "       \n",
        "               Parameters\n",
        "               ----------\n",
        "               distributions : array-like, shape (n_components,) or callable\n",
        "                       The components of the model. This should either be a single callable\n",
        "                       if all components will be the same distribution, or an array of\n",
        "                       callables, one for each feature.\n",
        "       \n",
        "               X : array-like, shape (n_samples, n_dimensions)\n",
        "                       This is the data to train on. Each row is a sample, and each column\n",
        "                       is a dimension to train on.\n",
        "       \n",
        "               y : array-like, shape (n_samples,)\n",
        "                       The labels for each sample. The labels should be integers between\n",
        "                       0 and k-1 for a problem with k classes, or -1 if the label is not\n",
        "                       known for that sample.\n",
        "       \n",
        "               "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bPQ_w7t5-R_o",
        "outputId": "d872271f-4694-4873-de1c-08dbdffe539f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "!pip install pomegranate\n",
        "from pomegranate import *\n",
        "\n",
        "y = y.values.ravel()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: pomegranate in /usr/local/lib/python3.6/dist-packages (0.13.3)\n",
            "Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.6/dist-packages (from pomegranate) (1.4.1)\n",
            "Requirement already satisfied: joblib>=0.9.0b4 in /usr/local/lib/python3.6/dist-packages (from pomegranate) (0.15.1)\n",
            "Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.6/dist-packages (from pomegranate) (2.4)\n",
            "Requirement already satisfied: numpy>=1.8.0 in /usr/local/lib/python3.6/dist-packages (from pomegranate) (1.18.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from pomegranate) (3.13)\n",
            "Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx>=2.0->pomegranate) (4.4.2)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KX3wAAfWlNIS"
      },
      "source": [
        "# docs - NaiveBayes: https://pomegranate.readthedocs.io/en/latest/NaiveBayes.html#pomegranate.NaiveBayes.NaiveBayes.from_samples\n",
        "# docs - distributions: https://pomegranate.readthedocs.io/en/latest/Distributions.html\n",
        "# help(NaiveBayes)\n",
        "# help(BayesClassifier)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lxUnbC3cDqQy",
        "outputId": "92cfed0f-c1f5-4a01-894b-e1e913cc67ec",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "model = NaiveBayes.from_samples(NormalDistribution, X, y)\n",
        "pred = model.predict(X_test)\n",
        "print(model.score(X_test,y_test))\n",
        "print(accuracy_score(y_test,pred)) # np.sum(pred == y_test)/ (len(y_test))\n",
        "confusion_matrix(y_test,pred)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.44176706827309237\n",
            "0.44176706827309237\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[110,   0],\n",
              "       [139,   0]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "j93ZLB0MheEh",
        "outputId": "04b9a298-cd41-4c3b-c08f-1881f7d70ead",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "model = BayesClassifier.from_samples(DirichletDistribution, X, y)\n",
        "pred = model.predict(X_test)\n",
        "print(model.score(X_test,y_test))\n",
        "print(accuracy_score(y_test,pred)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.5060240963855421\n",
            "0.5060240963855421\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jELL1UhN-R5O",
        "outputId": "6cb989d3-760b-41ad-9ef1-59ea150ee3fc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "model = BayesClassifier.from_samples(MultivariateGaussianDistribution, X, y)\n",
        "pred = model.predict(X_test)\n",
        "print(model.score(X_test,y_test))\n",
        "print(accuracy_score(y_test,pred)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.9196787148594378\n",
            "0.9196787148594378\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BMXGg4VthSN7",
        "outputId": "25f378de-5d86-4546-9554-673481c88dfd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "model = BayesClassifier.from_samples(MultivariateGaussianDistribution, X, y)\n",
        "pred = model.predict(X_test)\n",
        "print(model.score(X_test,y_test))\n",
        "print(accuracy_score(y_test,pred)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.9196787148594378\n",
            "0.9196787148594378\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OGXmg-YMhSRW"
      },
      "source": [
        "# docs - HMM: https://pomegranate.readthedocs.io/en/latest/MarkovNetwork.html\n",
        "# help(MarkovNetwork)\n",
        "# model = MarkovNetwork.from_samples(X, y, n_jobs=-1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QdijcL3CrPye"
      },
      "source": [
        "# pred = model.predict(X_test)\n",
        "# print(model.score(X_test,y_test))\n",
        "# print(accuracy_score(y_test,pred)) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BEP3S-d4rPvH"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a4PAnqx7-R3E"
      },
      "source": [
        "# auc, roc_curve, roc_auc_score"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LqmeisPr-RuO"
      },
      "source": [
        "# model = BayesClassifier.from_samples(BayesianNetwork, X, y)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4hpm97Av-RqH"
      },
      "source": [
        "# https://pomegranate.readthedocs.io/en/latest/BayesianNetwork.html\n",
        "model = BayesianNetwork.from_samples(X, y, n_jobs=-1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cyLy6yBDbITr"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9SVwr336uOyP"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XOTY7GnouO2H"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z7Ixw3hSuO5b"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2dy80vZ6uO8p"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}