ZHOU_MIRANDA_DMA_Fall19_Lab1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    },
    "colab": {
      "name": "ZHOU_MIRANDA_DMA_Fall19_Lab1.ipynb",
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vKduyi9dItO4"
      },
      "source": [
        "Before you turn this problem in, make sure everything runs as expected. First, **restart the kernel** (in the menubar, select Kernel$\\rightarrow$Restart) and then **run all cells** (in the menubar, select Cell$\\rightarrow$Run All).\n",
        "\n",
        "Make sure you fill in any place that says `YOUR CODE HERE` or \"YOUR ANSWER HERE\", as well as your name and collaborators below:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FsEVbU41ItO7"
      },
      "source": [
        "NAME = \"Miranda Zhou\"\n",
        "COLLABORATORS = \"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M7sDljEMItPA"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "720281502b9b9c7f8930d61c682cb061",
          "grade": false,
          "grade_id": "cell-731531e531b99c39",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "LJNmSc7sItPB"
      },
      "source": [
        "# DMA Assignment 1 - Data Preprocessing\n",
        "\n",
        "Data transformations are useful for preparing a dataset for answering a particular question. Part of this process involves generating features from the dataset you find relevant to the question at hand. For this lab, we will be using a Yelp reviews dataset. Each row in the dataset depicts one review along with the features of the review (the reviewer, the review text, etc.). The goal of this lab is to convert this reviews dataset into a reviewers dataset by creating different features of each reviewer.\n",
        "\n",
        "The submission for this assignment should be done *individually*, although you are allowed to work in groups of 2.\n",
        "\n",
        "# Environment Setup\n",
        "Run this cell to setup your environment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "c4f9a2a35aae087d70dccad8514e5e4f",
          "grade": false,
          "grade_id": "cell-88d067a4407f02bf",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "LIH96tFzItPC",
        "outputId": "7b328777-0d66-4c7d-efbf-7723f048b8c5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 391
        }
      },
      "source": [
        "# Importing libraries\n",
        "import pandas as pd\n",
        "import math\n",
        "import numpy as np\n",
        "import os\n",
        "print('Libraries Imported')\n",
        "\n",
        "#DOWNLOADING DATASET IF NOT PRESENT\n",
        "!wget -nc http://people.ischool.berkeley.edu/~zp/course_datasets/yelp_reviews.csv\n",
        "\n",
        "#!unzip yelp_reviews.zip\n",
        "print('Dataset Downloaded: yelp_reviews.csv')\n",
        "df=pd.read_csv('yelp_reviews.csv')\n",
        "print(df.head())\n",
        "\n",
        "print('Setup Complete')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Libraries Imported\n",
            "--2019-10-22 20:36:07--  http://people.ischool.berkeley.edu/~zp/course_datasets/yelp_reviews.csv\n",
            "Resolving people.ischool.berkeley.edu (people.ischool.berkeley.edu)... 128.32.78.16\n",
            "Connecting to people.ischool.berkeley.edu (people.ischool.berkeley.edu)|128.32.78.16|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 376638166 (359M) [text/csv]\n",
            "Saving to: ‘yelp_reviews.csv’\n",
            "\n",
            "yelp_reviews.csv    100%[===================>] 359.19M  1.40MB/s    in 3m 33s  \n",
            "\n",
            "2019-10-22 20:39:45 (1.69 MB/s) - ‘yelp_reviews.csv’ saved [376638166/376638166]\n",
            "\n",
            "Dataset Downloaded: yelp_reviews.csv\n",
            "     type             business_id  ... useful_votes  funny_votes\n",
            "0  review  mxrXVZWc6PWk81gvOVNOUw  ...            0            0\n",
            "1  review  mxrXVZWc6PWk81gvOVNOUw  ...            0            0\n",
            "2  review  kK4AzZ0YWI-U2G-paAL7Fg  ...            0            0\n",
            "3  review  mxrXVZWc6PWk81gvOVNOUw  ...            1            0\n",
            "4  review  mxrXVZWc6PWk81gvOVNOUw  ...            0            1\n",
            "\n",
            "[5 rows x 9 columns]\n",
            "Setup Complete\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "51be56b978a555a682bd1a02d85a7434",
          "grade": false,
          "grade_id": "cell-63034c0f0a81dc85",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "esN8qRHkItPG"
      },
      "source": [
        "**Q1: What was the highest number of reviews for any one business_id?**\n",
        "- For this task, we will need to group the reviews dataset by business_id. This will aggregate data for each business, which is what we need. This can be done using [groupby](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.groupby.html).\n",
        "    - ```yelp_businesses = yelp_dataset.groupby('business_id').size() ```\n",
        "    - The .size() counts the number of instances for each business_id, which gives us the number of reviews as each instance in this dataset is a review.\n",
        "    - The following command will sort this list, after which you can take note of the highest value: ```sorted_yelp_businesses = yelp_businesses.sort(ascending=False,inplace=False)```\n",
        "    - This approach allows you to see the data structure being used in the sort. A quicker approach to getting the max would be to use the max function: ```max(yelp_businesses)```\n",
        "<br>\n",
        "<br>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JyRC_9TVhFV3",
        "outputId": "23659bb4-230a-4bea-e282-9e912dd1a0fd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "df.groupby('business_id').size()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "business_id\n",
              "--5jkZ3-nUPZxUvtcbr8Uw     25\n",
              "--AKjxBmhm9DWrh-e0hTOw      1\n",
              "--BlvDO_RG2yElKu9XA1_g     10\n",
              "--Ol5mVSMaW8ExtmWRUmKA      1\n",
              "--Y_2lDOtVDioX5bwF6GIw      6\n",
              "--jFTZmywe7StuZ2hEjxyA      3\n",
              "--nQiUBxtpjd_ZBuO_PH6w      2\n",
              "--pOlFxITWnhzc7SHSIP0A      8\n",
              "-0Ackw6MF82PXO9f9Jh_Kg     97\n",
              "-0Oh0BEtQEC9OmmzZ_H5Bg     24\n",
              "-0ZDRoepf3gwd9fpsw1bRQ      1\n",
              "-0lRv242OzyPUB8c5E_UhQ      1\n",
              "-0vgpat36Zkkf7RRhcAchA      2\n",
              "-0xwm_AwS0d_5rwUprHGuw      5\n",
              "-1-sUowC7fJ-cn7mCxvJ5w      1\n",
              "-1B-DEGkLE1kDj5ENAF2NQ      4\n",
              "-1BzcQK-HDA6LVOThHMpsw      2\n",
              "-1ERbsOf9XOC9wGbZYFr7g     55\n",
              "-1HFRXSGTZwFe3LTO2LfbA      1\n",
              "-1Lu6lHl7S4_6tAOB8f9ig      2\n",
              "-1crNFo4E5rosISr59OOhA     12\n",
              "-1tQu_yKCOgOj-WtXmMYIA    130\n",
              "-1uA36nuZfBxQwmejyLcdg      1\n",
              "-1wwjybyjStmPoKZif_8tA      2\n",
              "-25FveJPYkjFxVJhu75F3w     11\n",
              "-27u-98Ugczq_xsNPdcvEg      3\n",
              "-2JwNR9k7Df7bkbl93IvJQ      1\n",
              "-2UjtyG3kx9Ob3qVMlqe-g      1\n",
              "-2_O3_WKgO5In5XO5extNQ      2\n",
              "-2b5uNj_OLmDrWYIQoSjWg      4\n",
              "                         ... \n",
              "zwHM6hz8swKIG-4-BSKMPA      1\n",
              "zwHcKioS_8sp--GKsg0wrw      5\n",
              "zwW9T7wOiMGvkUpRUwcDWQ      2\n",
              "zwWXHOPzfO6Yb32uOLVUuw      1\n",
              "zweNypn2M0mjNNDtNL-a_Q      2\n",
              "zx6iPJg8nLkgzD_7p3qOnA      3\n",
              "zx7luqamFItkjoZtzyhZJw      1\n",
              "zxD8WoHXyXDCD_sTywmogQ     43\n",
              "zxNT4025QCHXNf10WJYZkw      1\n",
              "zxjMiqlfUuIQRltSuj9FpQ      8\n",
              "zxo9nZwhlW1ZM7OUJP_1ag      9\n",
              "zxqvU415r_RtZRKDtdbIKQ     43\n",
              "zxviVY0ypMZMkZDfE0Ffqw      1\n",
              "zyCrtvCPdSZDhUrH86vbeA      2\n",
              "zyFZme7L_6kfvex2sgPfCA      1\n",
              "zyI8_HQPEwkJ22WMb9ILVQ      1\n",
              "zyIELEt6zTdBvGgxReq86A      1\n",
              "zyV7n8eXVTAtJrryRswHvA      1\n",
              "zyZ883vct64bKzNv4wy6Yg      4\n",
              "zyvkb5nMWUIGo8J2YirW8g      3\n",
              "zyvujnBGGpG3WWiDPkil9w     21\n",
              "zz9dGH66r16YLNaxN80_Bg      1\n",
              "zzGy7cD_5E6sEhs-__u4Pg      2\n",
              "zzIP8b0fpMEacKp5KzuLrA      1\n",
              "zzUUj9BUNn0_0F5hwy7RRw      2\n",
              "zzYURqVx-3W5STDMmh6oxw      2\n",
              "zzhykRiQh2FyrYEPMfBw0A      5\n",
              "zzknylIEbiITBePfIYjXfA      4\n",
              "zzrm5HEoYKEsfdi8XxSXuQ      1\n",
              "zzxb0YNkRvH1UgifuR3BvQ      1\n",
              "Length: 43317, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "a4b3a58ab839a93b83e514e1db0c11e3",
          "grade": false,
          "grade_id": "cell-b9fd95d084732ea0",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "AbKk0rohItPG"
      },
      "source": [
        "#Make sure you return the answer value in this function\n",
        "def q1(df):\n",
        "    yelp_businesses = df.groupby('business_id').size()\n",
        "  \n",
        "    return max(yelp_businesses)\n",
        "     # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "e989434a5e20d93eb1500308f7ecf163",
          "grade": true,
          "grade_id": "cell-1d1402069f0f336b",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "rzD83og_ItPJ",
        "outputId": "0369de41-3434-462a-fee3-1a1ccfaf9d7f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q1(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4128\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "6cde0278ab840df1fb90b16ca8c14a8a",
          "grade": false,
          "grade_id": "cell-c998b50208612779",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "fZ_McPZ-ItPL"
      },
      "source": [
        "**Q2: What was the average number of reviews for a business_id?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "6ff0153f15cd01aa6d958646c22cd2eb",
          "grade": false,
          "grade_id": "cell-bda529e7277d1978",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "0V-m-GMAItPM"
      },
      "source": [
        "#Make sure you return the answer value in this function\n",
        "def q2(df):\n",
        "    yelp_businesses = df.groupby('business_id').size()\n",
        "    return yelp_businesses.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "72d8375074f6e5eb294215044c460696",
          "grade": true,
          "grade_id": "cell-febce0a51b6da9da",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "V_9IXDmvItPO",
        "outputId": "9599493a-26dd-4355-f05b-2bd9214fa9de",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q2(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "12.63413902163123\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "d6874779b6c6d2e361fc9891455b6960",
          "grade": false,
          "grade_id": "cell-a568a611d600caa2",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "T9bRVaYZItPQ"
      },
      "source": [
        "**Q3: What is the average number of reviews per reviewer?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "369c594937118aa98e0aea635c345943",
          "grade": false,
          "grade_id": "cell-f92e05c4b7a28174",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "J58jfsKTItPQ"
      },
      "source": [
        "#Make sure you return the answer value in this function\n",
        "def q3(df):\n",
        "    yelp_users = df.groupby('user_id').size()\n",
        "    return yelp_users.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "7ed7ddd4ec46674bdd89445dc1034175",
          "grade": true,
          "grade_id": "cell-d76279e8e7d4a79f",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "Wx5wcx6pItPT",
        "outputId": "cfdd87ed-0874-4408-a7cb-7b71a1118c5d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q3(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3.188511934933203\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "e8d5fd42acde59c3bd19cc07db76ac1c",
          "grade": false,
          "grade_id": "cell-c76f8dc8692f2472",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "oa0doTGiItPW"
      },
      "source": [
        "**Q4: Calculate the total cool votes per reviewer, then average these totals across reviewers.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "2db144055be1d1f506af2bbdcf924e58",
          "grade": false,
          "grade_id": "cell-f06192fbabe75195",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "sWuWYjrSItPX"
      },
      "source": [
        "#Make sure you return the answer value in this function\n",
        "def q4(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    cool_votes = yelp_users['cool_votes']\n",
        "    return cool_votes.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "70b555da34836694349c4923beb1950b",
          "grade": true,
          "grade_id": "cell-cabbe00e6fee304e",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "z3o9lMo-ItPZ",
        "outputId": "80786453-4961-4f28-e86b-08dbd6f78d64",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q4(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.2417282785380945\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "1e80dfdd5311c1dd540fafe446ad10c3",
          "grade": false,
          "grade_id": "cell-9e278b8cd31ace62",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "fDD4Lh8fItPb"
      },
      "source": [
        "**Q5: Calculate the total funny votes per reviewer, then average these totals across reviewers.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "cb6393f6479792516d064ab368490d90",
          "grade": false,
          "grade_id": "cell-93635f0a6f297df9",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "KiVt8qB7ItPc"
      },
      "source": [
        "#Make sure you return the answer value in this function\n",
        "def q5(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    funny_votes = yelp_users['funny_votes']\n",
        "    return funny_votes.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "d3be5fcefa0f798be6504de7f8f1b344",
          "grade": true,
          "grade_id": "cell-4c41413214d2d4f1",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "bWzUljWAItPe",
        "outputId": "fab3b914-cf1f-4876-8e8a-5f45cc3202c7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q5(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1.10126486404605\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "e62dd2a28323b8d943f63dda321c3aea",
          "grade": false,
          "grade_id": "cell-9554ed7bdee76fb1",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "qx0rBKw9ItPg"
      },
      "source": [
        "**Q6: Calculate the total useful votes per reviewer, then average these totals across reviewers.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "4375ea8e9efa96b5ea65a2f4f553faad",
          "grade": false,
          "grade_id": "cell-2cebd87b7f8b37d1",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "KIYaRM3uItPh"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q6(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    useful_votes = yelp_users['useful_votes']\n",
        "    return useful_votes.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "2435ef28744576ab65e21558b65e0a5e",
          "grade": true,
          "grade_id": "cell-1a546329acbcd0e4",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "xrNe4FWdItPj",
        "outputId": "a17cfa7a-07ee-425a-9f13-69edc02be90a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q6(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.484476138872867\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "7bec70a026d4e6a31507554af78a52f5",
          "grade": false,
          "grade_id": "cell-707c44dc26d5fd6d",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "prqoI7WlItPl"
      },
      "source": [
        "**Q7: What is the average of the log of the number of reviews per reviewer?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "e811c6ccc9b4579d62f6be8bdd89400f",
          "grade": false,
          "grade_id": "cell-af3639b4d0ae4875",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "FJY8ZYe7ItPl"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q7(df):\n",
        "    yelp_users = df.groupby('user_id').size()\n",
        "    log = np.log(yelp_users)\n",
        "    return log.mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "402dec412c7f6a8ed0a470af889b7f0e",
          "grade": true,
          "grade_id": "cell-c751f158082e76b5",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "tUbcKssGItPn",
        "outputId": "35af5089-91bc-4dd8-f654-75598c9487ed",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q7(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.7681766036710075\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "7eac1174620a3636d28aedc382e088a6",
          "grade": false,
          "grade_id": "cell-4dae7fcb5a2a9834",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "p7XxeVByItPs"
      },
      "source": [
        "**Q8: What is the average of the log of the number of cool votes per reviewer?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "3a84de92b20769ba37eccf85971947fa",
          "grade": false,
          "grade_id": "cell-49a6702b0fbe20dc",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "_fME8KFEItPs"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q8(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    cool_votes_no0 = yelp_users[yelp_users['cool_votes'] != 0]\n",
        "    cool_votes_no0['log cool_votes'] = np.log(yelp_users['cool_votes'])\n",
        "    return cool_votes_no0['log cool_votes'].mean()\n",
        "    # YOUR CODE HERE\n",
        "    # raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "94ee17d58a566706dae0200e4048b879",
          "grade": true,
          "grade_id": "cell-7adc5d1c1446e986",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "p569N5qXItPw",
        "outputId": "ed06fb89-b0d3-4bc2-f80e-d561d98cc902",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 188
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q8(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.6653356886588697\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: RuntimeWarning: divide by zero encountered in log\n",
            "  after removing the cwd from sys.path.\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
            "  after removing the cwd from sys.path.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "94b3ae9f3e35636e940ed8b552fd7533",
          "grade": false,
          "grade_id": "cell-c818be35a93d3136",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "AsTuGdV4ItP0"
      },
      "source": [
        "**Q9: What is the average of the log of the number of funny votes per reviewer?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "e9cb4e524bd70de520ca477916741ae4",
          "grade": false,
          "grade_id": "cell-ee758931b311abee",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "FupQfKX0ItP1"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q9(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    funny_votes_no0 = yelp_users[yelp_users['funny_votes'] != 0]\n",
        "    funny_votes_no0['log funny_votes'] = np.log(yelp_users['funny_votes'])\n",
        "    return funny_votes_no0['log funny_votes'].mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "cc2f61a9ac1d7aef1c893435542d1f48",
          "grade": true,
          "grade_id": "cell-6b3ba1daa6c7a4ea",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "B_0q5D1pItP3",
        "outputId": "ded63485-1da0-470c-a9cb-10189a7a3003",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 188
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q9(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.660879397622696\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: RuntimeWarning: divide by zero encountered in log\n",
            "  after removing the cwd from sys.path.\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
            "  after removing the cwd from sys.path.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "51d8486d039a41032e8c104fbbd84a1d",
          "grade": false,
          "grade_id": "cell-db140007a8b4c696",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "fTmF5FMHItP5"
      },
      "source": [
        "**Q10: What is the average of the log of the number of useful votes per reviewer?**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "5fb1ae517ecf1a4b671d34b888f8d130",
          "grade": false,
          "grade_id": "cell-4c5cb9cd0c399dd1",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "tgKKaluaItP5"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q10(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    useful_votes_no0 = yelp_users[yelp_users['useful_votes'] != 0]\n",
        "    useful_votes_no0['log useful_votes'] = np.log(yelp_users['useful_votes'])\n",
        "    return useful_votes_no0['log useful_votes'].mean()\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "7d814f15431622a46272cec34282f321",
          "grade": true,
          "grade_id": "cell-de71e929088d9871",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "4UBoH1ZuItP8",
        "outputId": "7710d993-20e0-4a21-d646-f54970db8bba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 188
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(q10(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.9106025900072982\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: RuntimeWarning: divide by zero encountered in log\n",
            "  after removing the cwd from sys.path.\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
            "  after removing the cwd from sys.path.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "560323764f6c0effd9615228a4dd9537",
          "grade": false,
          "grade_id": "cell-328aa1a4c9a958d8",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "yUBt7rj9ItP-"
      },
      "source": [
        "**Q11: Find the average of the percentage of total cool votes out of total votes for each reviewer.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "c4ca0edbf727b2adc41527ebb2129cef",
          "grade": false,
          "grade_id": "cell-54fe5dd9777347a1",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "LuFzcyytItP-"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "#Remember to multiply by 100 for percentages\n",
        "def q11(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    cool_votes = yelp_users['cool_votes']\n",
        "    total_votes = yelp_users['cool_votes'] + yelp_users['useful_votes'] + yelp_users['funny_votes']\n",
        "    percent = cool_votes / total_votes\n",
        "    return percent.mean() * 100\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "6ba2d96b43cc0796b9c8ec31e17fefe9",
          "grade": true,
          "grade_id": "cell-594b1121d785d15e",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "ftY6isjeItQA",
        "outputId": "edb04c65-4fec-4de3-a7de-8587783088af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "#Remember to multiply by 100 for percentages\n",
        "print(q11(df))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "19.27040524186172\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "f7ca49268591f6d357d5e4d4be17e3c9",
          "grade": false,
          "grade_id": "cell-8b434399df89705e",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "zKe8LJJLItQC"
      },
      "source": [
        "**Q12: Find the average of the percentage of total funny votes out of total votes for each reviewer.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "b869bd5447e8ea81d0af3cae06012b94",
          "grade": false,
          "grade_id": "cell-a3dc739b16a53957",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "DkXVxKdXItQC"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "#Remember to multiply by 100 for percentages\n",
        "def q12(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    funny_votes = yelp_users['funny_votes']\n",
        "    total_votes = yelp_users['cool_votes'] + yelp_users['useful_votes'] + yelp_users['funny_votes']\n",
        "    percent = funny_votes / total_votes\n",
        "    return percent.mean() * 100\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "2ef3655caaaedc3e433262c7db94fb77",
          "grade": true,
          "grade_id": "cell-e7fdee0fd548baf1",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "q6uEzM9HItQE",
        "outputId": "3e397893-b92d-4471-f247-84147c51b6b1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(round(q12(df),2))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "18.26\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "3ce1c8eefc6053b8515dc4218c5a0afb",
          "grade": false,
          "grade_id": "cell-40cd96c922286117",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "wYs2eqhSItQI"
      },
      "source": [
        "**Q13: Find the average of the percentage of total useful votes out of total votes for each reviewer.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "c8e322eafbf209672785a2c9f7c97f52",
          "grade": false,
          "grade_id": "cell-38769f4cbbe9cb8f",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "56CBDjgrItQI"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q13(df):\n",
        "    yelp_users = df.groupby('user_id').sum()\n",
        "    useful_votes = yelp_users['useful_votes']\n",
        "    total_votes = yelp_users['cool_votes'] + yelp_users['useful_votes'] + yelp_users['funny_votes']\n",
        "    percent = useful_votes / total_votes\n",
        "    return percent.mean() * 100\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "d3aeca2366b4c12d564167261c6bc6a2",
          "grade": true,
          "grade_id": "cell-590fb1425068cc06",
          "locked": true,
          "points": 0.5,
          "schema_version": 1,
          "solution": false
        },
        "id": "LvE8ZWSOItQL",
        "outputId": "0fbff875-f53e-47f5-f0c0-136d5b07b491",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(round(q13(df),2))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "62.47\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "2efdbde3ea13f0e1829d7488da2c6d92",
          "grade": false,
          "grade_id": "cell-615a3d7b2c4e0e26",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "2ixankjPItQO"
      },
      "source": [
        "**Q14: Average review text length (in non-space characters)**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "59cc81340d9a1967bd6d3a0ffcacdcf3",
          "grade": false,
          "grade_id": "cell-75ec15303c64c506",
          "locked": false,
          "schema_version": 1,
          "solution": true
        },
        "id": "qgwHiHaVItQP"
      },
      "source": [
        "#Make sure you return the answer in this function\n",
        "def q14(df):\n",
        "#     print(df.shape)\n",
        "    text_len = []\n",
        "    for x in range(0,547273):\n",
        "      txt = len(df['text'][x]) - df['text'][x].count(' ')\n",
        "      text_len.append(txt)\n",
        "    return sum(text_len)/len(text_len)\n",
        "  \n",
        "     #df['text length'] = df['text']\n",
        "    # YOUR CODE HERE\n",
        "    #raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "083e5ff6b77629d8784b8f6a30352251",
          "grade": true,
          "grade_id": "cell-d9fb4fc39a0b21c4",
          "locked": true,
          "points": 1,
          "schema_version": 1,
          "solution": false
        },
        "id": "_XmxRXdFItQQ",
        "outputId": "d8e6f35b-e888-4a3e-8c1e-1c9da3a75067",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "#This is an autograded cell, do not edit\n",
        "print(round(q14(df),0))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "499.0\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "54c8c584ece619ea786165e50a4e1002",
          "grade": false,
          "grade_id": "cell-a21a423546eb8654",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "03gWN2PAItQS"
      },
      "source": [
        "**Q15: Year in which the reviewer wrote the most reviews. Once you have this for each reviewer, subtract the minimum possible year (2005) from each so that your final feature values are 0, 1, 2 etc.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "d6d5ba528ca188b45b1e36e5385caab0",
          "grade": true,
          "grade_id": "cell-5c7a809865483dca",
          "locked": false,
          "points": 1.5,
          "schema_version": 1,
          "solution": true
        },
        "id": "XideHYGkItQT",
        "outputId": "11230753-e66c-413d-87d1-61933abce737",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        }
      },
      "source": [
        "# YOUR CODE HERE\n",
        "import datetime\n",
        "df['date'] = pd.to_datetime(df['date'])\n",
        "df['year'] = df['date'].dt.year\n",
        "grouped = df.groupby(['user_id','year']).count()\n",
        "grouped = grouped[['type']].rename(columns= {'type' : 'count'})\n",
        "grouped = grouped.sort_values(by='count', ascending = False)\n",
        "reset = grouped.reset_index()\n",
        "sort = reset.reset_index().drop(columns='index')\n",
        "sort = sort.iloc[sort.groupby('user_id')['count'].agg(pd.Series.idxmax)]\n",
        "data = sort.sort_values(by='count', ascending=False)\n",
        "data['year of most reviews'] = data['year'] - 2005\n",
        "data = data.drop(columns='year')\n",
        "data.head()\n",
        "#raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>user_id</th>\n",
              "      <th>count</th>\n",
              "      <th>year of most reviews</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>QYS29txRosYV2mGu68Cnhg</td>\n",
              "      <td>347</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>dhVuDuzCimescgn83tWQOA</td>\n",
              "      <td>114</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>86EnhYP1N8VFmHyON_TcJA</td>\n",
              "      <td>96</td>\n",
              "      <td>10</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>YlpzPPySUxJsUJHRyRde8A</td>\n",
              "      <td>86</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>88OFCnILjT5vi2DhWX6rew</td>\n",
              "      <td>76</td>\n",
              "      <td>11</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                  user_id  count  year of most reviews\n",
              "0  QYS29txRosYV2mGu68Cnhg    347                     5\n",
              "1  dhVuDuzCimescgn83tWQOA    114                     7\n",
              "2  86EnhYP1N8VFmHyON_TcJA     96                    10\n",
              "3  YlpzPPySUxJsUJHRyRde8A     86                     6\n",
              "4  88OFCnILjT5vi2DhWX6rew     76                    11"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "deletable": false,
        "editable": false,
        "nbgrader": {
          "checksum": "93f7e9ef2385bdac5184802eb7e1ce75",
          "grade": false,
          "grade_id": "cell-da33d981591f0ff3",
          "locked": true,
          "schema_version": 1,
          "solution": false
        },
        "id": "waqvrQeDItQV"
      },
      "source": [
        "**Q16: Come up with a new feature for every review. This may be derived from existing features.\n",
        "Give your feature the name *my_new_feature.* Display head() of this new feature.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "deletable": false,
        "nbgrader": {
          "checksum": "787d48b01fdbfeab0810cceb64c1abb3",
          "grade": true,
          "grade_id": "cell-09bbdc6a2946b441",
          "locked": false,
          "points": 1,
          "schema_version": 1,
          "solution": true
        },
        "id": "j3TZ3TD2ItQW",
        "outputId": "477bed3b-77c1-42bc-f1ce-55e30e8831d9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 501
        }
      },
      "source": [
        "# YOUR CODE HERE\n",
        "avg_star = df.groupby('user_id').mean().reset_index()\n",
        "avg = avg_star[['user_id', 'stars']]\n",
        "df = pd.merge(df, avg, on = 'user_id')\n",
        "df = df.rename(columns={'stars_x': 'stars', 'stars_y':'my_new_feature'})\n",
        "df.head()\n",
        "#displays feature of total votes of a certain review\n",
        "# raise NotImplementedError()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>type</th>\n",
              "      <th>business_id</th>\n",
              "      <th>user_id</th>\n",
              "      <th>stars</th>\n",
              "      <th>text</th>\n",
              "      <th>date</th>\n",
              "      <th>cool_votes</th>\n",
              "      <th>useful_votes</th>\n",
              "      <th>funny_votes</th>\n",
              "      <th>year</th>\n",
              "      <th>my_new_feature</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>review</td>\n",
              "      <td>mxrXVZWc6PWk81gvOVNOUw</td>\n",
              "      <td>mv7shusL4Xb6TylVYBv4CA</td>\n",
              "      <td>4</td>\n",
              "      <td>Definitely try the duck dish.   I rank it amon...</td>\n",
              "      <td>2011-06-13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2011</td>\n",
              "      <td>4.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>review</td>\n",
              "      <td>mxrXVZWc6PWk81gvOVNOUw</td>\n",
              "      <td>0aN5QPhs-VwK2vusKG0waQ</td>\n",
              "      <td>5</td>\n",
              "      <td>Big Ass Burger was awesome! Great $5 mojitos. ...</td>\n",
              "      <td>2011-06-25</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2011</td>\n",
              "      <td>5.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>review</td>\n",
              "      <td>kK4AzZ0YWI-U2G-paAL7Fg</td>\n",
              "      <td>0aN5QPhs-VwK2vusKG0waQ</td>\n",
              "      <td>5</td>\n",
              "      <td>Unbelievable sandwiches! Good service.</td>\n",
              "      <td>2011-06-25</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2011</td>\n",
              "      <td>5.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>review</td>\n",
              "      <td>mxrXVZWc6PWk81gvOVNOUw</td>\n",
              "      <td>1JUwyYab-uJzEx_FRd81Zg</td>\n",
              "      <td>5</td>\n",
              "      <td>Awesome, awesome, awesome! My mom and sister a...</td>\n",
              "      <td>2011-07-18</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2011</td>\n",
              "      <td>5.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>review</td>\n",
              "      <td>mxrXVZWc6PWk81gvOVNOUw</td>\n",
              "      <td>2Zd3Xy8hUVmZkNg7RyNjhg</td>\n",
              "      <td>4</td>\n",
              "      <td>I had the ribs they were great.  The beer sele...</td>\n",
              "      <td>2011-07-19</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2011</td>\n",
              "      <td>3.470588</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     type             business_id  ...  year  my_new_feature\n",
              "0  review  mxrXVZWc6PWk81gvOVNOUw  ...  2011        4.000000\n",
              "1  review  mxrXVZWc6PWk81gvOVNOUw  ...  2011        5.000000\n",
              "2  review  kK4AzZ0YWI-U2G-paAL7Fg  ...  2011        5.000000\n",
              "3  review  mxrXVZWc6PWk81gvOVNOUw  ...  2011        5.000000\n",
              "4  review  mxrXVZWc6PWk81gvOVNOUw  ...  2011        3.470588\n",
              "\n",
              "[5 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Uscpj1FWrRgW"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}