conference_notebooks/ASONAM_2019/Spotify_Playlist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# RAPIDS & Music: Related Artists Prediction from Playlists\n",
    "\n",
    "### ASONAM 2019 Tutorial\n",
    "### Authors\n",
    " - Corey Nolet [cnolet@nvidia.com]\n",
    "\n",
    "### Table of Contents\n",
    "* Introduction\n",
    "* Data Importing and Formatting\n",
    "* Data Exploration\n",
    "  * Investigating artists\n",
    "  * \n",
    "* Build Playlist Predictor\n",
    "    \n",
    "### Development Notes\n",
    " - Developed using: RAPIDS v0.9.0\n",
    " - Last tested using: RAPIDS v0.9.0 on August 20, 2019"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cudf\n",
    "import cugraph\n",
    "import cuml\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from librmm_cffi import librmm as rmm\n",
    "from librmm_cffi import librmm_config as rmm_cfg\n",
    "\n",
    "rmm_cfg.use_managed_memory = True\n",
    "rmm_cfg.use_pool_allocator = False\n",
    "rmm.initialize() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "Predicting songs for users is a common problem faced by companies like Spotify and Tidal and a lot can be learned by analyzing the playlists generated by users. Playlists are formed by grouping artists and songs together into a themed collection. In this notebook, we will explore the Spotify Million Playlist Dataset, which was created for the RecSys 2018 competition, and see if we can build a playlist recommender. We will \n",
    "\n",
    "1. Perform exploratory analysis of the datast,\n",
    "2. \n",
    "3. Perform graph analytics on the graph representation of the cyber network data, and\n",
    "4. Prepare the results in a way that is suitable for visualization.\n",
    "\n",
    "\n",
    "\n",
    "### TODO: Add information about the dataset, along with a link to the RecSys 2018 dataset page"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Explore Dataset\n",
    "\n",
    "First let's load our dataset onto the GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12.3 s, sys: 5.22 s, total: 17.5 s\n",
      "Wall time: 17.6 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "gdf = cudf.read_csv(\"../data/million_playlist_dataset.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take an initial look at our data. We notice we have all the tracks for each playlist as seperate entries in the dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 576 ms, sys: 88 ms, total: 664 ms\n",
      "Wall time: 688 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_name</th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>description</th>\n",
       "      <th>name</th>\n",
       "      <th>num_followers</th>\n",
       "      <th>pid</th>\n",
       "      <th>pos</th>\n",
       "      <th>track_name</th>\n",
       "      <th>track_uri</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Futuristic</td>\n",
       "      <td>spotify:artist:5Z5jUyiNvFaqp0EVyLNf0p</td>\n",
       "      <td>null</td>\n",
       "      <td>Summer 2017</td>\n",
       "      <td>2</td>\n",
       "      <td>814000</td>\n",
       "      <td>0</td>\n",
       "      <td>Epiphany (feat. NF)</td>\n",
       "      <td>spotify:track:1QOqNh2GvfN0ToramLPw5R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Flame</td>\n",
       "      <td>spotify:artist:2s6kyMmJZFgPCHXU0QxJLp</td>\n",
       "      <td>null</td>\n",
       "      <td>Summer 2017</td>\n",
       "      <td>2</td>\n",
       "      <td>814000</td>\n",
       "      <td>1</td>\n",
       "      <td>Start Over (feat. NF)</td>\n",
       "      <td>spotify:track:57AlxjJrWpWg0vF7eGksir</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NF</td>\n",
       "      <td>spotify:artist:6fOMl44jA4Sp5b9PpYCkzz</td>\n",
       "      <td>null</td>\n",
       "      <td>Summer 2017</td>\n",
       "      <td>2</td>\n",
       "      <td>814000</td>\n",
       "      <td>2</td>\n",
       "      <td>Can You Hold Me</td>\n",
       "      <td>spotify:track:1isXdpzFMO6EIkgdWjV2D8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NF</td>\n",
       "      <td>spotify:artist:6fOMl44jA4Sp5b9PpYCkzz</td>\n",
       "      <td>null</td>\n",
       "      <td>Summer 2017</td>\n",
       "      <td>2</td>\n",
       "      <td>814000</td>\n",
       "      <td>3</td>\n",
       "      <td>Just Being Me</td>\n",
       "      <td>spotify:track:44vMHn5aDNEkUdQmonvHp3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NF</td>\n",
       "      <td>spotify:artist:6fOMl44jA4Sp5b9PpYCkzz</td>\n",
       "      <td>null</td>\n",
       "      <td>Summer 2017</td>\n",
       "      <td>2</td>\n",
       "      <td>814000</td>\n",
       "      <td>4</td>\n",
       "      <td>Only One</td>\n",
       "      <td>spotify:track:1iAqQMUVtp5E1EYJkqZw2g</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  artist_name                             artist_uri description         name  \\\n",
       "0  Futuristic  spotify:artist:5Z5jUyiNvFaqp0EVyLNf0p        null  Summer 2017   \n",
       "1       Flame  spotify:artist:2s6kyMmJZFgPCHXU0QxJLp        null  Summer 2017   \n",
       "2          NF  spotify:artist:6fOMl44jA4Sp5b9PpYCkzz        null  Summer 2017   \n",
       "3          NF  spotify:artist:6fOMl44jA4Sp5b9PpYCkzz        null  Summer 2017   \n",
       "4          NF  spotify:artist:6fOMl44jA4Sp5b9PpYCkzz        null  Summer 2017   \n",
       "\n",
       "   num_followers     pid  pos             track_name  \\\n",
       "0              2  814000    0    Epiphany (feat. NF)   \n",
       "1              2  814000    1  Start Over (feat. NF)   \n",
       "2              2  814000    2        Can You Hold Me   \n",
       "3              2  814000    3          Just Being Me   \n",
       "4              2  814000    4               Only One   \n",
       "\n",
       "                              track_uri  \n",
       "0  spotify:track:1QOqNh2GvfN0ToramLPw5R  \n",
       "1  spotify:track:57AlxjJrWpWg0vF7eGksir  \n",
       "2  spotify:track:1isXdpzFMO6EIkgdWjV2D8  \n",
       "3  spotify:track:44vMHn5aDNEkUdQmonvHp3  \n",
       "4  spotify:track:1iAqQMUVtp5E1EYJkqZw2g  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore Tracks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can count the number of unique playlists in our dataset. Not surprisingly, we have 1M of them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 412 ms, sys: 128 ms, total: 540 ms\n",
      "Wall time: 556 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(1000000,)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gdf.pid.unique().shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And see that we have 66M entries of tracks across all the playlists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 26.2 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(66346428, 9)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gdf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset contains nearly 2.2M unique tracks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 6.07 s, sys: 3 s, total: 9.07 s\n",
      "Wall time: 9.06 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "2262292"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "n_tracks = gdf.track_uri.unique().shape[0]\n",
    "n_tracks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly, it also contains a large number of different tracks with duplicate names. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.92 s, sys: 824 ms, total: 2.74 s\n",
      "Wall time: 2.74 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "1483760"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "n_track_names = gdf.track_name.unique().shape[0]\n",
    "n_track_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Over 34% of the track names are duplicates?!? \n",
    "\n",
    "Seems strange, but no. Spotify categorizes a `track` as a song from an artist on an album. This means the same song appearing on multiple albums (eg various artist compilations, greatest hits) will occur more than once. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3441341789654032"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 - (n_track_names / n_tracks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can verify this by counting the unique tracks by `artist_uri` without regard to the `track_uri`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2190142"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unique_tracks = gdf[[\"artist_uri\", \"track_name\"]].drop_duplicates().shape[0]\n",
    "unique_tracks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That seems much more reasonable. Just over 3% are duplicate songs from different artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.03189243475201253"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 - (unique_tracks / n_tracks)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore Artists\n",
    "\n",
    "To get a general feel for the distribution of artists across playlists, let's count the number of times each artist occurs in playlists\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Artists can occur in playlists more than once. It's also possible they could occur hundreds to thousands of times in the same playlist. For instance, playlists created by labels and promotional companies will often do this. \n",
    "\n",
    "Since we care about the spread and not the depth, let's drop the duplicate artists for each playlist id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.47 s, sys: 1.06 s, total: 3.53 s\n",
      "Wall time: 3.53 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists = gdf[[\"artist_uri\", \"artist_name\", \"pid\"]].drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the effect of dropping duplicates cut the number of total rows by over `40%`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(38088211, 3)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "artists.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4259191919118841"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "1 - (artists.shape[0] / gdf.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Grouping by the artists and performing a `count()` aggregation allows us to count their spread across playlists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.23 s, sys: 656 ms, total: 1.89 s\n",
      "Wall time: 1.89 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists = artists.groupby([\"artist_uri\", \"artist_name\"]).count().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 1.31 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists = artists.rename({\"pid\":\"count\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's peek at the artists that occur in the most playlists. The results are what we'd expect- the top 40 artists ranging between 2010 and 2017"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>131666</th>\n",
       "      <td>spotify:artist:3TVXtAsR1Inumwj472S9r4</td>\n",
       "      <td>Drake</td>\n",
       "      <td>203345</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>221282</th>\n",
       "      <td>spotify:artist:5pKCCKE2ajJHZ9KAiaK11H</td>\n",
       "      <td>Rihanna</td>\n",
       "      <td>150344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>202188</th>\n",
       "      <td>spotify:artist:5K4W6rqBFWDnAN6FQUkS6x</td>\n",
       "      <td>Kanye West</td>\n",
       "      <td>141223</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58791</th>\n",
       "      <td>spotify:artist:1Xyo4u8uXC1ZmMpatF05PJ</td>\n",
       "      <td>The Weeknd</td>\n",
       "      <td>125236</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96863</th>\n",
       "      <td>spotify:artist:2YZyLoL8N0Wb9xBt1NhZWg</td>\n",
       "      <td>Kendrick Lamar</td>\n",
       "      <td>120901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233658</th>\n",
       "      <td>spotify:artist:69GGBxA162lTqCwzJG5jLp</td>\n",
       "      <td>The Chainsmokers</td>\n",
       "      <td>111562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>252550</th>\n",
       "      <td>spotify:artist:6eUKZXaKkcviH0Ku9w2n3V</td>\n",
       "      <td>Ed Sheeran</td>\n",
       "      <td>111422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>273619</th>\n",
       "      <td>spotify:artist:7CajNmpbOovFoOoasH2HaY</td>\n",
       "      <td>Calvin Harris</td>\n",
       "      <td>110881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>263166</th>\n",
       "      <td>spotify:artist:6vWDO969PvNqNYHIOW5v0m</td>\n",
       "      <td>Beyoncé</td>\n",
       "      <td>97468</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2844</th>\n",
       "      <td>spotify:artist:04gDigrS5kc9YWfZHwBETP</td>\n",
       "      <td>Maroon 5</td>\n",
       "      <td>97256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23186</th>\n",
       "      <td>spotify:artist:0c173mlxpT3dSFRgMO8XPh</td>\n",
       "      <td>Big Sean</td>\n",
       "      <td>97174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55016</th>\n",
       "      <td>spotify:artist:1RyvyyTE3xzB2ZywiAwp0i</td>\n",
       "      <td>Future</td>\n",
       "      <td>96258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>280808</th>\n",
       "      <td>spotify:artist:7bXgB6jMjp9ATFy66eO08Z</td>\n",
       "      <td>Chris Brown</td>\n",
       "      <td>95501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78224</th>\n",
       "      <td>spotify:artist:246dkjvS1zLTtiykXe5h60</td>\n",
       "      <td>Post Malone</td>\n",
       "      <td>93227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72426</th>\n",
       "      <td>spotify:artist:1uNFoZAHBGtllmzznpCI3s</td>\n",
       "      <td>Justin Bieber</td>\n",
       "      <td>92492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>178017</th>\n",
       "      <td>spotify:artist:4gzpq5DPGxSnKTe4SA8HAU</td>\n",
       "      <td>Coldplay</td>\n",
       "      <td>87039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24453</th>\n",
       "      <td>spotify:artist:0du5cEVh5yTK9QJze8zA0C</td>\n",
       "      <td>Bruno Mars</td>\n",
       "      <td>86338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16065</th>\n",
       "      <td>spotify:artist:0QHgL1lAIqAw0HtD7YldmP</td>\n",
       "      <td>DJ Khaled</td>\n",
       "      <td>86128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>256675</th>\n",
       "      <td>spotify:artist:6l3HvQ5sa6mXTsMTB19rO5</td>\n",
       "      <td>J. Cole</td>\n",
       "      <td>85749</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>267819</th>\n",
       "      <td>spotify:artist:738wLrAtLtCtFOLvQBXOXp</td>\n",
       "      <td>Major Lazer</td>\n",
       "      <td>85441</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>285132</th>\n",
       "      <td>spotify:artist:7iZtZyCzp3LItcw1wtPI3D</td>\n",
       "      <td>Rae Sremmurd</td>\n",
       "      <td>82758</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>143524</th>\n",
       "      <td>spotify:artist:3nFkdlSjzX9mRTtwJOzDYB</td>\n",
       "      <td>JAY Z</td>\n",
       "      <td>82530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192021</th>\n",
       "      <td>spotify:artist:53XhwfbYqKCa1cC15pYq2q</td>\n",
       "      <td>Imagine Dragons</td>\n",
       "      <td>82516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193022</th>\n",
       "      <td>spotify:artist:55Aa2cqylxrFIXC767Z865</td>\n",
       "      <td>Lil Wayne</td>\n",
       "      <td>81635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>268287</th>\n",
       "      <td>spotify:artist:73sIBHcqh3Z3NyqHKZ7FOL</td>\n",
       "      <td>Childish Gambino</td>\n",
       "      <td>79154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1619</th>\n",
       "      <td>spotify:artist:02kJSzxNuaWGqwubyUba0Z</td>\n",
       "      <td>G-Eazy</td>\n",
       "      <td>77334</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26482</th>\n",
       "      <td>spotify:artist:0hCNtLu0JehylgoiP8L4Gh</td>\n",
       "      <td>Nicki Minaj</td>\n",
       "      <td>76812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>281821</th>\n",
       "      <td>spotify:artist:7dGJo4pcD2V6oG8kP0tJRR</td>\n",
       "      <td>Eminem</td>\n",
       "      <td>76657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258731</th>\n",
       "      <td>spotify:artist:6oMuImdp5ZcFhWP0ESe6mG</td>\n",
       "      <td>Migos</td>\n",
       "      <td>75016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4652</th>\n",
       "      <td>spotify:artist:07YZf4WDAMNwqr4jfgOZ8y</td>\n",
       "      <td>Jason Derulo</td>\n",
       "      <td>74693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28070</th>\n",
       "      <td>spotify:artist:0jnsk9HBra6NMjO2oANoPY</td>\n",
       "      <td>Flo Rida</td>\n",
       "      <td>74522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78151</th>\n",
       "      <td>spotify:artist:23zg3TcAtWQy7J6upgbUnj</td>\n",
       "      <td>Usher</td>\n",
       "      <td>72566</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39911</th>\n",
       "      <td>spotify:artist:137W8MRPWKqSmrBGDBFSop</td>\n",
       "      <td>Wiz Khalifa</td>\n",
       "      <td>71940</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>231763</th>\n",
       "      <td>spotify:artist:66CXWjxzNUsdJxJ2JdwvnR</td>\n",
       "      <td>Ariana Grande</td>\n",
       "      <td>71892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>114439</th>\n",
       "      <td>spotify:artist:31TPClRtHm23RisEBtV3X7</td>\n",
       "      <td>Justin Timberlake</td>\n",
       "      <td>71479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134562</th>\n",
       "      <td>spotify:artist:3YQKmKGau1PzlVlkL1iodx</td>\n",
       "      <td>Twenty One Pilots</td>\n",
       "      <td>70699</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>210823</th>\n",
       "      <td>spotify:artist:5YGY8feqx7naU7z4HrwZM6</td>\n",
       "      <td>Miley Cyrus</td>\n",
       "      <td>70627</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45897</th>\n",
       "      <td>spotify:artist:1Cs0zKBU1kc0i8ypK3B9ai</td>\n",
       "      <td>David Guetta</td>\n",
       "      <td>68325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20794</th>\n",
       "      <td>spotify:artist:0Y5tJX1MQlPlqiwlOH1tJY</td>\n",
       "      <td>Travis Scott</td>\n",
       "      <td>67804</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192307</th>\n",
       "      <td>spotify:artist:540vIaP2JwjQb9dm3aArA4</td>\n",
       "      <td>DJ Snake</td>\n",
       "      <td>66035</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   artist_uri        artist_name   count\n",
       "131666  spotify:artist:3TVXtAsR1Inumwj472S9r4              Drake  203345\n",
       "221282  spotify:artist:5pKCCKE2ajJHZ9KAiaK11H            Rihanna  150344\n",
       "202188  spotify:artist:5K4W6rqBFWDnAN6FQUkS6x         Kanye West  141223\n",
       "58791   spotify:artist:1Xyo4u8uXC1ZmMpatF05PJ         The Weeknd  125236\n",
       "96863   spotify:artist:2YZyLoL8N0Wb9xBt1NhZWg     Kendrick Lamar  120901\n",
       "233658  spotify:artist:69GGBxA162lTqCwzJG5jLp   The Chainsmokers  111562\n",
       "252550  spotify:artist:6eUKZXaKkcviH0Ku9w2n3V         Ed Sheeran  111422\n",
       "273619  spotify:artist:7CajNmpbOovFoOoasH2HaY      Calvin Harris  110881\n",
       "263166  spotify:artist:6vWDO969PvNqNYHIOW5v0m            Beyoncé   97468\n",
       "2844    spotify:artist:04gDigrS5kc9YWfZHwBETP           Maroon 5   97256\n",
       "23186   spotify:artist:0c173mlxpT3dSFRgMO8XPh           Big Sean   97174\n",
       "55016   spotify:artist:1RyvyyTE3xzB2ZywiAwp0i             Future   96258\n",
       "280808  spotify:artist:7bXgB6jMjp9ATFy66eO08Z        Chris Brown   95501\n",
       "78224   spotify:artist:246dkjvS1zLTtiykXe5h60        Post Malone   93227\n",
       "72426   spotify:artist:1uNFoZAHBGtllmzznpCI3s      Justin Bieber   92492\n",
       "178017  spotify:artist:4gzpq5DPGxSnKTe4SA8HAU           Coldplay   87039\n",
       "24453   spotify:artist:0du5cEVh5yTK9QJze8zA0C         Bruno Mars   86338\n",
       "16065   spotify:artist:0QHgL1lAIqAw0HtD7YldmP          DJ Khaled   86128\n",
       "256675  spotify:artist:6l3HvQ5sa6mXTsMTB19rO5            J. Cole   85749\n",
       "267819  spotify:artist:738wLrAtLtCtFOLvQBXOXp        Major Lazer   85441\n",
       "285132  spotify:artist:7iZtZyCzp3LItcw1wtPI3D       Rae Sremmurd   82758\n",
       "143524  spotify:artist:3nFkdlSjzX9mRTtwJOzDYB              JAY Z   82530\n",
       "192021  spotify:artist:53XhwfbYqKCa1cC15pYq2q    Imagine Dragons   82516\n",
       "193022  spotify:artist:55Aa2cqylxrFIXC767Z865          Lil Wayne   81635\n",
       "268287  spotify:artist:73sIBHcqh3Z3NyqHKZ7FOL   Childish Gambino   79154\n",
       "1619    spotify:artist:02kJSzxNuaWGqwubyUba0Z             G-Eazy   77334\n",
       "26482   spotify:artist:0hCNtLu0JehylgoiP8L4Gh        Nicki Minaj   76812\n",
       "281821  spotify:artist:7dGJo4pcD2V6oG8kP0tJRR             Eminem   76657\n",
       "258731  spotify:artist:6oMuImdp5ZcFhWP0ESe6mG              Migos   75016\n",
       "4652    spotify:artist:07YZf4WDAMNwqr4jfgOZ8y       Jason Derulo   74693\n",
       "28070   spotify:artist:0jnsk9HBra6NMjO2oANoPY           Flo Rida   74522\n",
       "78151   spotify:artist:23zg3TcAtWQy7J6upgbUnj              Usher   72566\n",
       "39911   spotify:artist:137W8MRPWKqSmrBGDBFSop        Wiz Khalifa   71940\n",
       "231763  spotify:artist:66CXWjxzNUsdJxJ2JdwvnR      Ariana Grande   71892\n",
       "114439  spotify:artist:31TPClRtHm23RisEBtV3X7  Justin Timberlake   71479\n",
       "134562  spotify:artist:3YQKmKGau1PzlVlkL1iodx  Twenty One Pilots   70699\n",
       "210823  spotify:artist:5YGY8feqx7naU7z4HrwZM6        Miley Cyrus   70627\n",
       "45897   spotify:artist:1Cs0zKBU1kc0i8ypK3B9ai       David Guetta   68325\n",
       "20794   spotify:artist:0Y5tJX1MQlPlqiwlOH1tJY       Travis Scott   67804\n",
       "192307  spotify:artist:540vIaP2JwjQb9dm3aArA4           DJ Snake   66035"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "artists.sort_values(\"count\", ascending=False).head(40)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can peek at the bottom 40 artists (occurring in the lowest number of playlists) and see they are likely not very well-known musicians. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 52 ms, sys: 32 ms, total: 84 ms\n",
      "Wall time: 84.8 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>spotify:artist:0001cekkfdEBoMlwVQvpLg</td>\n",
       "      <td>Jordan Colle</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>spotify:artist:000DnGPNOsxvqb2YEHBePR</td>\n",
       "      <td>The Ruins</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>spotify:artist:000UxvYLQuybj6iVRRCAw1</td>\n",
       "      <td>Primera Etica</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>spotify:artist:000YStE477SUgwTTYceARC</td>\n",
       "      <td>Fiona Xsay</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>spotify:artist:000h2XLY65iWC9u5zgcL1M</td>\n",
       "      <td>Kosmose</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>spotify:artist:000spuc3oKgwYmfg5IE26s</td>\n",
       "      <td>Parliament Syndicate</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>spotify:artist:000xagx3GkcunHTFdB4ly0</td>\n",
       "      <td>Moxa</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>spotify:artist:0011DpkmiLGXnibZ4oKGB5</td>\n",
       "      <td>Green and Yellow</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>spotify:artist:0012MbXnWE3SOtQdfg8CBb</td>\n",
       "      <td>Olivia Cella</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>spotify:artist:001NaPzWk3GKPMkuVNMdGd</td>\n",
       "      <td>Cuzzo Shay</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>spotify:artist:001XyVfgdcNbhtA5dIH07V</td>\n",
       "      <td>Pocketbooks</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>spotify:artist:001zgg6v3jBCLr1Qe25cC3</td>\n",
       "      <td>Diggy Metro</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>spotify:artist:002EU3CH8nAyVzWOS0RFrn</td>\n",
       "      <td>Justine Dorsey &amp; Kerris Dorsey</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>spotify:artist:002TRaMMHsPgv11qv0Oe4N</td>\n",
       "      <td>Daan Kleijn Trio</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>spotify:artist:002bMfvkA3TDMG7mNl2FQl</td>\n",
       "      <td>Twila Paris</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>spotify:artist:002qS9Hyk2fS9Cc0Xe1eBU</td>\n",
       "      <td>Little Distractions</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>spotify:artist:003BVdmFsDDbCyTKt32Bjl</td>\n",
       "      <td>Knut Bell</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>spotify:artist:003KhzdYcJduhb6M8WlYAQ</td>\n",
       "      <td>Germz</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>spotify:artist:003PCFEqETGNDYCMy8JIvu</td>\n",
       "      <td>B.O.U.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>spotify:artist:003XB0vnFW8niBSEHkM82a</td>\n",
       "      <td>Mad</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>spotify:artist:003kJVejiby1fQgWdSy3dw</td>\n",
       "      <td>Edmund Pendelton</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>spotify:artist:003qxd7dVynp8vPbOdV6Ri</td>\n",
       "      <td>Audio Alterna</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>spotify:artist:003xAcE0iwqQKEqqdwbs7i</td>\n",
       "      <td>Student Teachers</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>spotify:artist:0046tfXJIwmZXoclXKQO7a</td>\n",
       "      <td>Various Artists: Karaoke - Ameritz</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>spotify:artist:004COHk61u8a2GAbLeTpl1</td>\n",
       "      <td>Hans Felix Koller</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>spotify:artist:0054Ye49KrJfOOETeTdRog</td>\n",
       "      <td>Chris McGregor's Brotherhood Of Breath</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>spotify:artist:005A36KboUZ12FJMpO9GwF</td>\n",
       "      <td>The Tyrant Lizard Kings</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>spotify:artist:005IA5advoiAuGB0ZBzC2y</td>\n",
       "      <td>Bill Curtis and Friends</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61</th>\n",
       "      <td>spotify:artist:005fY9ByADStuYjcIIN5bQ</td>\n",
       "      <td>Johann Brandl</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>spotify:artist:006Bvo4yIgIiLY7gAmuaaz</td>\n",
       "      <td>L'elan Vital</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69</th>\n",
       "      <td>spotify:artist:006EQjXYKEcojXAkJiJwsZ</td>\n",
       "      <td>Restriction</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>spotify:artist:006Emvooxohe5K7q31mb8d</td>\n",
       "      <td>LeLe</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>spotify:artist:006KL09zdkklU6SjWxBE9W</td>\n",
       "      <td>Drum Origins</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>spotify:artist:006OrdbRA5i2ArxeCm6iEr</td>\n",
       "      <td>JSKEEN</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>spotify:artist:006ZADyabxpgKqMmh8W2yz</td>\n",
       "      <td>Vyacheslav Sketch</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>spotify:artist:007OEblSdHKZolLJZwHPi2</td>\n",
       "      <td>Baby Sleep Ensemble</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>spotify:artist:007jdNDl62xieDnxfg4Q3c</td>\n",
       "      <td>Mucho Manolo</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>spotify:artist:007tQeSyOXgkqXeUNr22wB</td>\n",
       "      <td>Steven Clawson</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>spotify:artist:008ZQu59ackLc9pMyPwGE9</td>\n",
       "      <td>Essev Bar</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>spotify:artist:008oVJK83eqsWrEu1Ljm3W</td>\n",
       "      <td>The Haunting</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                               artist_uri  \\\n",
       "1   spotify:artist:0001cekkfdEBoMlwVQvpLg   \n",
       "4   spotify:artist:000DnGPNOsxvqb2YEHBePR   \n",
       "7   spotify:artist:000UxvYLQuybj6iVRRCAw1   \n",
       "8   spotify:artist:000YStE477SUgwTTYceARC   \n",
       "9   spotify:artist:000h2XLY65iWC9u5zgcL1M   \n",
       "10  spotify:artist:000spuc3oKgwYmfg5IE26s   \n",
       "11  spotify:artist:000xagx3GkcunHTFdB4ly0   \n",
       "12  spotify:artist:0011DpkmiLGXnibZ4oKGB5   \n",
       "13  spotify:artist:0012MbXnWE3SOtQdfg8CBb   \n",
       "15  spotify:artist:001NaPzWk3GKPMkuVNMdGd   \n",
       "18  spotify:artist:001XyVfgdcNbhtA5dIH07V   \n",
       "21  spotify:artist:001zgg6v3jBCLr1Qe25cC3   \n",
       "24  spotify:artist:002EU3CH8nAyVzWOS0RFrn   \n",
       "28  spotify:artist:002TRaMMHsPgv11qv0Oe4N   \n",
       "29  spotify:artist:002bMfvkA3TDMG7mNl2FQl   \n",
       "32  spotify:artist:002qS9Hyk2fS9Cc0Xe1eBU   \n",
       "35  spotify:artist:003BVdmFsDDbCyTKt32Bjl   \n",
       "36  spotify:artist:003KhzdYcJduhb6M8WlYAQ   \n",
       "38  spotify:artist:003PCFEqETGNDYCMy8JIvu   \n",
       "40  spotify:artist:003XB0vnFW8niBSEHkM82a   \n",
       "42  spotify:artist:003kJVejiby1fQgWdSy3dw   \n",
       "44  spotify:artist:003qxd7dVynp8vPbOdV6Ri   \n",
       "45  spotify:artist:003xAcE0iwqQKEqqdwbs7i   \n",
       "46  spotify:artist:0046tfXJIwmZXoclXKQO7a   \n",
       "48  spotify:artist:004COHk61u8a2GAbLeTpl1   \n",
       "56  spotify:artist:0054Ye49KrJfOOETeTdRog   \n",
       "58  spotify:artist:005A36KboUZ12FJMpO9GwF   \n",
       "59  spotify:artist:005IA5advoiAuGB0ZBzC2y   \n",
       "61  spotify:artist:005fY9ByADStuYjcIIN5bQ   \n",
       "68  spotify:artist:006Bvo4yIgIiLY7gAmuaaz   \n",
       "69  spotify:artist:006EQjXYKEcojXAkJiJwsZ   \n",
       "70  spotify:artist:006Emvooxohe5K7q31mb8d   \n",
       "72  spotify:artist:006KL09zdkklU6SjWxBE9W   \n",
       "74  spotify:artist:006OrdbRA5i2ArxeCm6iEr   \n",
       "75  spotify:artist:006ZADyabxpgKqMmh8W2yz   \n",
       "81  spotify:artist:007OEblSdHKZolLJZwHPi2   \n",
       "83  spotify:artist:007jdNDl62xieDnxfg4Q3c   \n",
       "87  spotify:artist:007tQeSyOXgkqXeUNr22wB   \n",
       "90  spotify:artist:008ZQu59ackLc9pMyPwGE9   \n",
       "93  spotify:artist:008oVJK83eqsWrEu1Ljm3W   \n",
       "\n",
       "                               artist_name  count  \n",
       "1                             Jordan Colle      1  \n",
       "4                                The Ruins      1  \n",
       "7                            Primera Etica      1  \n",
       "8                               Fiona Xsay      1  \n",
       "9                                  Kosmose      1  \n",
       "10                    Parliament Syndicate      1  \n",
       "11                                    Moxa      1  \n",
       "12                        Green and Yellow      1  \n",
       "13                            Olivia Cella      1  \n",
       "15                              Cuzzo Shay      1  \n",
       "18                             Pocketbooks      1  \n",
       "21                             Diggy Metro      1  \n",
       "24          Justine Dorsey & Kerris Dorsey      1  \n",
       "28                        Daan Kleijn Trio      1  \n",
       "29                             Twila Paris      1  \n",
       "32                     Little Distractions      1  \n",
       "35                               Knut Bell      1  \n",
       "36                                   Germz      1  \n",
       "38                                  B.O.U.      1  \n",
       "40                                     Mad      1  \n",
       "42                        Edmund Pendelton      1  \n",
       "44                           Audio Alterna      1  \n",
       "45                        Student Teachers      1  \n",
       "46      Various Artists: Karaoke - Ameritz      1  \n",
       "48                       Hans Felix Koller      1  \n",
       "56  Chris McGregor's Brotherhood Of Breath      1  \n",
       "58                 The Tyrant Lizard Kings      1  \n",
       "59                 Bill Curtis and Friends      1  \n",
       "61                           Johann Brandl      1  \n",
       "68                            L'elan Vital      1  \n",
       "69                             Restriction      1  \n",
       "70                                    LeLe      1  \n",
       "72                            Drum Origins      1  \n",
       "74                                  JSKEEN      1  \n",
       "75                       Vyacheslav Sketch      1  \n",
       "81                     Baby Sleep Ensemble      1  \n",
       "83                            Mucho Manolo      1  \n",
       "87                          Steven Clawson      1  \n",
       "90                               Essev Bar      1  \n",
       "93                            The Haunting      1  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists.sort_values(\"count\", ascending=True).head(40)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can query our favorite artists by their name to see how many unique playlists contain them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 708 ms, sys: 28 ms, total: 736 ms\n",
      "Wall time: 749 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>117508</th>\n",
       "      <td>spotify:artist:36QJpDe2go2KgaRleHCDTp</td>\n",
       "      <td>Led Zeppelin</td>\n",
       "      <td>29498</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   artist_uri   artist_name  count\n",
       "117508  spotify:artist:36QJpDe2go2KgaRleHCDTp  Led Zeppelin  29498"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists[artists.artist_name == \"Led Zeppelin\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many unique artists in the entire dataset?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 14.1 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(295856, 3)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a lot of artists and, from our bottom-40 dataframe above, it looks like there's likely many artists that occur so infrequently that they are unlikely to have any predictive qualities.\n",
    "\n",
    "We can get a good feel for the actual distribution of artists and the number of playlists that contain them by log transforming our counts. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 72 ms, sys: 0 ns, total: 72 ms\n",
      "Wall time: 72.8 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists[\"log_count\"] = np.log1p(artists[\"count\"].to_pandas())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's not surprising that this distribution has a very long tail. \n",
    "\n",
    "This tells us most of the artists don't occur in many playlists at all. It's likely these are aspiring / bedroom musicians and local artists. Unfortunately, they aren't going to help us predict music for playlists and related artists if we don't have enough data about them to know who their most similar to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 24 ms, sys: 4 ms, total: 28 ms\n",
      "Wall time: 27.2 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7ff6d0630048>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD4CAYAAADy46FuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAagUlEQVR4nO3df4xd5Z3f8fenOJs6JBADYeT1eDuOcLIL9q5Tjxy3qNG03thuEsVkBc0gFxvhahIEbVJZauzdlYhArqBbQot2ceSsXRtKwciExVrwkinkil0JDIagGENYD8ELE7t4g13jSRbKON/+cZ5Jzgx3nhnfO3Pv3JvPS7q6537Pec48X7D88flx5ygiMDMzG88/avYEzMxsZnNQmJlZloPCzMyyHBRmZpbloDAzs6xZzZ7AVLvooouiq6ur2dOo289+9jPOPffcZk9jSrmn1tBuPbVbPzA9PT333HM/jYiPVVvXdkHR1dXFgQMHmj2NulUqFXp6epo9jSnlnlpDu/XUbv3A9PQk6e/GW+dTT2ZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyJgwKSfMlfV/Sy5IOSfpaql8gqV/S4fQ+pzRms6QBSa9IWlWqL5V0MK27U5JS/YOSdqf6fkldpTHr0884LGn9VDZvZmYTm8w3s4eBjRHxvKSPAM9J6geuBR6PiFslbQI2Ad+QdCnQC1wG/CbwvyV9IiLOAFuBPuBp4FFgNbAP2ACcjIhLJPUCtwFflnQBcBPQDUT62Xsj4uRU/QcYq2vTI9O166wjt36+KT/XzGwiEx5RRMSxiHg+LZ8GXgbmAWuAXWmzXcAVaXkNcH9EvBsRrwEDwDJJc4HzIuKpKB6rd/eYMSP72gOsSEcbq4D+iDiRwqGfIlzMzKxBzup3PaVTQp8C9gMdEXEMijCRdHHabB7FEcOIwVR7Ly2PrY+MeSPta1jSKeDCcr3KmPK8+iiOVOjo6KBSqZxNW6NsXDxc89h6jJ3z0NBQXX3MRO6pNbRbT+3WDzS+p0kHhaQPAw8CX4+It9PlhaqbVqlFpl7rmF8VIrYB2wC6u7ujnl+WdW2zTj2t7Rn12b/IrDW4p5mv3fqBxvc0qbueJH2AIiTujYjvpvKb6XQS6f14qg8C80vDO4Gjqd5ZpT5qjKRZwPnAicy+zMysQSZz15OA7cDLEfGt0qq9wMhdSOuBh0v13nQn0wJgIfBMOk11WtLytM91Y8aM7OtK4Il0HeMxYKWkOemuqpWpZmZmDTKZU0+XA9cAByW9kGp/CNwKPCBpA/A6cBVARByS9ADwEsUdUzekO54Argd2ArMp7nbal+rbgXskDVAcSfSmfZ2QdAvwbNru5og4UWOvZmZWgwmDIiL+hurXCgBWjDNmC7ClSv0AsKhK/R1S0FRZtwPYMdE8zcxsevib2WZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaWNZlnZu+QdFzSi6XabkkvpNeRkUekSuqS9A+ldd8ujVkq6aCkAUl3pudmk56tvTvV90vqKo1ZL+lweq3HzMwabjLPzN4J/Clw90ghIr48sizpduBUaftXI2JJlf1sBfqAp4FHgdUUz8zeAJyMiEsk9QK3AV+WdAFwE9ANBPCcpL0RcXLy7ZmZWb0mPKKIiCeBE9XWpaOCfwPcl9uHpLnAeRHxVEQERehckVavAXal5T3AirTfVUB/RJxI4dBPES5mZtZAkzmiyPkXwJsRcbhUWyDpB8DbwB9HxF8D84DB0jaDqUZ6fwMgIoYlnQIuLNerjBlFUh/F0QodHR1UKpWaG9q4eLjmsfUYO+ehoaG6+piJ3FNraLee2q0faHxP9QbF1Yw+mjgG/FZEvCVpKfAXki4DVGVspPfx1uXGjC5GbAO2AXR3d0dPT8/kZl/FtZseqXlsPY6s7Rn1uVKpUE8fM5F7ag3t1lO79QON76nmu54kzQL+ANg9UouIdyPirbT8HPAq8AmKo4HO0vBO4GhaHgTml/Z5PsWprl/Wq4wxM7MGqef22N8HfhQRvzylJOljks5Jyx8HFgI/johjwGlJy9P1h3XAw2nYXmDkjqYrgSfSdYzHgJWS5kiaA6xMNTMza6AJTz1Jug/oAS6SNAjcFBHbgV7efxH7M8DNkoaBM8BXI2LkQvj1FHdQzaa422lfqm8H7pE0QHEk0QsQESck3QI8m7a7ubQvMzNrkAmDIiKuHqd+bZXag8CD42x/AFhUpf4OcNU4Y3YAOyaao5mZTR9/M9vMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWdaEQSFph6Tjkl4s1b4p6SeSXkivz5XWbZY0IOkVSatK9aWSDqZ1d6ZHoiLpg5J2p/p+SV2lMeslHU6vkcelmplZA03miGInsLpK/Y6IWJJejwJIupTiUaaXpTF3jTxDG9gK9FE8R3thaZ8bgJMRcQlwB3Bb2tcFwE3Ap4FlwE3p2dlmZtZAEwZFRDxJ8SzryVgD3B8R70bEa8AAsEzSXOC8iHgqIgK4G7iiNGZXWt4DrEhHG6uA/og4EREngX6qB5aZmU2jCZ+ZnXGjpHXAAWBj+st8HvB0aZvBVHsvLY+tk97fAIiIYUmngAvL9SpjRpHUR3G0QkdHB5VKpeamNi4ernlsPcbOeWhoqK4+ZiL31Brarad26wca31OtQbEVuAWI9H47cB2gKttGpk6NY0YXI7YB2wC6u7ujp6cnM/W8azc9UvPYehxZ2zPqc6VSoZ4+ZiL31Brarad26wca31NNdz1FxJsRcSYifgF8h+IaAhT/6p9f2rQTOJrqnVXqo8ZImgWcT3Gqa7x9mZlZA9UUFOmaw4gvASN3RO0FetOdTAsoLlo/ExHHgNOSlqfrD+uAh0tjRu5ouhJ4Il3HeAxYKWlOuoi9MtXMzKyBJjz1JOk+oAe4SNIgxZ1IPZKWUJwKOgJ8BSAiDkl6AHgJGAZuiIgzaVfXU9xBNRvYl14A24F7JA1QHEn0pn2dkHQL8Gza7uaImOxFdTMzmyITBkVEXF2lvD2z/RZgS5X6AWBRlfo7wFXj7GsHsGOiOZqZ2fTxN7PNzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLGvCoJC0Q9JxSS+Wan8i6UeSfijpIUkfTfUuSf8g6YX0+nZpzFJJByUNSLozPTub9Hzt3am+X1JXacx6SYfTaz1mZtZwkzmi2AmsHlPrBxZFxO8CfwtsLq17NSKWpNdXS/WtQB+wML1G9rkBOBkRlwB3ALcBSLqA4vncnwaWATdJmnMWvZmZ2RSYMCgi4kngxJja9yJiOH18GujM7UPSXOC8iHgqIgK4G7girV4D7ErLe4AV6WhjFdAfESci4iRFOI0NLDMzm2azpmAf1wG7S58XSPoB8DbwxxHx18A8YLC0zWCqkd7fAIiIYUmngAvL9SpjRpHUR3G0QkdHB5VKpeZmNi4ennijaTB2zkNDQ3X1MRO5p9bQbj21Wz/Q+J7qCgpJfwQMA/em0jHgtyLiLUlLgb+QdBmgKsNjZDfjrMuNGV2M2AZsA+ju7o6enp5J9zDWtZseqXlsPY6s7Rn1uVKpUE8fM5F7ag3t1lO79QON76nmu57SxeUvAGvT6SQi4t2IeCstPwe8CnyC4migfHqqEzialgeB+Wmfs4DzKU51/bJeZYyZmTVITUEhaTXwDeCLEfHzUv1jks5Jyx+nuGj944g4BpyWtDxdf1gHPJyG7QVG7mi6EngiBc9jwEpJc9JF7JWpZmZmDTThqSdJ9wE9wEWSBinuRNoMfBDoT3e5Pp3ucPoMcLOkYeAM8NWIGLkQfj3FHVSzgX3pBbAduEfSAMWRRC9ARJyQdAvwbNru5tK+zMysQSYMioi4ukp5+zjbPgg8OM66A8CiKvV3gKvGGbMD2DHRHM3MbPr4m9lmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLImDApJOyQdl/RiqXaBpH5Jh9P7nNK6zZIGJL0iaVWpvlTSwbTuzvRIVCR9UNLuVN8vqas0Zn36GYfTM7rNzKzBJnNEsRNYPaa2CXg8IhYCj6fPSLqU4lGml6Uxd408QxvYCvRRPEd7YWmfG4CTEXEJcAdwW9rXBRSPXf00sAy4qRxIZmbWGBMGRUQ8SfEs67I1wK60vAu4olS/PyLejYjXgAFgmaS5wHkR8VREBHD3mDEj+9oDrEhHG6uA/og4EREngX7eH1hmZjbNar1G0RERxwDS+8WpPg94o7TdYKrNS8tj66PGRMQwcAq4MLMvMzNroFlTvD9VqUWmXuuY0T9U6qM4rUVHRweVSmXCiY5n4+LhmsfWY+ych4aG6upjJnJPraHdemq3fqDxPdUaFG9KmhsRx9JppeOpPgjML23XCRxN9c4q9fKYQUmzgPMpTnUNAj1jxlSqTSYitgHbALq7u6Onp6faZpNy7aZHah5bjyNre0Z9rlQq1NPHTOSeWkO79dRu/UDje6r11NNeYOQupPXAw6V6b7qTaQHFRetn0ump05KWp+sP68aMGdnXlcAT6TrGY8BKSXPSReyVqWZmZg004RGFpPso/mV/kaRBijuRbgUekLQBeB24CiAiDkl6AHgJGAZuiIgzaVfXU9xBNRvYl14A24F7JA1QHEn0pn2dkHQL8Gza7uaIGHtR3czMptmEQRERV4+zasU4228BtlSpHwAWVam/QwqaKut2ADsmmqOZmU0ffzPbzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLJqDgpJn5T0Qun1tqSvS/qmpJ+U6p8rjdksaUDSK5JWlepLJR1M6+5Mz9UmPXt7d6rvl9RVT7NmZnb2ag6KiHglIpZExBJgKfBz4KG0+o6RdRHxKICkSymeh30ZsBq4S9I5afutQB+wML1Wp/oG4GREXALcAdxW63zNzKw2U3XqaQXwakT8XWabNcD9EfFuRLwGDADLJM0FzouIpyIigLuBK0pjdqXlPcCKkaMNMzNrjFlTtJ9e4L7S5xslrQMOABsj4iQwD3i6tM1gqr2XlsfWSe9vAETEsKRTwIXAT8s/XFIfxREJHR0dVCqVmhvZuHi45rH1GDvnoaGhuvqYidxTa2i3ntqtH2h8T3UHhaTfAL4IbE6lrcAtQKT324HrgGpHApGpM8G6XxUitgHbALq7u6Onp2fyDYxx7aZHah5bjyNre0Z9rlQq1NPHTOSeWkO79dRu/UDje5qKU0//Gng+It4EiIg3I+JMRPwC+A6wLG03CMwvjesEjqZ6Z5X6qDGSZgHnAyemYM5mZjZJUxEUV1M67ZSuOYz4EvBiWt4L9KY7mRZQXLR+JiKOAaclLU/XH9YBD5fGrE/LVwJPpOsYZmbWIHWdepL0IeCzwFdK5f8iaQnFKaIjI+si4pCkB4CXgGHghog4k8ZcD+wEZgP70gtgO3CPpAGKI4neeuZrZmZnr66giIifU1xcLteuyWy/BdhSpX4AWFSl/g5wVT1zNDOz+vib2WZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMsuoKCklHJB2U9IKkA6l2gaR+SYfT+5zS9pslDUh6RdKqUn1p2s+ApDvTI1FJj03dner7JXXVM18zMzt7U3FE8S8jYklEdKfPm4DHI2Ih8Hj6jKRLKR5lehmwGrhL0jlpzFagj+I52gvTeoANwMmIuAS4A7htCuZrZmZnYTpOPa0BdqXlXcAVpfr9EfFuRLwGDADLJM0FzouIpyIigLvHjBnZ1x5gxcjRhpmZNUa9QRHA9yQ9J6kv1Toi4hhAer841ecBb5TGDqbavLQ8tj5qTEQMA6cY84xuMzObXrPqHH95RByVdDHQL+lHmW2rHQlEpp4bM3rHRUj1AXR0dFCpVLKTztm4eLjmsfUYO+ehoaG6+piJ3FNraLee2q0faHxPdQVFRBxN78clPQQsA96UNDcijqXTSsfT5oPA/NLwTuBoqndWqZfHDEqaBZwPnKgyj23ANoDu7u7o6empuadrNz1S89h6HFnbM+pzpVKhnj5mIvfUGtqtp3brBxrfU82nniSdK+kjI8vASuBFYC+wPm22Hng4Le8FetOdTAsoLlo/k05PnZa0PF1/WDdmzMi+rgSeSNcxzMysQeo5ougAHkrXlmcB/ysi/krSs8ADkjYArwNXAUTEIUkPAC8Bw8ANEXEm7et6YCcwG9iXXgDbgXskDVAcSfTWMV8zM6tBzUERET8Gfq9K/S1gxThjtgBbqtQPAIuq1N8hBY2ZmTWHv5ltZmZZ9d71ZFOka8xF9I2Lhxt2Yf3IrZ9vyM8xs9bkIwozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCyrnmdmz5f0fUkvSzok6Wup/k1JP5H0Qnp9rjRms6QBSa9IWlWqL5V0MK27Mz07m/R87d2pvl9SV+2tmplZLeo5ohgGNkbE7wDLgRskXZrW3RERS9LrUYC0rhe4DFgN3CXpnLT9VqAPWJheq1N9A3AyIi4B7gBuq2O+ZmZWg5qDIiKORcTzafk08DIwLzNkDXB/RLwbEa8BA8AySXOB8yLiqYgI4G7gitKYXWl5D7Bi5GjDzMwaY0oehZpOCX0K2A9cDtwoaR1wgOKo4yRFiDxdGjaYau+l5bF10vsbABExLOkUcCHw0zE/v4/iiISOjg4qlUrNvWxcPFzz2KnUMbtxc6nnv9fZGBoaatjPahT3NPO1Wz/Q+J7qDgpJHwYeBL4eEW9L2grcAkR6vx24Dqh2JBCZOhOs+1UhYhuwDaC7uzt6enrOsotfadRzqieycfEwtx9szCPNj6ztacjPqVQq1PP/ZiZyTzNfu/UDje+prr+JJH2AIiTujYjvAkTEm6X13wH+Mn0cBOaXhncCR1O9s0q9PGZQ0izgfOBEPXO29+tqUDhuXDw8KoiP3Pr5hvxcM6tPPXc9CdgOvBwR3yrV55Y2+xLwYlreC/SmO5kWUFy0fiYijgGnJS1P+1wHPFwasz4tXwk8ka5jmJlZg9RzRHE5cA1wUNILqfaHwNWSllCcIjoCfAUgIg5JegB4ieKOqRsi4kwadz2wE5gN7EsvKILoHkkDFEcSvXXM18zMalBzUETE31D9GsKjmTFbgC1V6geARVXq7wBX1TpHMzOrn7+ZbWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzrMb8jgizKhr1jfCx/I1ws7PjIwozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWX59lj7tTOVt+WOfRjTRHxrrrUiH1GYmVmWg8LMzLJ86smsgfxtdGtFLXFEIWm1pFckDUja1Oz5mJn9OpnxRxSSzgH+DPgsMAg8K2lvRLzU3JmZtY6zOZI52wv0E/HRTOub8UEBLAMGIuLHAJLuB9YADgqzFtCs020jpjr4JqPdwlER0ew5ZEm6ElgdEf8ufb4G+HRE3Fjapg/oSx8/CbzS8IlOvYuAnzZ7ElPMPbWGduup3fqB6enpn0TEx6qtaIUjClWpjUq3iNgGbGvMdBpD0oGI6G72PKaSe2oN7dZTu/UDje+pFS5mDwLzS587gaNNmouZ2a+dVgiKZ4GFkhZI+g2gF9jb5DmZmf3amPGnniJiWNKNwGPAOcCOiDjU5Gk1QludSkvcU2tot57arR9ocE8z/mK2mZk1VyucejIzsyZyUJiZWZaDYoaRNF/S9yW9LOmQpK81e05TQdI5kn4g6S+bPZepIOmjkvZI+lH6f/XPmj2nekn6j+nP3IuS7pP0j5s9p7MlaYek45JeLNUukNQv6XB6n9PMOZ6tcXr6k/Rn74eSHpL00emcg4Ni5hkGNkbE7wDLgRskXdrkOU2FrwEvN3sSU+i/A38VEb8N/B4t3pukecB/ALojYhHFjSO9zZ1VTXYCq8fUNgGPR8RC4PH0uZXs5P099QOLIuJ3gb8FNk/nBBwUM0xEHIuI59PyaYq/gOY1d1b1kdQJfB7482bPZSpIOg/4DLAdICL+X0T83+bOakrMAmZLmgV8iBb8vlJEPAmcGFNeA+xKy7uAKxo6qTpV6ykivhcRw+nj0xTfL5s2DooZTFIX8Clgf3NnUrf/Bvwn4BfNnsgU+Tjw98D/SKfT/lzSuc2eVD0i4ifAfwVeB44BpyLie82d1ZTpiIhjUPxDDLi4yfOZatcB+6bzBzgoZihJHwYeBL4eEW83ez61kvQF4HhEPNfsuUyhWcA/BbZGxKeAn9F6pzNGSeft1wALgN8EzpX0b5s7K5uIpD+iOF1973T+HAfFDCTpAxQhcW9EfLfZ86nT5cAXJR0B7gf+laT/2dwp1W0QGIyIkSO9PRTB0cp+H3gtIv4+It4Dvgv88ybPaaq8KWkuQHo/3uT5TAlJ64EvAGtjmr8Q56CYYSSJ4tz3yxHxrWbPp14RsTkiOiOii+Li6BMR0dL/Uo2I/wO8IemTqbSC1v+1968DyyV9KP0ZXEGLX6Av2QusT8vrgYebOJcpIWk18A3gixHx8+n+eQ6Kmedy4BqKf3m/kF6fa/ak7H3+PXCvpB8CS4D/3OT51CUdHe0BngcOUvzd0HK/+kLSfcBTwCclDUraANwKfFbSYYoHoN3azDmerXF6+lPgI0B/+jvi29M6B/8KDzMzy/ERhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW9f8BG9WGw0iOB0kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%time\n",
    "artists[\"log_count\"].to_pandas().hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's count exactly how many artists only occur in only a single playlist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 32 ms, sys: 0 ns, total: 32 ms\n",
      "Wall time: 37.4 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(119746, 4)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists[artists[\"count\"]==1].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reduce Artists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can investigate these counts to find a good threshold that we can use to filter our dataset down a little bit.\n",
    "\n",
    "By filtering for artists that occur in 50 or more playlists, we could whittle down our list of artists by nearly 8x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 20 ms, sys: 8 ms, total: 28 ms\n",
      "Wall time: 35 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(29880, 4)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists[artists[\"count\"]>=50].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 32 ms, sys: 0 ns, total: 32 ms\n",
      "Wall time: 30.5 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "9.901472556894243"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists.shape[0] / artists[artists[\"count\"]>=50].shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try a larger number "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 28 ms, sys: 0 ns, total: 28 ms\n",
      "Wall time: 33.5 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(4596, 4)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "artists[artists[\"count\"]>1000].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like 97% of the artists were in less than 1 thousand playlists. This seems like a reasonable noise filter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 28 ms, sys: 0 ns, total: 28 ms\n",
      "Wall time: 28.8 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.9844654156075929"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "1 - (artists[artists[\"count\"]>1000].shape[0] / artists.shape[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 20 ms, sys: 8 ms, total: 28 ms\n",
      "Wall time: 29.3 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists = artists[artists[\"count\"]>1000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still, none of these artists are familiar but `1K` out of `1M` playlists is `0.1%` and that seems like a reasonable enough popularity to keep them. Also, if you have heard of these artists then you could lower the threshold even more. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>count</th>\n",
       "      <th>log_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>66929</th>\n",
       "      <td>spotify:artist:1lQO7ePBTp5yOfVK9hicoJ</td>\n",
       "      <td>Fidel Rueda</td>\n",
       "      <td>1001</td>\n",
       "      <td>6.909753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156206</th>\n",
       "      <td>spotify:artist:47opbYEKDjXnRk9uLscp11</td>\n",
       "      <td>Yolanda Adams</td>\n",
       "      <td>1001</td>\n",
       "      <td>6.909753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>284020</th>\n",
       "      <td>spotify:artist:7gkRNHOOt7QfhhXf0rEnmj</td>\n",
       "      <td>+44</td>\n",
       "      <td>1001</td>\n",
       "      <td>6.909753</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20797</th>\n",
       "      <td>spotify:artist:0Y619QjWhmBbSV5LGZN7oi</td>\n",
       "      <td>The Real Tune</td>\n",
       "      <td>1002</td>\n",
       "      <td>6.910751</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>226941</th>\n",
       "      <td>spotify:artist:5yCd7bxcAc3MdQ1h54ESsD</td>\n",
       "      <td>Houston</td>\n",
       "      <td>1002</td>\n",
       "      <td>6.910751</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   artist_uri    artist_name  count  log_count\n",
       "66929   spotify:artist:1lQO7ePBTp5yOfVK9hicoJ    Fidel Rueda   1001   6.909753\n",
       "156206  spotify:artist:47opbYEKDjXnRk9uLscp11  Yolanda Adams   1001   6.909753\n",
       "284020  spotify:artist:7gkRNHOOt7QfhhXf0rEnmj            +44   1001   6.909753\n",
       "20797   spotify:artist:0Y619QjWhmBbSV5LGZN7oi  The Real Tune   1002   6.910751\n",
       "226941  spotify:artist:5yCd7bxcAc3MdQ1h54ESsD        Houston   1002   6.910751"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "artists.sort_values(\"count\", ascending=True).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 995 µs\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "artists = artists[[\"artist_uri\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filter Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can filter the full dataset down using a left inner join on the `artist_uri`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 3min 1s, sys: 1min 24s, total: 4min 25s\n",
      "Wall time: 4min 25s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "gdf = artists.merge(gdf, on=\"artist_uri\", how=\"left\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>description</th>\n",
       "      <th>name</th>\n",
       "      <th>num_followers</th>\n",
       "      <th>pid</th>\n",
       "      <th>pos</th>\n",
       "      <th>track_name</th>\n",
       "      <th>track_uri</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>spotify:artist:7sgWBYtJpblXpJl2lU5WVs</td>\n",
       "      <td>Electric Guest</td>\n",
       "      <td>null</td>\n",
       "      <td>WORKOUT</td>\n",
       "      <td>3</td>\n",
       "      <td>814479</td>\n",
       "      <td>81</td>\n",
       "      <td>Oh Devil</td>\n",
       "      <td>spotify:track:1DnJlbiELCQcZJwywwBOKC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>spotify:artist:7siPLyFwRFYQkKgWKJ5Sod</td>\n",
       "      <td>The Dandy Warhols</td>\n",
       "      <td>null</td>\n",
       "      <td>Test</td>\n",
       "      <td>1</td>\n",
       "      <td>814288</td>\n",
       "      <td>29</td>\n",
       "      <td>The Last High</td>\n",
       "      <td>spotify:track:4jWtqHOLwFoXMsXtlnBK7v</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spotify:artist:7slfeZO9LsJbWgpkIoXBUJ</td>\n",
       "      <td>Ricky Martin</td>\n",
       "      <td>null</td>\n",
       "      <td>Soft</td>\n",
       "      <td>2</td>\n",
       "      <td>814708</td>\n",
       "      <td>56</td>\n",
       "      <td>El Ultimo Adiós - Varios Artistas Version</td>\n",
       "      <td>spotify:track:0tzixmHNQfE6S6SirSToxW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>spotify:artist:7so1HCxGkBmfqdvIF5HzkT</td>\n",
       "      <td>Lawrence Taylor</td>\n",
       "      <td>null</td>\n",
       "      <td>VIBES</td>\n",
       "      <td>1</td>\n",
       "      <td>814686</td>\n",
       "      <td>16</td>\n",
       "      <td>Bang Bang</td>\n",
       "      <td>spotify:track:3lVRPH9P6BpD9lamNSyrp2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>spotify:artist:7t0rwkOPGlDPEhaOcVtOt9</td>\n",
       "      <td>The Cranberries</td>\n",
       "      <td>null</td>\n",
       "      <td>80</td>\n",
       "      <td>1</td>\n",
       "      <td>814689</td>\n",
       "      <td>26</td>\n",
       "      <td>Linger</td>\n",
       "      <td>spotify:track:1aLNpg2kKvUncflV8xE5AA</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              artist_uri        artist_name description  \\\n",
       "0  spotify:artist:7sgWBYtJpblXpJl2lU5WVs     Electric Guest        null   \n",
       "1  spotify:artist:7siPLyFwRFYQkKgWKJ5Sod  The Dandy Warhols        null   \n",
       "2  spotify:artist:7slfeZO9LsJbWgpkIoXBUJ       Ricky Martin        null   \n",
       "3  spotify:artist:7so1HCxGkBmfqdvIF5HzkT    Lawrence Taylor        null   \n",
       "4  spotify:artist:7t0rwkOPGlDPEhaOcVtOt9    The Cranberries        null   \n",
       "\n",
       "      name  num_followers     pid  pos  \\\n",
       "0  WORKOUT              3  814479   81   \n",
       "1     Test              1  814288   29   \n",
       "2    Soft               2  814708   56   \n",
       "3    VIBES              1  814686   16   \n",
       "4       80              1  814689   26   \n",
       "\n",
       "                                  track_name  \\\n",
       "0                                   Oh Devil   \n",
       "1                              The Last High   \n",
       "2  El Ultimo Adiós - Varios Artistas Version   \n",
       "3                                  Bang Bang   \n",
       "4                                     Linger   \n",
       "\n",
       "                              track_uri  \n",
       "0  spotify:track:1DnJlbiELCQcZJwywwBOKC  \n",
       "1  spotify:track:4jWtqHOLwFoXMsXtlnBK7v  \n",
       "2  spotify:track:0tzixmHNQfE6S6SirSToxW  \n",
       "3  spotify:track:3lVRPH9P6BpD9lamNSyrp2  \n",
       "4  spotify:track:1aLNpg2kKvUncflV8xE5AA  "
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56103086, 9)"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build Link Predictor for Related Artists\n",
    "\n",
    "Now that we've reduced our dataset, we can begin to create a small link predictor to find artists related to a given playlist. \n",
    "\n",
    "We can use cuGraph's `Jaccard similarity` analytic to help us with the link prediction.\n",
    "\n",
    "\n",
    "**Problem:** Given a playlist, we want to see if we can find similar artists by looking at other similar playlists. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Construct Bipartite Playlist/Artist Graph\n",
    "\n",
    "To save memory, we can select only the columns we need for our graph. \n",
    "\n",
    "The fields `artist_uri` and `pid` will become our `src` and `dst` edge lists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 581 µs\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "graph_gdf = gdf[[\"artist_uri\", \"pid\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>pid</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>spotify:artist:7sgWBYtJpblXpJl2lU5WVs</td>\n",
       "      <td>814479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>spotify:artist:7siPLyFwRFYQkKgWKJ5Sod</td>\n",
       "      <td>814288</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spotify:artist:7slfeZO9LsJbWgpkIoXBUJ</td>\n",
       "      <td>814708</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>spotify:artist:7so1HCxGkBmfqdvIF5HzkT</td>\n",
       "      <td>814686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>spotify:artist:7t0rwkOPGlDPEhaOcVtOt9</td>\n",
       "      <td>814689</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                              artist_uri     pid\n",
       "0  spotify:artist:7sgWBYtJpblXpJl2lU5WVs  814479\n",
       "1  spotify:artist:7siPLyFwRFYQkKgWKJ5Sod  814288\n",
       "2  spotify:artist:7slfeZO9LsJbWgpkIoXBUJ  814708\n",
       "3  spotify:artist:7so1HCxGkBmfqdvIF5HzkT  814686\n",
       "4  spotify:artist:7t0rwkOPGlDPEhaOcVtOt9  814689"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_gdf.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Duplicate edges in the graph aren't needed, so we can drop them to save even more space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.45 s, sys: 1.29 s, total: 3.74 s\n",
      "Wall time: 3.74 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "graph_gdf = graph_gdf.drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(31279849, 2)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_gdf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cuGraph will expect us to provide vertices in the form of 32-bit integers. We can use cuML's `LabelEncoder` to do this transformation on the GPU.\n",
    "\n",
    "Since we need to treat our `artist_uri` and `pid` vertices are different types, we will need to make sure their encoded categories won't collide. This can be accomplished by adding a simple offset to the `artist_uri`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 524 ms, sys: 252 ms, total: 776 ms\n",
      "Wall time: 783 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "le = cuml.preprocessing.LabelEncoder()\n",
    "graph_gdf[\"artist_enc\"] = le.fit_transform(graph_gdf[\"artist_uri\"])+1000000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sanity check to make sure we have the expected number of artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 120 ms, sys: 44 ms, total: 164 ms\n",
      "Wall time: 162 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(4596,)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "graph_gdf[\"artist_enc\"].unique().shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many playlists & songs we have now. Still fairly close to `1M`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(988467,)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "graph_gdf.pid.unique().shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our bipartite graph needs to be undirected, however, it currently only has edges pointing from artists to playlists. \n",
    "\n",
    "We can symmetrize our graph on GPU by concatenating the playlist and artist edge lists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 1.41 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "graph_gdf.pid = graph_gdf.pid.astype(\"int32\")\n",
    "graph_gdf.artist_enc = graph_gdf.artist_enc.astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "src_edges = cudf.concat([graph_gdf.pid, graph_gdf.artist_enc]).reset_index()[\"pid\"]\n",
    "dst_edges = cudf.concat([graph_gdf.artist_enc, graph_gdf.pid]).reset_index()[\"artist_enc\"]\n",
    "weights = cudf.concat([cudf.Series(np.ones(graph_gdf.pid.shape[0])), cudf.Series(np.zeros(graph_gdf.pid.shape[0]))])\n",
    "\n",
    "new_graph_gdf = cudf.DataFrame()\n",
    "new_graph_gdf[\"src\"] = src_edges\n",
    "new_graph_gdf[\"dst\"] = dst_edges\n",
    "new_graph_gdf[\"weights\"] = weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we can renumber our vertices to make sure they start at zero and increase monotonically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 256 ms, sys: 104 ms, total: 360 ms\n",
      "Wall time: 361 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "new_graph_gdf[\"src_id\"], new_graph_gdf[\"dst_id\"], numbering = cugraph.renumber(src_edges, dst_edges)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have combine the artist and playlist id vertex numbers, we can be confident ids >= 1M are going to be artists. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>src</th>\n",
       "      <th>dst</th>\n",
       "      <th>weights</th>\n",
       "      <th>src_id</th>\n",
       "      <th>dst_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>16</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1952</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>33</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4021</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>97</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>11797</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>112</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13624</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>138</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16787</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   src      dst  weights  src_id  dst_id\n",
       "0   16  1000000      1.0    1952   84993\n",
       "1   33  1000000      1.0    4021   84993\n",
       "2   97  1000000      1.0   11797   84993\n",
       "3  112  1000000      1.0   13624   84993\n",
       "4  138  1000000      1.0   16787   84993"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_graph_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>src</th>\n",
       "      <th>dst</th>\n",
       "      <th>weights</th>\n",
       "      <th>src_id</th>\n",
       "      <th>dst_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>151718</th>\n",
       "      <td>0</td>\n",
       "      <td>1000021</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>87552</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>594415</th>\n",
       "      <td>0</td>\n",
       "      <td>1000084</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>95215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>893899</th>\n",
       "      <td>0</td>\n",
       "      <td>1000121</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>99720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1511321</th>\n",
       "      <td>0</td>\n",
       "      <td>1000224</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>112250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1803555</th>\n",
       "      <td>0</td>\n",
       "      <td>1000257</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>116259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2938040</th>\n",
       "      <td>0</td>\n",
       "      <td>1000402</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>133888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3625809</th>\n",
       "      <td>0</td>\n",
       "      <td>1000490</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>144601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3983760</th>\n",
       "      <td>0</td>\n",
       "      <td>1000564</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>153612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5200895</th>\n",
       "      <td>0</td>\n",
       "      <td>1000752</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>176462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6450644</th>\n",
       "      <td>0</td>\n",
       "      <td>1000929</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>197967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7622909</th>\n",
       "      <td>0</td>\n",
       "      <td>1001150</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>224831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7985279</th>\n",
       "      <td>0</td>\n",
       "      <td>1001193</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>230052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8252648</th>\n",
       "      <td>0</td>\n",
       "      <td>1001216</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>232856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8532032</th>\n",
       "      <td>0</td>\n",
       "      <td>1001251</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>237108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8947571</th>\n",
       "      <td>0</td>\n",
       "      <td>1001283</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>241004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9004058</th>\n",
       "      <td>0</td>\n",
       "      <td>1001290</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>241857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9622247</th>\n",
       "      <td>0</td>\n",
       "      <td>1001390</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>254021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11201324</th>\n",
       "      <td>0</td>\n",
       "      <td>1001641</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>284570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11827634</th>\n",
       "      <td>0</td>\n",
       "      <td>1001747</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>297460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12198520</th>\n",
       "      <td>0</td>\n",
       "      <td>1001809</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>305005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14634128</th>\n",
       "      <td>0</td>\n",
       "      <td>1002153</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>346852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16009679</th>\n",
       "      <td>0</td>\n",
       "      <td>1002348</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>370544</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17940562</th>\n",
       "      <td>0</td>\n",
       "      <td>1002663</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>408839</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20779054</th>\n",
       "      <td>0</td>\n",
       "      <td>1003090</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>460770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20883093</th>\n",
       "      <td>0</td>\n",
       "      <td>1003112</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>463457</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21343973</th>\n",
       "      <td>0</td>\n",
       "      <td>1003152</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>468315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21917332</th>\n",
       "      <td>0</td>\n",
       "      <td>1003241</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>479144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23143983</th>\n",
       "      <td>0</td>\n",
       "      <td>1003415</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>500317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23290293</th>\n",
       "      <td>0</td>\n",
       "      <td>1003445</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>503958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24553448</th>\n",
       "      <td>0</td>\n",
       "      <td>1003642</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>527920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25019780</th>\n",
       "      <td>0</td>\n",
       "      <td>1003707</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>535820</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25460246</th>\n",
       "      <td>0</td>\n",
       "      <td>1003763</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>542627</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27537066</th>\n",
       "      <td>0</td>\n",
       "      <td>1004070</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>579932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27729732</th>\n",
       "      <td>0</td>\n",
       "      <td>1004088</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>582116</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29500028</th>\n",
       "      <td>0</td>\n",
       "      <td>1004347</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>613599</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29903383</th>\n",
       "      <td>0</td>\n",
       "      <td>1004399</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>619907</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31023098</th>\n",
       "      <td>0</td>\n",
       "      <td>1004540</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>637042</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          src      dst  weights  src_id  dst_id\n",
       "151718      0  1000021      1.0       0   87552\n",
       "594415      0  1000084      1.0       0   95215\n",
       "893899      0  1000121      1.0       0   99720\n",
       "1511321     0  1000224      1.0       0  112250\n",
       "1803555     0  1000257      1.0       0  116259\n",
       "2938040     0  1000402      1.0       0  133888\n",
       "3625809     0  1000490      1.0       0  144601\n",
       "3983760     0  1000564      1.0       0  153612\n",
       "5200895     0  1000752      1.0       0  176462\n",
       "6450644     0  1000929      1.0       0  197967\n",
       "7622909     0  1001150      1.0       0  224831\n",
       "7985279     0  1001193      1.0       0  230052\n",
       "8252648     0  1001216      1.0       0  232856\n",
       "8532032     0  1001251      1.0       0  237108\n",
       "8947571     0  1001283      1.0       0  241004\n",
       "9004058     0  1001290      1.0       0  241857\n",
       "9622247     0  1001390      1.0       0  254021\n",
       "11201324    0  1001641      1.0       0  284570\n",
       "11827634    0  1001747      1.0       0  297460\n",
       "12198520    0  1001809      1.0       0  305005\n",
       "14634128    0  1002153      1.0       0  346852\n",
       "16009679    0  1002348      1.0       0  370544\n",
       "17940562    0  1002663      1.0       0  408839\n",
       "20779054    0  1003090      1.0       0  460770\n",
       "20883093    0  1003112      1.0       0  463457\n",
       "21343973    0  1003152      1.0       0  468315\n",
       "21917332    0  1003241      1.0       0  479144\n",
       "23143983    0  1003415      1.0       0  500317\n",
       "23290293    0  1003445      1.0       0  503958\n",
       "24553448    0  1003642      1.0       0  527920\n",
       "25019780    0  1003707      1.0       0  535820\n",
       "25460246    0  1003763      1.0       0  542627\n",
       "27537066    0  1004070      1.0       0  579932\n",
       "27729732    0  1004088      1.0       0  582116\n",
       "29500028    0  1004347      1.0       0  613599\n",
       "29903383    0  1004399      1.0       0  619907\n",
       "31023098    0  1004540      1.0       0  637042"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_graph_gdf[new_graph_gdf.src_id==0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create our cuGraph using our edge lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 1.42 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "G = cugraph.Graph()\n",
    "G.add_edge_list(new_graph_gdf[\"src_id\"], new_graph_gdf[\"dst_id\"], new_graph_gdf[\"weights\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our graph is bipartite, we cannot just pass it into the Jaccard computation as-is. This would cause it only to traverse a single hop (eg from playlists to users and users to playlists) and would not find any intersections. \n",
    "\n",
    "We want to get the edge lists for the two-hop neighbors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.49 s, sys: 860 ms, total: 2.35 s\n",
      "Wall time: 2.34 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "two_hop = G.get_two_hop_neighbors()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 3.3 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>second</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>848487</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>848487</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>848487</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>848487</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>848487</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    first  second\n",
       "0  848487       0\n",
       "1  848487       1\n",
       "2  848487       3\n",
       "3  848487       4\n",
       "4  848487       6"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "two_hop.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first</th>\n",
       "      <th>second</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [first, second]\n",
       "Index: []"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "two_hop[two_hop.first==0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore Some Playlists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's explore a playlist by querying our original dataframe for a specific artist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12.5 s, sys: 5.89 s, total: 18.4 s\n",
      "Wall time: 18.4 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>artist_name</th>\n",
       "      <th>description</th>\n",
       "      <th>name</th>\n",
       "      <th>num_followers</th>\n",
       "      <th>pid</th>\n",
       "      <th>pos</th>\n",
       "      <th>track_name</th>\n",
       "      <th>track_uri</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>23821851</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>144</td>\n",
       "      <td>Teenagers</td>\n",
       "      <td>spotify:track:7j31rVgGX9Q2blT92VBEA0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821858</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>145</td>\n",
       "      <td>Helena (So Long &amp; Goodnight)</td>\n",
       "      <td>spotify:track:5dTHtzHFPyi8TlTtzoz1J9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821865</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>146</td>\n",
       "      <td>Famous Last Words</td>\n",
       "      <td>spotify:track:2d6m2F4I7wCuAKtSsdhh83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821873</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>147</td>\n",
       "      <td>Na Na Na [Na Na Na Na Na Na Na Na Na]</td>\n",
       "      <td>spotify:track:5BB0Jzw60KyfSTyjJqtely</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821881</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>148</td>\n",
       "      <td>Fake Your Death</td>\n",
       "      <td>spotify:track:2M8g01v2vUMn39YQ10l7ef</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821887</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>149</td>\n",
       "      <td>I'm Not Okay (I Promise)</td>\n",
       "      <td>spotify:track:7lRlq939cDG4SzWOF4VAnd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821895</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>161</td>\n",
       "      <td>House of Wolves</td>\n",
       "      <td>spotify:track:7j2Bmzpnf6RwEWEQ2sv8Ho</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821902</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Everything</td>\n",
       "      <td>9</td>\n",
       "      <td>231</td>\n",
       "      <td>143</td>\n",
       "      <td>Welcome to the Black Parade</td>\n",
       "      <td>spotify:track:5wQnmLuC1W7ATsArWACrgW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821917</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>songs that make me think of you</td>\n",
       "      <td>1</td>\n",
       "      <td>394</td>\n",
       "      <td>5</td>\n",
       "      <td>You Know What They Do To Guys Like Us In Prison</td>\n",
       "      <td>spotify:track:5bTuwAgYUD5MABOIPH8ZMt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831844</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>49</td>\n",
       "      <td>The Jetset Life Is Gonna Kill You</td>\n",
       "      <td>spotify:track:7HKfrU3iuEFrKxTAfVznrG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831853</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>50</td>\n",
       "      <td>Interlude</td>\n",
       "      <td>spotify:track:4vpaz6338VqJIeJHq6z2mU</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831863</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>51</td>\n",
       "      <td>Thank You For The Venom</td>\n",
       "      <td>spotify:track:0BpKPyXhWsTaWkgXeOd1Sn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831871</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>52</td>\n",
       "      <td>Hang 'Em High</td>\n",
       "      <td>spotify:track:262RJtiqytpugV1UvDicN9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831877</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>53</td>\n",
       "      <td>It's Not A Fashion Statement It's A Deathwish</td>\n",
       "      <td>spotify:track:4fGIqy3bvjX0jz1SJHqjjo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831885</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>54</td>\n",
       "      <td>Cemetery Drive</td>\n",
       "      <td>spotify:track:2cViIXIe8Pbd1sOJExMJlK</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831893</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>55</td>\n",
       "      <td>I Never Told You What I Do For A Living</td>\n",
       "      <td>spotify:track:6gLn8QhAYL4dEwdpVjfCPl</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831901</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>56</td>\n",
       "      <td>The End.</td>\n",
       "      <td>spotify:track:6s2yqZbiPNPL1B2IeCMsUA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831910</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>57</td>\n",
       "      <td>Dead!</td>\n",
       "      <td>spotify:track:0uukw2CgEIApv4IWAjXrBC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23841973</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>58</td>\n",
       "      <td>This Is How I Disappear</td>\n",
       "      <td>spotify:track:0rUNZQuYQvOz6A6zwyT6tM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23841981</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>59</td>\n",
       "      <td>The Sharpest Lives</td>\n",
       "      <td>spotify:track:2kMjk14RmYyYhhSbipoa9U</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23841990</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>60</td>\n",
       "      <td>Welcome to the Black Parade</td>\n",
       "      <td>spotify:track:5wQnmLuC1W7ATsArWACrgW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23841998</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>61</td>\n",
       "      <td>I Don't Love You</td>\n",
       "      <td>spotify:track:4RAOI1etsgbh5NP3T5R8rN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842007</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>62</td>\n",
       "      <td>House of Wolves</td>\n",
       "      <td>spotify:track:7j2Bmzpnf6RwEWEQ2sv8Ho</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842016</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>63</td>\n",
       "      <td>Cancer</td>\n",
       "      <td>spotify:track:0GgN4MhR5GKn5IcKN0e0rG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842022</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>64</td>\n",
       "      <td>Mama</td>\n",
       "      <td>spotify:track:0Zh3tKIphLOvQux4dA6PFZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842028</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>65</td>\n",
       "      <td>Sleep</td>\n",
       "      <td>spotify:track:2pONuBYwWrNm9jcJH6hmPc</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842034</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>66</td>\n",
       "      <td>Teenagers</td>\n",
       "      <td>spotify:track:7j31rVgGX9Q2blT92VBEA0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842041</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>67</td>\n",
       "      <td>Disenchanted</td>\n",
       "      <td>spotify:track:6T7MAQCekVb3UnCykjX3BP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842049</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>68</td>\n",
       "      <td>Famous Last Words</td>\n",
       "      <td>spotify:track:2d6m2F4I7wCuAKtSsdhh83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842057</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>69</td>\n",
       "      <td>Blood - Hidden Track</td>\n",
       "      <td>spotify:track:1rNSLiEFpAF4amVhkzfTUG</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23842066</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>43</td>\n",
       "      <td>Helena (So Long &amp; Goodnight)</td>\n",
       "      <td>spotify:track:5dTHtzHFPyi8TlTtzoz1J9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852466</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>44</td>\n",
       "      <td>Give 'Em Hell, Kid</td>\n",
       "      <td>spotify:track:0axF06XwNLobcx5z8BVtYw</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852473</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>45</td>\n",
       "      <td>To The End</td>\n",
       "      <td>spotify:track:1GErReAT0swX36x3O8GyQn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852480</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>46</td>\n",
       "      <td>You Know What They Do To Guys Like Us In Prison</td>\n",
       "      <td>spotify:track:5bTuwAgYUD5MABOIPH8ZMt</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852489</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>47</td>\n",
       "      <td>I'm Not Okay (I Promise)</td>\n",
       "      <td>spotify:track:7lRlq939cDG4SzWOF4VAnd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852497</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>⚡️</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>48</td>\n",
       "      <td>The Ghost Of You</td>\n",
       "      <td>spotify:track:1yKAqZoi8xWGLCf5vajroL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852504</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Metal</td>\n",
       "      <td>1</td>\n",
       "      <td>410</td>\n",
       "      <td>69</td>\n",
       "      <td>Welcome to the Black Parade</td>\n",
       "      <td>spotify:track:5wQnmLuC1W7ATsArWACrgW</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831835</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>yo</td>\n",
       "      <td>3</td>\n",
       "      <td>426</td>\n",
       "      <td>31</td>\n",
       "      <td>Teenagers</td>\n",
       "      <td>spotify:track:7j31rVgGX9Q2blT92VBEA0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821924</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>10</td>\n",
       "      <td>Planetary [GO!]</td>\n",
       "      <td>spotify:track:3iitOoZUtWEgwZOEYFITFo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821932</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>30</td>\n",
       "      <td>Mama</td>\n",
       "      <td>spotify:track:0Zh3tKIphLOvQux4dA6PFZ</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821939</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>6</td>\n",
       "      <td>Dead!</td>\n",
       "      <td>spotify:track:0uukw2CgEIApv4IWAjXrBC</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23821947</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>7</td>\n",
       "      <td>Vampire Money</td>\n",
       "      <td>spotify:track:4kXHIm097G3CemN3PzcpgL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831818</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>8</td>\n",
       "      <td>Party Poison</td>\n",
       "      <td>spotify:track:1RT6zSjs59gJdFfUshT465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23831826</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Driving</td>\n",
       "      <td>2</td>\n",
       "      <td>436</td>\n",
       "      <td>9</td>\n",
       "      <td>Na Na Na [Na Na Na Na Na Na Na Na Na]</td>\n",
       "      <td>spotify:track:5BB0Jzw60KyfSTyjJqtely</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852512</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>A little bit of everything</td>\n",
       "      <td>Melting Pot</td>\n",
       "      <td>2</td>\n",
       "      <td>491</td>\n",
       "      <td>108</td>\n",
       "      <td>Na Na Na [Na Na Na Na Na Na Na Na Na]</td>\n",
       "      <td>spotify:track:5BB0Jzw60KyfSTyjJqtely</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852520</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Sleep~</td>\n",
       "      <td>1</td>\n",
       "      <td>551</td>\n",
       "      <td>14</td>\n",
       "      <td>Helena (So Long &amp; Goodnight)</td>\n",
       "      <td>spotify:track:5dTHtzHFPyi8TlTtzoz1J9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852527</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Sleep~</td>\n",
       "      <td>1</td>\n",
       "      <td>551</td>\n",
       "      <td>26</td>\n",
       "      <td>The Ghost Of You</td>\n",
       "      <td>spotify:track:1yKAqZoi8xWGLCf5vajroL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852536</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Calming Music</td>\n",
       "      <td>2</td>\n",
       "      <td>573</td>\n",
       "      <td>61</td>\n",
       "      <td>The World Is Ugly</td>\n",
       "      <td>spotify:track:6VtcgrVYo2xfygcWAfRpd1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852544</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Calming Music</td>\n",
       "      <td>2</td>\n",
       "      <td>573</td>\n",
       "      <td>63</td>\n",
       "      <td>Early Sunsets Over Monroeville</td>\n",
       "      <td>spotify:track:2aHqdN4K9naX3oBuUxp6zf</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23852553</th>\n",
       "      <td>spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb</td>\n",
       "      <td>My Chemical Romance</td>\n",
       "      <td>null</td>\n",
       "      <td>Calming Music</td>\n",
       "      <td>2</td>\n",
       "      <td>573</td>\n",
       "      <td>60</td>\n",
       "      <td>The Light Behind Your Eyes</td>\n",
       "      <td>spotify:track:3HyDpKAuR3e4l6QB7hSB2l</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                     artist_uri          artist_name  \\\n",
       "23821851  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821858  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821865  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821873  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821881  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821887  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821895  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821902  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821917  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831844  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831853  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831863  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831871  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831877  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831885  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831893  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831901  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831910  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23841973  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23841981  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23841990  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23841998  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842007  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842016  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842022  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842028  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842034  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842041  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842049  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842057  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23842066  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852466  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852473  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852480  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852489  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852497  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852504  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831835  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821924  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821932  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821939  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23821947  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831818  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23831826  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852512  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852520  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852527  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852536  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852544  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "23852553  spotify:artist:7FBcuc1gsnv6Y1nwFtNRCb  My Chemical Romance   \n",
       "\n",
       "                         description                              name  \\\n",
       "23821851                        null                        Everything   \n",
       "23821858                        null                        Everything   \n",
       "23821865                        null                        Everything   \n",
       "23821873                        null                        Everything   \n",
       "23821881                        null                        Everything   \n",
       "23821887                        null                        Everything   \n",
       "23821895                        null                        Everything   \n",
       "23821902                        null                        Everything   \n",
       "23821917                        null  songs that make me think of you    \n",
       "23831844                        null                                ⚡️   \n",
       "23831853                        null                                ⚡️   \n",
       "23831863                        null                                ⚡️   \n",
       "23831871                        null                                ⚡️   \n",
       "23831877                        null                                ⚡️   \n",
       "23831885                        null                                ⚡️   \n",
       "23831893                        null                                ⚡️   \n",
       "23831901                        null                                ⚡️   \n",
       "23831910                        null                                ⚡️   \n",
       "23841973                        null                                ⚡️   \n",
       "23841981                        null                                ⚡️   \n",
       "23841990                        null                                ⚡️   \n",
       "23841998                        null                                ⚡️   \n",
       "23842007                        null                                ⚡️   \n",
       "23842016                        null                                ⚡️   \n",
       "23842022                        null                                ⚡️   \n",
       "23842028                        null                                ⚡️   \n",
       "23842034                        null                                ⚡️   \n",
       "23842041                        null                                ⚡️   \n",
       "23842049                        null                                ⚡️   \n",
       "23842057                        null                                ⚡️   \n",
       "23842066                        null                                ⚡️   \n",
       "23852466                        null                                ⚡️   \n",
       "23852473                        null                                ⚡️   \n",
       "23852480                        null                                ⚡️   \n",
       "23852489                        null                                ⚡️   \n",
       "23852497                        null                                ⚡️   \n",
       "23852504                        null                             Metal   \n",
       "23831835                        null                                yo   \n",
       "23821924                        null                           Driving   \n",
       "23821932                        null                           Driving   \n",
       "23821939                        null                           Driving   \n",
       "23821947                        null                           Driving   \n",
       "23831818                        null                           Driving   \n",
       "23831826                        null                           Driving   \n",
       "23852512  A little bit of everything                       Melting Pot   \n",
       "23852520                        null                            Sleep~   \n",
       "23852527                        null                            Sleep~   \n",
       "23852536                        null                     Calming Music   \n",
       "23852544                        null                     Calming Music   \n",
       "23852553                        null                     Calming Music   \n",
       "\n",
       "          num_followers  pid  pos  \\\n",
       "23821851              9  231  144   \n",
       "23821858              9  231  145   \n",
       "23821865              9  231  146   \n",
       "23821873              9  231  147   \n",
       "23821881              9  231  148   \n",
       "23821887              9  231  149   \n",
       "23821895              9  231  161   \n",
       "23821902              9  231  143   \n",
       "23821917              1  394    5   \n",
       "23831844              1  404   49   \n",
       "23831853              1  404   50   \n",
       "23831863              1  404   51   \n",
       "23831871              1  404   52   \n",
       "23831877              1  404   53   \n",
       "23831885              1  404   54   \n",
       "23831893              1  404   55   \n",
       "23831901              1  404   56   \n",
       "23831910              1  404   57   \n",
       "23841973              1  404   58   \n",
       "23841981              1  404   59   \n",
       "23841990              1  404   60   \n",
       "23841998              1  404   61   \n",
       "23842007              1  404   62   \n",
       "23842016              1  404   63   \n",
       "23842022              1  404   64   \n",
       "23842028              1  404   65   \n",
       "23842034              1  404   66   \n",
       "23842041              1  404   67   \n",
       "23842049              1  404   68   \n",
       "23842057              1  404   69   \n",
       "23842066              1  404   43   \n",
       "23852466              1  404   44   \n",
       "23852473              1  404   45   \n",
       "23852480              1  404   46   \n",
       "23852489              1  404   47   \n",
       "23852497              1  404   48   \n",
       "23852504              1  410   69   \n",
       "23831835              3  426   31   \n",
       "23821924              2  436   10   \n",
       "23821932              2  436   30   \n",
       "23821939              2  436    6   \n",
       "23821947              2  436    7   \n",
       "23831818              2  436    8   \n",
       "23831826              2  436    9   \n",
       "23852512              2  491  108   \n",
       "23852520              1  551   14   \n",
       "23852527              1  551   26   \n",
       "23852536              2  573   61   \n",
       "23852544              2  573   63   \n",
       "23852553              2  573   60   \n",
       "\n",
       "                                               track_name  \\\n",
       "23821851                                        Teenagers   \n",
       "23821858                     Helena (So Long & Goodnight)   \n",
       "23821865                                Famous Last Words   \n",
       "23821873            Na Na Na [Na Na Na Na Na Na Na Na Na]   \n",
       "23821881                                  Fake Your Death   \n",
       "23821887                         I'm Not Okay (I Promise)   \n",
       "23821895                                  House of Wolves   \n",
       "23821902                      Welcome to the Black Parade   \n",
       "23821917  You Know What They Do To Guys Like Us In Prison   \n",
       "23831844                The Jetset Life Is Gonna Kill You   \n",
       "23831853                                        Interlude   \n",
       "23831863                          Thank You For The Venom   \n",
       "23831871                                    Hang 'Em High   \n",
       "23831877    It's Not A Fashion Statement It's A Deathwish   \n",
       "23831885                                   Cemetery Drive   \n",
       "23831893          I Never Told You What I Do For A Living   \n",
       "23831901                                         The End.   \n",
       "23831910                                            Dead!   \n",
       "23841973                          This Is How I Disappear   \n",
       "23841981                               The Sharpest Lives   \n",
       "23841990                      Welcome to the Black Parade   \n",
       "23841998                                 I Don't Love You   \n",
       "23842007                                  House of Wolves   \n",
       "23842016                                           Cancer   \n",
       "23842022                                             Mama   \n",
       "23842028                                            Sleep   \n",
       "23842034                                        Teenagers   \n",
       "23842041                                     Disenchanted   \n",
       "23842049                                Famous Last Words   \n",
       "23842057                             Blood - Hidden Track   \n",
       "23842066                     Helena (So Long & Goodnight)   \n",
       "23852466                               Give 'Em Hell, Kid   \n",
       "23852473                                       To The End   \n",
       "23852480  You Know What They Do To Guys Like Us In Prison   \n",
       "23852489                         I'm Not Okay (I Promise)   \n",
       "23852497                                 The Ghost Of You   \n",
       "23852504                      Welcome to the Black Parade   \n",
       "23831835                                        Teenagers   \n",
       "23821924                                  Planetary [GO!]   \n",
       "23821932                                             Mama   \n",
       "23821939                                            Dead!   \n",
       "23821947                                    Vampire Money   \n",
       "23831818                                     Party Poison   \n",
       "23831826            Na Na Na [Na Na Na Na Na Na Na Na Na]   \n",
       "23852512            Na Na Na [Na Na Na Na Na Na Na Na Na]   \n",
       "23852520                     Helena (So Long & Goodnight)   \n",
       "23852527                                 The Ghost Of You   \n",
       "23852536                                The World Is Ugly   \n",
       "23852544                   Early Sunsets Over Monroeville   \n",
       "23852553                       The Light Behind Your Eyes   \n",
       "\n",
       "                                     track_uri  \n",
       "23821851  spotify:track:7j31rVgGX9Q2blT92VBEA0  \n",
       "23821858  spotify:track:5dTHtzHFPyi8TlTtzoz1J9  \n",
       "23821865  spotify:track:2d6m2F4I7wCuAKtSsdhh83  \n",
       "23821873  spotify:track:5BB0Jzw60KyfSTyjJqtely  \n",
       "23821881  spotify:track:2M8g01v2vUMn39YQ10l7ef  \n",
       "23821887  spotify:track:7lRlq939cDG4SzWOF4VAnd  \n",
       "23821895  spotify:track:7j2Bmzpnf6RwEWEQ2sv8Ho  \n",
       "23821902  spotify:track:5wQnmLuC1W7ATsArWACrgW  \n",
       "23821917  spotify:track:5bTuwAgYUD5MABOIPH8ZMt  \n",
       "23831844  spotify:track:7HKfrU3iuEFrKxTAfVznrG  \n",
       "23831853  spotify:track:4vpaz6338VqJIeJHq6z2mU  \n",
       "23831863  spotify:track:0BpKPyXhWsTaWkgXeOd1Sn  \n",
       "23831871  spotify:track:262RJtiqytpugV1UvDicN9  \n",
       "23831877  spotify:track:4fGIqy3bvjX0jz1SJHqjjo  \n",
       "23831885  spotify:track:2cViIXIe8Pbd1sOJExMJlK  \n",
       "23831893  spotify:track:6gLn8QhAYL4dEwdpVjfCPl  \n",
       "23831901  spotify:track:6s2yqZbiPNPL1B2IeCMsUA  \n",
       "23831910  spotify:track:0uukw2CgEIApv4IWAjXrBC  \n",
       "23841973  spotify:track:0rUNZQuYQvOz6A6zwyT6tM  \n",
       "23841981  spotify:track:2kMjk14RmYyYhhSbipoa9U  \n",
       "23841990  spotify:track:5wQnmLuC1W7ATsArWACrgW  \n",
       "23841998  spotify:track:4RAOI1etsgbh5NP3T5R8rN  \n",
       "23842007  spotify:track:7j2Bmzpnf6RwEWEQ2sv8Ho  \n",
       "23842016  spotify:track:0GgN4MhR5GKn5IcKN0e0rG  \n",
       "23842022  spotify:track:0Zh3tKIphLOvQux4dA6PFZ  \n",
       "23842028  spotify:track:2pONuBYwWrNm9jcJH6hmPc  \n",
       "23842034  spotify:track:7j31rVgGX9Q2blT92VBEA0  \n",
       "23842041  spotify:track:6T7MAQCekVb3UnCykjX3BP  \n",
       "23842049  spotify:track:2d6m2F4I7wCuAKtSsdhh83  \n",
       "23842057  spotify:track:1rNSLiEFpAF4amVhkzfTUG  \n",
       "23842066  spotify:track:5dTHtzHFPyi8TlTtzoz1J9  \n",
       "23852466  spotify:track:0axF06XwNLobcx5z8BVtYw  \n",
       "23852473  spotify:track:1GErReAT0swX36x3O8GyQn  \n",
       "23852480  spotify:track:5bTuwAgYUD5MABOIPH8ZMt  \n",
       "23852489  spotify:track:7lRlq939cDG4SzWOF4VAnd  \n",
       "23852497  spotify:track:1yKAqZoi8xWGLCf5vajroL  \n",
       "23852504  spotify:track:5wQnmLuC1W7ATsArWACrgW  \n",
       "23831835  spotify:track:7j31rVgGX9Q2blT92VBEA0  \n",
       "23821924  spotify:track:3iitOoZUtWEgwZOEYFITFo  \n",
       "23821932  spotify:track:0Zh3tKIphLOvQux4dA6PFZ  \n",
       "23821939  spotify:track:0uukw2CgEIApv4IWAjXrBC  \n",
       "23821947  spotify:track:4kXHIm097G3CemN3PzcpgL  \n",
       "23831818  spotify:track:1RT6zSjs59gJdFfUshT465  \n",
       "23831826  spotify:track:5BB0Jzw60KyfSTyjJqtely  \n",
       "23852512  spotify:track:5BB0Jzw60KyfSTyjJqtely  \n",
       "23852520  spotify:track:5dTHtzHFPyi8TlTtzoz1J9  \n",
       "23852527  spotify:track:1yKAqZoi8xWGLCf5vajroL  \n",
       "23852536  spotify:track:6VtcgrVYo2xfygcWAfRpd1  \n",
       "23852544  spotify:track:2aHqdN4K9naX3oBuUxp6zf  \n",
       "23852553  spotify:track:3HyDpKAuR3e4l6QB7hSB2l  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gdf[gdf.artist_name == \"My Chemical Romance\"].sort_values(\"pid\", ascending=True).head(50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the full set of artists in one of the playlists above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 708 ms, sys: 292 ms, total: 1 s\n",
      "Wall time: 1 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "2785018                Go Radio\n",
       "6063682           Peter Hollens\n",
       "10020486    Andrew Lloyd Webber\n",
       "16882518        Plain White T's\n",
       "18135123      A Day To Remember\n",
       "18521027              Passenger\n",
       "19748310        Christina Perri\n",
       "19748322        Christina Perri\n",
       "19911513           All Time Low\n",
       "19911520           All Time Low\n",
       "19925834           All Time Low\n",
       "21944781    5 Seconds of Summer\n",
       "21944792    5 Seconds of Summer\n",
       "21944797    5 Seconds of Summer\n",
       "21944802    5 Seconds of Summer\n",
       "21944808    5 Seconds of Summer\n",
       "21944814    5 Seconds of Summer\n",
       "21944821    5 Seconds of Summer\n",
       "21944827    5 Seconds of Summer\n",
       "21944834    5 Seconds of Summer\n",
       "21958311    5 Seconds of Summer\n",
       "21958318    5 Seconds of Summer\n",
       "21958325    5 Seconds of Summer\n",
       "21958330    5 Seconds of Summer\n",
       "21958336    5 Seconds of Summer\n",
       "21958342    5 Seconds of Summer\n",
       "21958348    5 Seconds of Summer\n",
       "21958353    5 Seconds of Summer\n",
       "21958360    5 Seconds of Summer\n",
       "21958367    5 Seconds of Summer\n",
       "                   ...         \n",
       "34056811           Fall Out Boy\n",
       "35537122            Troye Sivan\n",
       "35537127            Troye Sivan\n",
       "35537130            Troye Sivan\n",
       "35537132            Troye Sivan\n",
       "35537135            Troye Sivan\n",
       "35537139            Troye Sivan\n",
       "35841824             Bruno Mars\n",
       "35841826             Bruno Mars\n",
       "41433669             Ed Sheeran\n",
       "41433670             Ed Sheeran\n",
       "41433671             Ed Sheeran\n",
       "41433672             Ed Sheeran\n",
       "41607277    Panic! At The Disco\n",
       "41607280    Panic! At The Disco\n",
       "41607283    Panic! At The Disco\n",
       "41607286    Panic! At The Disco\n",
       "41607290    Panic! At The Disco\n",
       "41607292    Panic! At The Disco\n",
       "41607295    Panic! At The Disco\n",
       "41607298    Panic! At The Disco\n",
       "41607317    Panic! At The Disco\n",
       "41607321    Panic! At The Disco\n",
       "41607324    Panic! At The Disco\n",
       "41607328    Panic! At The Disco\n",
       "41607331    Panic! At The Disco\n",
       "41607334    Panic! At The Disco\n",
       "41607338    Panic! At The Disco\n",
       "41607342    Panic! At The Disco\n",
       "42577919                 Eminem\n",
       "Name: artist_name, Length: 76, dtype: object"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "gdf[gdf.pid==573].artist_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And, to make it easier to read, we can list just the unique set of artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 660 ms, sys: 348 ms, total: 1.01 s\n",
      "Wall time: 1.01 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0     5 Seconds of Summer\n",
       "1       A Day To Remember\n",
       "2            All Time Low\n",
       "3     Andrew Lloyd Webber\n",
       "4              Bruno Mars\n",
       "5         Christina Perri\n",
       "6               Disturbed\n",
       "7              Ed Sheeran\n",
       "8                  Eminem\n",
       "9            Fall Out Boy\n",
       "10               Go Radio\n",
       "11              Green Day\n",
       "12           Led Zeppelin\n",
       "13    My Chemical Romance\n",
       "14    Panic! At The Disco\n",
       "15              Passenger\n",
       "16          Peter Hollens\n",
       "17        Plain White T's\n",
       "18            Troye Sivan\n",
       "19              blink-182\n",
       "Name: artist_name, dtype: object"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "plist_orig = gdf[gdf.pid==573].artist_name.unique()\n",
    "plist_orig"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cuGraph's Jaccard computation allows us to pass a list of src/dst vertices to compute, so that we don't have to compute all possible combinations.\n",
    "\n",
    "We can use this playlist id as our source and compute the Jaccard similarities from that playlist to every other playlist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 5.22 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "original_pid = 573\n",
    "pid = numbering[numbering == original_pid].index[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.82 s, sys: 284 ms, total: 2.11 s\n",
      "Wall time: 2.11 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "clist = cudf.DataFrame()\n",
    "clist[\"second\"] = two_hop.second.unique().astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 56 ms, sys: 4 ms, total: 60 ms\n",
      "Wall time: 64.1 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "clist[\"first\"] = pid.astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 24 ms, sys: 4 ms, total: 28 ms\n",
      "Wall time: 28.6 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>second</th>\n",
       "      <th>first</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>985010</th>\n",
       "      <td>993062</td>\n",
       "      <td>69684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985009</th>\n",
       "      <td>993061</td>\n",
       "      <td>69684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985008</th>\n",
       "      <td>993060</td>\n",
       "      <td>69684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985007</th>\n",
       "      <td>993059</td>\n",
       "      <td>69684</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985006</th>\n",
       "      <td>993058</td>\n",
       "      <td>69684</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        second  first\n",
       "985010  993062  69684\n",
       "985009  993061  69684\n",
       "985008  993060  69684\n",
       "985007  993059  69684\n",
       "985006  993058  69684"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "clist.sort_values(\"second\", ascending=False).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 8 ms, sys: 0 ns, total: 8 ms\n",
      "Wall time: 7.9 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "jacc = cugraph.jaccard(G, first=clist.first, second=clist.second)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sorting our results in descending order by the Jaccard coefficient allows us to look at the most similar playlists first. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 32 ms, sys: 8 ms, total: 40 ms\n",
      "Wall time: 36.2 ms\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>69424</th>\n",
       "      <td>69684</td>\n",
       "      <td>69684</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>628238</th>\n",
       "      <td>69684</td>\n",
       "      <td>635007</td>\n",
       "      <td>0.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>721866</th>\n",
       "      <td>69684</td>\n",
       "      <td>729053</td>\n",
       "      <td>0.320000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>271176</th>\n",
       "      <td>69684</td>\n",
       "      <td>273662</td>\n",
       "      <td>0.291667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>853047</th>\n",
       "      <td>69684</td>\n",
       "      <td>860661</td>\n",
       "      <td>0.291667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85772</th>\n",
       "      <td>69684</td>\n",
       "      <td>86102</td>\n",
       "      <td>0.289474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>862395</th>\n",
       "      <td>69684</td>\n",
       "      <td>870035</td>\n",
       "      <td>0.280000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207958</th>\n",
       "      <td>69684</td>\n",
       "      <td>209698</td>\n",
       "      <td>0.275862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>252146</th>\n",
       "      <td>69684</td>\n",
       "      <td>254416</td>\n",
       "      <td>0.275862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>876096</th>\n",
       "      <td>69684</td>\n",
       "      <td>883779</td>\n",
       "      <td>0.275000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>818561</th>\n",
       "      <td>69684</td>\n",
       "      <td>826056</td>\n",
       "      <td>0.272727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>614347</th>\n",
       "      <td>69684</td>\n",
       "      <td>620945</td>\n",
       "      <td>0.269231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>791593</th>\n",
       "      <td>69684</td>\n",
       "      <td>799008</td>\n",
       "      <td>0.269231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>568400</th>\n",
       "      <td>69684</td>\n",
       "      <td>574446</td>\n",
       "      <td>0.266667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26308</th>\n",
       "      <td>69684</td>\n",
       "      <td>26397</td>\n",
       "      <td>0.263158</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>223150</th>\n",
       "      <td>69684</td>\n",
       "      <td>225074</td>\n",
       "      <td>0.260870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>357134</th>\n",
       "      <td>69684</td>\n",
       "      <td>360683</td>\n",
       "      <td>0.260870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>546094</th>\n",
       "      <td>69684</td>\n",
       "      <td>551887</td>\n",
       "      <td>0.260870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>915506</th>\n",
       "      <td>69684</td>\n",
       "      <td>923326</td>\n",
       "      <td>0.260870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31065</th>\n",
       "      <td>69684</td>\n",
       "      <td>31176</td>\n",
       "      <td>0.259259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40290</th>\n",
       "      <td>69684</td>\n",
       "      <td>40441</td>\n",
       "      <td>0.259259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>503575</th>\n",
       "      <td>69684</td>\n",
       "      <td>508870</td>\n",
       "      <td>0.258065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28300</th>\n",
       "      <td>69684</td>\n",
       "      <td>28401</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132299</th>\n",
       "      <td>69684</td>\n",
       "      <td>133170</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>267471</th>\n",
       "      <td>69684</td>\n",
       "      <td>269914</td>\n",
       "      <td>0.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        source  destination  jaccard_coeff\n",
       "69424    69684        69684       1.000000\n",
       "628238   69684       635007       0.333333\n",
       "721866   69684       729053       0.320000\n",
       "271176   69684       273662       0.291667\n",
       "853047   69684       860661       0.291667\n",
       "85772    69684        86102       0.289474\n",
       "862395   69684       870035       0.280000\n",
       "207958   69684       209698       0.275862\n",
       "252146   69684       254416       0.275862\n",
       "876096   69684       883779       0.275000\n",
       "818561   69684       826056       0.272727\n",
       "614347   69684       620945       0.269231\n",
       "791593   69684       799008       0.269231\n",
       "568400   69684       574446       0.266667\n",
       "26308    69684        26397       0.263158\n",
       "223150   69684       225074       0.260870\n",
       "357134   69684       360683       0.260870\n",
       "546094   69684       551887       0.260870\n",
       "915506   69684       923326       0.260870\n",
       "31065    69684        31176       0.259259\n",
       "40290    69684        40441       0.259259\n",
       "503575   69684       508870       0.258065\n",
       "28300    69684        28401       0.250000\n",
       "132299   69684       133170       0.250000\n",
       "267471   69684       269914       0.250000"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "jacc.sort_values(\"jaccard_coeff\", ascending=False).head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's reverse the encoding of our playlist id so that we can look at the artists in a couple of the most similar playlists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 379 µs\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "218188"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "pid = numbering[635007]\n",
    "pid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 712 ms, sys: 308 ms, total: 1.02 s\n",
      "Wall time: 1.02 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0      5 Seconds of Summer\n",
       "1        A Day To Remember\n",
       "2             All Time Low\n",
       "3     Bring Me The Horizon\n",
       "4                Green Day\n",
       "5      My Chemical Romance\n",
       "6                 Paramore\n",
       "7          Plain White T's\n",
       "8              Simple Plan\n",
       "9              Troye Sivan\n",
       "10            We The Kings\n",
       "11               blink-182\n",
       "Name: artist_name, dtype: object"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "plist_a = gdf[gdf.pid == pid].artist_name.unique()\n",
    "plist_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 652 ms, sys: 372 ms, total: 1.02 s\n",
      "Wall time: 1.02 s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0                   3OH!3\n",
       "1            All Time Low\n",
       "2              Ed Sheeran\n",
       "3            Fall Out Boy\n",
       "4               Green Day\n",
       "5                  Halsey\n",
       "6        Melanie Martinez\n",
       "7     My Chemical Romance\n",
       "8     Panic! At The Disco\n",
       "9     Secondhand Serenade\n",
       "10            Troye Sivan\n",
       "11      Twenty One Pilots\n",
       "12              blink-182\n",
       "Name: artist_name, dtype: object"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "pid = numbering[729053]\n",
    "plist_b = gdf[gdf.pid == pid].artist_name.unique()\n",
    "plist_b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can remove the intersection of all the playlists to get the artists that weren't in the original. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "union = cudf.concat([plist_a, plist_b]).reset_index().drop_duplicates()\n",
    "union[\"new\"] = union[\"artist_name\"].isin(plist_orig)\n",
    "union[union.new == False].artist_name.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try another artist. We can go for one of the most popular- The Beatles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf[gdf.artist_name == \"The Beatles\"].head(50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the problems using playlists to detminermine artist similarity is the playlists vary widely in their levels of entropy- some are composed of a single style and time period while others are spread over several styles and time periods. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf[gdf.pid == 155151].artist_name.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clist[\"second\"] = two_hop.second.unique().astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clist[\"first\"] = numbering[numbering==155151].index[0].astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clist.sort_values(\"second\", ascending=False).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "jacc = cugraph.jaccard(G, first=clist.first, second=clist.second)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "jacc.sort_values(\"jaccard_coeff\", ascending=False).head(15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pid = numbering[782501]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf[gdf.pid==pid].artist_name.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's conclude our artist similarity predictor with one more example, taken from 90's rock bands. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf[gdf.artist_name == \"Foo Fighters\"].head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can choose an unencoded playlist id from the list above and take a look at all the unique artists in it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[gdf.pid==814771].artist_name.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to query this playlist in our graph, we'll encode it using our vertex renumbering mapping,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "pid_enc = numbering[numbering == 814771].index[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "build our Jaccard filter list..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "clist[\"second\"] = two_hop.second.unique().astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "clist[\"first\"] = pid_enc.astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clist.sort_values(\"second\", ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and compute our similarities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "jacc = cugraph.jaccard(G, first=clist.first, second=clist.second)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "jacc.sort_values(\"jaccard_coeff\", ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, instead of just manually looking up one playlist at a time, let's take a different approach and grab artists from the `n` closest playlist ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "n = 50\n",
    "recc_pid_enc = jacc.sort_values(\"jaccard_coeff\", ascending=False)[\"destination\"][1:n]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pids = numbering[recc_pid_enc].to_array()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pids"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also rank the similarity of the artists based on how many of the above playlists they occur in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "query = gdf[gdf.pid.isin(pids)][[\"artist_name\", \"artist_uri\", \"pid\"]].drop_duplicates()\n",
    "query = query.groupby([\"artist_name\"]).count().reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "query = query.rename({\"artist_uri\": \"count\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "query.sort_values(\"count\", ascending=False).head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can look at some of the \"noise\" at the bottom of the list and see that they are less relevant to our query artist than the ones at the top of the list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "query.sort_values(\"count\", ascending=True).head(30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build Collaborative Filtering Dataset\n",
    "\n",
    "Let's conclude by transforming our dataset into a format that can be used to train collaborative filtering models. \n",
    "\n",
    "We don't have any explicit ratings of artists, but we do have the jaccard similarities for each of the playlists. This can allow us to do playlist-playlist similarity by embedding our jaccard scores into a continuous vector space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 384 ms, sys: 340 ms, total: 724 ms\n",
      "Wall time: 720 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "jacc = cugraph.jaccard(G, two_hop.first, two_hop.second)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>848487</td>\n",
       "      <td>0</td>\n",
       "      <td>0.019417</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>848487</td>\n",
       "      <td>1</td>\n",
       "      <td>0.033058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>848487</td>\n",
       "      <td>3</td>\n",
       "      <td>0.008696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>848487</td>\n",
       "      <td>4</td>\n",
       "      <td>0.014085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>848487</td>\n",
       "      <td>6</td>\n",
       "      <td>0.013158</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   source  destination  jaccard_coeff\n",
       "0  848487            0       0.019417\n",
       "1  848487            1       0.033058\n",
       "2  848487            3       0.008696\n",
       "3  848487            4       0.014085\n",
       "4  848487            6       0.013158"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jacc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(139802415, 3)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jacc.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [source, destination, jaccard_coeff]\n",
       "Index: []"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jacc[jacc.source >= 1000000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We only want the source ids for artists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>src</th>\n",
       "      <th>dst</th>\n",
       "      <th>weights</th>\n",
       "      <th>src_id</th>\n",
       "      <th>dst_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>16</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1952</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>33</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4021</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>97</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>11797</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>112</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13624</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>138</td>\n",
       "      <td>1000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16787</td>\n",
       "      <td>84993</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   src      dst  weights  src_id  dst_id\n",
       "0   16  1000000      1.0    1952   84993\n",
       "1   33  1000000      1.0    4021   84993\n",
       "2   97  1000000      1.0   11797   84993\n",
       "3  112  1000000      1.0   13624   84993\n",
       "4  138  1000000      1.0   16787   84993"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_graph_gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(62559698, 5)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_graph_gdf.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can store our original dataframe out to host memory to save some space on the GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 23.2 s, sys: 7.03 s, total: 30.2 s\n",
      "Wall time: 30.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "gdf = gdf.to_pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will allow us to merge our jaccard weights into our graph dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 332 ms, sys: 152 ms, total: 484 ms\n",
      "Wall time: 481 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "tmp_df = cudf.DataFrame()\n",
    "tmp_df[\"src\"] = new_graph_gdf[\"src\"]\n",
    "tmp_df[\"src_id\"] = new_graph_gdf[\"src_id\"]\n",
    "\n",
    "tmp_df = tmp_df.drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [source, destination, jaccard_coeff]\n",
       "Index: []"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jacc[jacc.jaccard_coeff>=1000000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "collab_filt = tmp_df.merge(jacc, right_on=\"source\", left_on=\"src_id\", how=\"left\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>src</th>\n",
       "      <th>src_id</th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>64779</td>\n",
       "      <td>902782</td>\n",
       "      <td>902782</td>\n",
       "      <td>null</td>\n",
       "      <td>null</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>64780</td>\n",
       "      <td>902903</td>\n",
       "      <td>902903</td>\n",
       "      <td>null</td>\n",
       "      <td>null</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>64781</td>\n",
       "      <td>903024</td>\n",
       "      <td>903024</td>\n",
       "      <td>null</td>\n",
       "      <td>null</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>64782</td>\n",
       "      <td>903142</td>\n",
       "      <td>903142</td>\n",
       "      <td>null</td>\n",
       "      <td>null</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>64783</td>\n",
       "      <td>903263</td>\n",
       "      <td>903263</td>\n",
       "      <td>null</td>\n",
       "      <td>null</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     src  src_id  source destination jaccard_coeff\n",
       "0  64779  902782  902782        null          null\n",
       "1  64780  902903  902903        null          null\n",
       "2  64781  903024  903024        null          null\n",
       "3  64782  903142  903142        null          null\n",
       "4  64783  903263  903263        null          null"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filt.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>src</th>\n",
       "      <th>src_id</th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3137185</th>\n",
       "      <td>15184</td>\n",
       "      <td>848621</td>\n",
       "      <td>848621</td>\n",
       "      <td>25809</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3137889</th>\n",
       "      <td>15184</td>\n",
       "      <td>848621</td>\n",
       "      <td>848621</td>\n",
       "      <td>19813</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3142077</th>\n",
       "      <td>15184</td>\n",
       "      <td>848621</td>\n",
       "      <td>848621</td>\n",
       "      <td>49975</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3142405</th>\n",
       "      <td>15184</td>\n",
       "      <td>848621</td>\n",
       "      <td>848621</td>\n",
       "      <td>49055</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3144521</th>\n",
       "      <td>15184</td>\n",
       "      <td>848621</td>\n",
       "      <td>848621</td>\n",
       "      <td>71929</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           src  src_id  source  destination  jaccard_coeff\n",
       "3137185  15184  848621  848621        25809            1.0\n",
       "3137889  15184  848621  848621        19813            1.0\n",
       "3142077  15184  848621  848621        49975            1.0\n",
       "3142405  15184  848621  848621        49055            1.0\n",
       "3144521  15184  848621  848621        71929            1.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filt.sort_values(\"jaccard_coeff\", ascending=False).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 288 ms, sys: 136 ms, total: 424 ms\n",
      "Wall time: 423 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "tmp_df = cudf.DataFrame()\n",
    "tmp_df[\"dst\"] = new_graph_gdf[\"dst\"]\n",
    "tmp_df[\"dst_id\"] = new_graph_gdf[\"dst_id\"]\n",
    "\n",
    "tmp_df = tmp_df.drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(993063, 2)"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "collab_filt = tmp_df.merge(collab_filt, right_on=\"destination\", left_on=\"dst_id\", how=\"left\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dst</th>\n",
       "      <th>dst_id</th>\n",
       "      <th>src</th>\n",
       "      <th>src_id</th>\n",
       "      <th>source</th>\n",
       "      <th>destination</th>\n",
       "      <th>jaccard_coeff</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77059858</th>\n",
       "      <td>551912</td>\n",
       "      <td>378892</td>\n",
       "      <td>187196</td>\n",
       "      <td>848764</td>\n",
       "      <td>848764</td>\n",
       "      <td>378892</td>\n",
       "      <td>0.003021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>104483148</th>\n",
       "      <td>748369</td>\n",
       "      <td>363479</td>\n",
       "      <td>187196</td>\n",
       "      <td>848764</td>\n",
       "      <td>848764</td>\n",
       "      <td>363479</td>\n",
       "      <td>0.003135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3531115</th>\n",
       "      <td>47101</td>\n",
       "      <td>746483</td>\n",
       "      <td>228151</td>\n",
       "      <td>848769</td>\n",
       "      <td>848769</td>\n",
       "      <td>746483</td>\n",
       "      <td>0.003175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52927036</th>\n",
       "      <td>379527</td>\n",
       "      <td>333392</td>\n",
       "      <td>187196</td>\n",
       "      <td>848764</td>\n",
       "      <td>848764</td>\n",
       "      <td>333392</td>\n",
       "      <td>0.003247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10775752</th>\n",
       "      <td>83820</td>\n",
       "      <td>232259</td>\n",
       "      <td>228151</td>\n",
       "      <td>848769</td>\n",
       "      <td>848769</td>\n",
       "      <td>232259</td>\n",
       "      <td>0.003268</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              dst  dst_id     src  src_id  source  destination  jaccard_coeff\n",
       "                                                                             \n",
       "77059858   551912  378892  187196  848764  848764       378892       0.003021\n",
       "104483148  748369  363479  187196  848764  848764       363479       0.003135\n",
       "3531115     47101  746483  228151  848769  848769       746483       0.003175\n",
       "52927036   379527  333392  187196  848764  848764       333392       0.003247\n",
       "10775752    83820  232259  228151  848769  848769       232259       0.003268"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filt.sort_values(\"jaccard_coeff\", ascending=True).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "collab_filt.to_csv(\"../data/collab_filter_train.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(139805871, 7)"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filt[collab_filt.dst<1000000].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(139810467, 7)"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filt.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pa = np.array(gdf[gdf.pid == numbering[848487]].artist_name.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pb = np.array(gdf[gdf.pid==numbering[299262]].artist_name.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(np.intersect1d(pa, pb))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "39/len(pa)+len(pb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf.num_followers.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[gdf.num_followers < 20].shape[0] / gdf.shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[gdf.num_followers<20].num_followers.to_pandas().hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[\"num_followers_norm\"] = np.log1p(gdf.num_followers.to_pandas())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "min_val = gdf[\"num_followers_norm\"].min()\n",
    "max_val = gdf[\"num_followers_norm\"].max()\n",
    "gdf[\"num_followers_norm\"] = ((gdf[\"num_followers_norm\"]-min_val) / (max_val-min_val)) * (10-1)+1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[\"num_followers_norm\"].to_pandas().hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[\"num_followers_norm\"].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "gdf[\"num_followers_norm\"].min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gdf.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [],
   "source": [
    "collab_filter_train = gdf[[\"artist_uri\", \"pid\", \"num_followers_norm\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.collab import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2.26 s, sys: 1.04 s, total: 3.3 s\n",
      "Wall time: 3.29 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "le = cuml.preprocessing.LabelEncoder()\n",
    "collab_filter_train[\"artist_uri\"] = le.fit_transform(collab_filter_train[\"artist_uri\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.77 s, sys: 796 ms, total: 2.57 s\n",
      "Wall time: 2.57 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "pid_le = cuml.preprocessing.LabelEncoder()\n",
    "collab_filter_train[\"pid\"] = pid_le.fit_transform(collab_filter_train[\"pid\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>pid</th>\n",
       "      <th>num_followers_norm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2464</td>\n",
       "      <td>789601</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2465</td>\n",
       "      <td>789738</td>\n",
       "      <td>1.594901</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2466</td>\n",
       "      <td>789737</td>\n",
       "      <td>2.836736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2467</td>\n",
       "      <td>61274</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2468</td>\n",
       "      <td>789708</td>\n",
       "      <td>1.347995</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   artist_uri     pid  num_followers_norm\n",
       "0        2464  789601            1.000000\n",
       "1        2465  789738            1.594901\n",
       "2        2466  789737            2.836736\n",
       "3        2467   61274            1.000000\n",
       "4        2468  789708            1.347995"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filter_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 40.4 s, sys: 16.6 s, total: 57 s\n",
      "Wall time: 56.3 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "data = CollabDataBunch.from_df(collab_filter_train.to_pandas(), \n",
    "                               user_name=\"pid\", \n",
    "                               item_name=\"artist_uri\", \n",
    "                               rating_name=\"num_followers_norm\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 4 ms, sys: 0 ns, total: 4 ms\n",
      "Wall time: 3.65 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "y_range = [collab_filter_train[\"num_followers_norm\"].min(), collab_filter_train[\"num_followers_norm\"].max()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1.0, 10.0]"
      ]
     },
     "execution_count": 151,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_range"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1.34 s, sys: 76 ms, total: 1.41 s\n",
      "Wall time: 405 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "learn = collab_learner(data, n_factors=25, y_range=y_range, path=\"collab_filter_model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.199590</td>\n",
       "      <td>0.212780</td>\n",
       "      <td>1:10:13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.186751</td>\n",
       "      <td>0.192267</td>\n",
       "      <td>1:09:53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.090717</td>\n",
       "      <td>0.094251</td>\n",
       "      <td>1:09:40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.041679</td>\n",
       "      <td>0.043450</td>\n",
       "      <td>1:09:52</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(4, 5e-3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist_uri</th>\n",
       "      <th>pid_y</th>\n",
       "      <th>num_followers_norm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5500</th>\n",
       "      <td>4700</td>\n",
       "      <td>61173</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5532</th>\n",
       "      <td>4700</td>\n",
       "      <td>61715</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5564</th>\n",
       "      <td>4700</td>\n",
       "      <td>61693</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23228</th>\n",
       "      <td>4700</td>\n",
       "      <td>61747</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23260</th>\n",
       "      <td>4700</td>\n",
       "      <td>794638</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23292</th>\n",
       "      <td>4700</td>\n",
       "      <td>794638</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44044</th>\n",
       "      <td>4700</td>\n",
       "      <td>794792</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44076</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44108</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62690</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62722</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62754</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79343</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79375</th>\n",
       "      <td>4700</td>\n",
       "      <td>908766</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79407</th>\n",
       "      <td>4700</td>\n",
       "      <td>908834</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96857</th>\n",
       "      <td>4700</td>\n",
       "      <td>908879</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96889</th>\n",
       "      <td>4700</td>\n",
       "      <td>298396</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96921</th>\n",
       "      <td>4700</td>\n",
       "      <td>298396</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115677</th>\n",
       "      <td>4700</td>\n",
       "      <td>298828</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115709</th>\n",
       "      <td>4700</td>\n",
       "      <td>298877</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>115741</th>\n",
       "      <td>4700</td>\n",
       "      <td>298877</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136815</th>\n",
       "      <td>4700</td>\n",
       "      <td>298877</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136847</th>\n",
       "      <td>4700</td>\n",
       "      <td>298864</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>136879</th>\n",
       "      <td>4700</td>\n",
       "      <td>299246</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152802</th>\n",
       "      <td>4700</td>\n",
       "      <td>365663</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152834</th>\n",
       "      <td>4700</td>\n",
       "      <td>365927</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152866</th>\n",
       "      <td>4700</td>\n",
       "      <td>366351</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167620</th>\n",
       "      <td>4700</td>\n",
       "      <td>821762</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167652</th>\n",
       "      <td>4700</td>\n",
       "      <td>821860</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167684</th>\n",
       "      <td>4700</td>\n",
       "      <td>42118</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11044725</th>\n",
       "      <td>4700</td>\n",
       "      <td>338361</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11044743</th>\n",
       "      <td>4700</td>\n",
       "      <td>338710</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11044759</th>\n",
       "      <td>4700</td>\n",
       "      <td>339087</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11044776</th>\n",
       "      <td>4700</td>\n",
       "      <td>428676</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061363</th>\n",
       "      <td>4700</td>\n",
       "      <td>428676</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061379</th>\n",
       "      <td>4700</td>\n",
       "      <td>429492</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061396</th>\n",
       "      <td>4700</td>\n",
       "      <td>930905</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061413</th>\n",
       "      <td>4700</td>\n",
       "      <td>931131</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061430</th>\n",
       "      <td>4700</td>\n",
       "      <td>931131</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11061447</th>\n",
       "      <td>4700</td>\n",
       "      <td>207945</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11076975</th>\n",
       "      <td>4700</td>\n",
       "      <td>208380</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11076992</th>\n",
       "      <td>4700</td>\n",
       "      <td>424857</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11077010</th>\n",
       "      <td>4700</td>\n",
       "      <td>425162</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11077045</th>\n",
       "      <td>4700</td>\n",
       "      <td>576441</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11077062</th>\n",
       "      <td>4700</td>\n",
       "      <td>435484</td>\n",
       "      <td>1.609438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11093876</th>\n",
       "      <td>4700</td>\n",
       "      <td>435484</td>\n",
       "      <td>1.609438</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11093909</th>\n",
       "      <td>4700</td>\n",
       "      <td>319349</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11093926</th>\n",
       "      <td>4700</td>\n",
       "      <td>86408</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11093960</th>\n",
       "      <td>4700</td>\n",
       "      <td>707183</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11111810</th>\n",
       "      <td>4700</td>\n",
       "      <td>707228</td>\n",
       "      <td>2.890372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11111827</th>\n",
       "      <td>4700</td>\n",
       "      <td>707515</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11111861</th>\n",
       "      <td>4700</td>\n",
       "      <td>358026</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11111879</th>\n",
       "      <td>4700</td>\n",
       "      <td>969821</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11111896</th>\n",
       "      <td>4700</td>\n",
       "      <td>114969</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127669</th>\n",
       "      <td>4700</td>\n",
       "      <td>115756</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127687</th>\n",
       "      <td>4700</td>\n",
       "      <td>645584</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127704</th>\n",
       "      <td>4700</td>\n",
       "      <td>646214</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127722</th>\n",
       "      <td>4700</td>\n",
       "      <td>646214</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127738</th>\n",
       "      <td>4700</td>\n",
       "      <td>646214</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11127756</th>\n",
       "      <td>4700</td>\n",
       "      <td>646214</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2475 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          artist_uri   pid_y  num_followers_norm\n",
       "5500            4700   61173            0.693147\n",
       "5532            4700   61715            0.693147\n",
       "5564            4700   61693            0.693147\n",
       "23228           4700   61747            0.693147\n",
       "23260           4700  794638            0.693147\n",
       "23292           4700  794638            0.693147\n",
       "44044           4700  794792            1.098612\n",
       "44076           4700  908766            0.693147\n",
       "44108           4700  908766            0.693147\n",
       "62690           4700  908766            0.693147\n",
       "62722           4700  908766            0.693147\n",
       "62754           4700  908766            0.693147\n",
       "79343           4700  908766            0.693147\n",
       "79375           4700  908766            0.693147\n",
       "79407           4700  908834            0.693147\n",
       "96857           4700  908879            0.693147\n",
       "96889           4700  298396            1.098612\n",
       "96921           4700  298396            1.098612\n",
       "115677          4700  298828            0.693147\n",
       "115709          4700  298877            1.098612\n",
       "115741          4700  298877            1.098612\n",
       "136815          4700  298877            1.098612\n",
       "136847          4700  298864            1.098612\n",
       "136879          4700  299246            0.693147\n",
       "152802          4700  365663            0.693147\n",
       "152834          4700  365927            0.693147\n",
       "152866          4700  366351            0.693147\n",
       "167620          4700  821762            0.693147\n",
       "167652          4700  821860            0.693147\n",
       "167684          4700   42118            0.693147\n",
       "...              ...     ...                 ...\n",
       "11044725        4700  338361            0.693147\n",
       "11044743        4700  338710            0.693147\n",
       "11044759        4700  339087            0.693147\n",
       "11044776        4700  428676            1.098612\n",
       "11061363        4700  428676            1.098612\n",
       "11061379        4700  429492            0.693147\n",
       "11061396        4700  930905            0.693147\n",
       "11061413        4700  931131            0.693147\n",
       "11061430        4700  931131            0.693147\n",
       "11061447        4700  207945            0.693147\n",
       "11076975        4700  208380            0.693147\n",
       "11076992        4700  424857            1.098612\n",
       "11077010        4700  425162            0.693147\n",
       "11077045        4700  576441            0.693147\n",
       "11077062        4700  435484            1.609438\n",
       "11093876        4700  435484            1.609438\n",
       "11093909        4700  319349            1.098612\n",
       "11093926        4700   86408            1.098612\n",
       "11093960        4700  707183            0.693147\n",
       "11111810        4700  707228            2.890372\n",
       "11111827        4700  707515            0.693147\n",
       "11111861        4700  358026            0.693147\n",
       "11111879        4700  969821            0.693147\n",
       "11111896        4700  114969            0.693147\n",
       "11127669        4700  115756            0.693147\n",
       "11127687        4700  645584            0.693147\n",
       "11127704        4700  646214            0.693147\n",
       "11127722        4700  646214            0.693147\n",
       "11127738        4700  646214            0.693147\n",
       "11127756        4700  646214            0.693147\n",
       "\n",
       "[2475 rows x 3 columns]"
      ]
     },
     "execution_count": 242,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filter_train[collab_filter_train.artist_uri==4700]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.export('collaborative_filtering_model.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(FloatItem 0.6958642, tensor(0.6959), tensor(0.6959))"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.predict(collab_filter_train.to_pandas().iloc[50000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "embeds = to_np(learn.model.u_weight.weight)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cuml.manifold import UMAP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/raid/cjnolet/conda/cuml_dev_10/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Parameter should_downcast is deprecated, use convert_dtype in fit, fit_transform and transform  methods instead. \n",
      "  \"\"\"Entry point for launching an IPython kernel.\n",
      "/raid/cjnolet/conda/cuml_dev_10/lib/python3.7/site-packages/ipykernel_launcher.py:2: UserWarning: Parameter should_downcast is deprecated, use convert_dtype in fit, fit_transform and transform  methods instead. \n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "UMAP(n_neighbors=15, verbose=False, handle=<cuml.common.handle.Handle object at 0x7f0bc222c168>, callback=None)"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "umap = UMAP()\n",
    "umap.fit(embeds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/raid/cjnolet/conda/cuml_dev_10/lib/python3.7/site-packages/ipykernel_launcher.py:1: UserWarning: Parameter should_downcast is deprecated, use convert_dtype in fit, fit_transform and transform  methods instead. \n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    }
   ],
   "source": [
    "umap_embeddings = umap.fit_transform(embeds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df3TcdZ3v8ec76RRSxJsi5VegBqGWlS0k3iy0p2f3ggIFqhC5i4UF17t7Dl1c9VhgqyntSvFW27WKdS9evaDes3vo0YBCRFMpKHh35VIkmJRSodsWa2nKpVUIIA1tmr7vHzMTJpOZ/JjM98fM9/U4Jyfz45uZd9PJ9/39/Hp/zN0REZHkqok6ABERiZYSgYhIwikRiIgknBKBiEjCKRGIiCTclKgDKMXxxx/vjY2NUYchIlJRnn766d+7+4z8xysyETQ2NtLV1RV1GCIiFcXMflfocXUNiYgknBKBiEjCKRGIiCScEoGISMIpEYiIJFxFzhqS0jW2dY54bNeahRFEIiJxoRZBghRKAqM9LiLJoEQgIpJwSgQiIgmnRCAiknBKBCIiCadEkCDFZgdp1pBIsmn6aMLopC8i+dQiEBFJOCUCEZGEUyIQEUk4JQIRkYRTIhARSTglAhGRhAs0EZjZbDPryfl63cyW5B1zgZm9lnPM54OMSUREhgt0HYG7bwOaAMysFugFHihw6L+7+4eCjEVERAoLs2vog8BOd/9diO8pIiJjCDMRXAN8r8hz88xss5n91MzOLnSAmS02sy4z69q/f39wUYqIJEwoicDMpgJXAPcVePrXwLvd/VzgfwAdhV7D3e9y9xZ3b5kxY0ZwwYqIJExYLYLLgF+7+8v5T7j76+7+x8ztDUDKzI4PKS4RkcQLKxFcS5FuITM7ycwsc/u8TEx/CCkuEZHEC7z6qJlNAy4G/i7nsRsB3P1bwF8CnzCzw0A/cI27e9BxiYhIWuCJwN0PAO/Ke+xbObfvBO4MOg4RESlMK4tFRBJOiUBEJOGUCEREEk6JQEQk4bRnsSTOdXc/weM7Xyn55w1woKG+jqULZtPa3FC22ESiYJU4U7OlpcW7urqiDkMqwGRP+hNx/dyZrGqdE8p7iZTCzJ5295b8x9UikKqzomML6zftJuxLnHs27ebep17k0KAPtRryTZ+W4rYPn61WhMSKWgRSVU5v6ww9AZQiVQNrr25SQpBQFWsRKBFI1aiUJFBMXaqG1Vedo+QggSmWCDRrSKrCio4tFZ0EAPoHjrCkvYfGtk7mr3mUju7eqEOShFAikKpwz6bdUYdQVr19/Sy7f4uSgYRCg8VS8ar1ZNk/MMiS9h6WtPcMPbZukcYVpPzUIpCKd/uPt0YdQmiWtPdweltn1SY/iYYSgVS8Vw8MRB1CqJx0Qpi94qdKCFIWSgRS0ZJ8Ijx4OD24fNbyDVGHIhVOiUAq2soHk9MtVMxbg65kIJMSeCIws11mtsXMesxsxOR/S/tnM9thZs+Y2fuDjkmqR19/srqFinlr0DljmcYOpDRhzRq60N1/X+S5y4BZma/zgW9mvovIBAw6w2YZqSiejFccuoauBP7V0zYB9WZ2ctRBSfyt6NgSdQix1tvXz83tPWolyJjCSAQOPGxmT5vZ4gLPNwAv5tzfk3lsGDNbbGZdZta1f//+gEKVSlJti8iCcIR0K0ErlWU0YSSC+e7+ftJdQJ80s7/Ie94K/MyIagHufpe7t7h7y4wZM4KIU6Rq9fb1s6S9hz/5R005lZECTwTuvjfzfR/wAHBe3iF7gNNy7p8K7A06LpEkytYzOr2tU11rMiTQwWIzOwaocfc3MrcvAb6Qd9iDwKfM7PukB4lfc/eXgoxLJOmcdNdabvea9kpIrqBbBCcCvzSzzcCvgE53f8jMbjSzGzPHbABeAHYAdwN/H3BMIlLAqwcGWNLew5nLOqMORUIWaIvA3V8Azi3w+LdybjvwySDjEJHxO+zQ2NbJrjULow5FQhKH6aMiJZk+LRV1CFXturufiDoECYkSgVSswcEjUYdQ1R7f+UrUIUhIlAikIq3o2MLrBwejDqPqaappMigRSEXSYrJwLNHK5ERQIpCKoxNTuJa092jNQZVTIpCKk6QdyeLink27Oee2h6IOQwKSuD2LO7p7WbtxG3v7+jlF1RkrUtJ2JIuL1w8O0tjWiYH+dqpMYhJBR3fvsE3AIV1/Zdn96SavPtAi4+Pob6faJKJrqFASyOofGGTtxm0hRySTUV+XjPUDhaoxxkn/wKAqm1aJRCSCsU70e/v6Q4pEymHlFWdHHUJZjHaib6iv42uLmli3qIlUTbxTQrayaWNbpxahVahEdA2NdaI/pb4upEikHLJdEcVaeZViRK31DAMeb/vAsMcq5d/6+M5XaGzrZIrBjtUqUVEpEtEiGOtEv3TB7JAikXJpbW5g15qFXD93ZtShlF3+57W1uYFZJxwTUTSlOeyoeF0FSUQiGOtEr8GuyrWqdQ671iwc+mqooNZdfV2KulTtsMfqUrUjPq/X3f0E2/e9GWZoZXG4WJNHYicRiaC1uYF1i5oKPqcKi9Vl6YLZFfGhrrH0WMfqq+YMG/w+OjU8+o7uXtX8kcAlYowA0slAV/7Vr1LGD46aUkNrcwMd3b0cPPx28bxXDwwMTcsEuOXezVGEJwmTmEQgyZFN+tfd/URsr6b7B47Q0d3LLfduZtA977lBVj64lYOHj4x4rpJMifdkJ8kRaCvazE4zs8fM7Dkz22pmnylwzAVm9pqZ9WS+Ph9kTJIc62+YNzSgXGvps1KtGdfPncm6RU1EPStz6Q9GJoGsvv4B+gcqt7qqoVlDlSToFsFh4BZ3/7WZHQs8bWaPuPtv8o77d3f/UMCxSEKtap3DqtY5Ix5vbW6gsS26mS0Dg5V7tT+WulQljNRIVtBbVb4EvJS5/YaZPQc0APmJQCQSDfV19GpBYdkdGDgylGRnnXAMj9x8QbQByahCGyMws0agGXiywNPzMhvc7wX+wd1HlJc0s8XAYoCZM6tv7rhEY+mC2dxy32YGj8Tr6ry+LsXBw0cqunsoa/u+N7n4jl8kNhlcfMcvCk7/jdOMxVASgZm9A/ghsMTdX897+tfAu939j2Z2OdABzMp/DXe/C7gLoKWlJV5/tVKxsrOMlj+whTcPxeOkW5eqHSqjcev9z3BgoPK35KzEdRCTNVa3Y2NbZ2ySQeAdeWaWIp0E1rv7/fnPu/vr7v7HzO0NQMrMjg86LpGs1uYGtn7hUnatWcj8M46LNJaG+jpWXzVnaObTb/77ZdRq9k3FiXLsqRSBtgjMzIDvAM+5+x1FjjkJeNnd3czOI52c/hBkXJWk0AcqLlcR1Wj9DfOG9qwYa+ygvi6FGfQdGChaN2gijHQBt2yRxGxr5asfbYr9ugh5WyVWYg26a2g+8DFgi5llP8m3AjMB3P1bwF8CnzCzw0A/cI17BU+eLqNiVxVxalJWo9zFh/PXPFowITTU1w0rDFfsuInIfujza/23NjdUfCKIe62kju5ebv/x1oKbHtXXpXitf4BT6uu48KwZI/bLrjFwf3uznkosax/0rKFfMkZZdXe/E7gzyDhESrV0wWyW3b9l2KBtoXpAhU4Qk5HdJyObkCp5dtPRtRbLgeLxtvz6+tPJobevv+D/cXaeQW9fPze195SldRg2rSwWGUX2RJy/vSlA0+0PD50kgpBbPr1QQqoUb8VwvURHd28gv8+J/Evj1KpXIhAZQ36dqo7uXpbet5mBgKec1pjR0d079N5Hp2qGTlzTUjVMnVJLX/8AtWYVXYoiCms3bgs0qRqFk0Jc11QoEYhM0NqN2wJPAgCD7ixp72HpfT3kzyB1jJVXnD0sQcV5psqKji0FV3dHJehdCZ10d15uKzLORS+VCCJQ6A923aKmER+UXWsWatZQDIW9tWmhZQT5YwgQ73GEezbtpuXdx8XmZHhKwL+r/MkEcWeVOEGnpaXFu7q6og6jJKNdtc0/4zjW3zAvxGikFM1feLjg7JKoTJ+W4rYPpxegxX12UVwuYjq6ewP7XdWlaofWgkw0pvyxqHInTjN72t1bRjyuRBCusZrvBvw2Jn8sUljQg8Slun7uTDqfeSlWSSpfraXXRcShZRDU/2Oh1v1YxjN4XY7xhWKJQCUCY8ZJJ4vGtk7mr3m0IhenVLvXYpgEANZv2s3Cc04esf1lnAw63HLf5lh8rrNlPMrpmKm1JSW58Qxeb9/3ZmDjQEoEMdbb18+S9h7OWr4h6lAkR/7m8mMJq0KEA489v5/VV81h+rTUmMdHZfCIc/uPR9SVDF0QrZIDJdarmsi4UxDJQIkgRKVeBb016Jxz20NljkZKtXTB7KJX3XWpGtYtamLXmoVDX19b1ETDBJNHqbInlLdiXqju1QMDNH/h4chbBu88qrytp4leJJT6c2cuK28y0BhBiN7T1slk/jzjMtAmkxvYy99CM1UDa69uKlriYCKyCSeus4cKiXpufTmvsEsZH4DSFriVcj4oNkag6aMh6ejunVQSkHjJX2Q2EaPNDLv53h5KXaKQLX1xU8xnDuXbvu9Nzv/iIzy5/OKoQ5m0Uj8T2Z8rx8VAKdQ1FJCO7l7mr3mU0zODvnHoE5V4a21u4I6PNk3oZxrq6zCGl68utXsiSi+/cYgVHVuiDmNS1i2a2P9dvtbmBro/f8m4X+e6u5+Y1PvlUougjDq6e/nsDzZzKK+2SiU10yVarc0NdP3ulXEVsJs+LVVw0VKl1iW6Z9PuWK0+Ho1lKo5CutzHl646p2yDz9nW5lhdVrndi5OlRDBJxbahC0JjW2fJfZBSOVa1zuHJF/4w5ufqtf4BVnRs4bHn9xccq8iOYVTSKOA5tz3EM7dfGnUYo6o1Y+fqywN/n/lnHFfWk/1oNFg8hhUdWwpendUYzHjHVF5+41AoceRSMkiGWbd2FiwvMZpssbOGTFIYb+siTsJeYZ8/eD+W6+fODK3lMlarYKIDxlpZXIJiSSBqtWYcca+IYlZSuslWOU3VWCjF8YIQ5skWxj9zKMy4zlq+YcwS3uVKBGHsWXypmW0zsx1m1lbgeTOzf848/4yZvT/omMYrjkkA0lUpnbcXnFX6IJsU1trcwNqrz6UuVdqfaaUmAYjf315dqpZ1i5pCSQLn3PYQjW2doe7jEGgiMLNa4BvAZcD7gGvN7H15h10GzMp8LQa+GWRM4xXWQpdUzeRXnt6zaTeNbZ1adFal+gNcHGakr3Kzs4/i5OzPPxT5grOsUorITVRHdy+NbZ28fjD8Qf6gWwTnATvc/QV3PwR8H7gy75grgX/1tE1AvZmdHHBcYyrnvqPrFjVRXzd8yf/0aSnWLWpi+5cW8tvMCtT5Zxw3qfd5/eAgjW2dnP/FRyb1OhIP2UVGQXLgh0/3snTB7NgVO3zz0CA3xaDFe+KxU0NJAhNd/1HOVdFBzxpqAF7Mub8HOH8cxzQAL+UeZGaLSbcYmDlzZtkDzVeumvPzzzhu3IuPcgfIJjM+8fIbhzhr+Qae/2LwMxskOEHvopXVPzDILfdu5qb2nmHTIuPAebubKMhumWIzdKYYoSx0W7tx24Rmd73zqNqyzq4KukVQqLWZ/+8dzzG4+13u3uLuLTNmzChLcKMpx6Kcycx+WNU6Z1i9ml1rFk6okNhbgx75lZRMTpjrT7LjTuVIAvV1Ka6fO5PamvJ1Nt2zaXeg3UTrb5g3okU+/4zj2LE6nFbSRC48d61ZWPYptkG3CPYAp+XcPxXYW8IxoVu6YHbJG1cENb3ztg+fzS33bWZwnIOA33vyxYpZoCPDxaVvvBTHHDWFVa1zaHn3cSx/YAtvlliRM98t9/YE2kUT5aZQ490xbdYJxwTy/kG3CJ4CZpnZ6WY2FbgGeDDvmAeBv87MHpoLvObuL+W/UNhamxsm3GefvXIP6sPa2tzAV68+lynjvNLShuaVq5xjVGHr7evn9GWdLGnvKVsSgPReBuUsqxAnSxfMHnOwPsjifIEmAnc/DHwK2Ag8B9zr7lvN7EYzuzFz2AbgBWAHcDfw90HGNBHrb5jH9XPHNx4x2YHe8WptbmDHly4f10BRrcVtHoiMV9j7IpdbUNcgj+98paJbS8W0NjfwtUVNTK0d/jebqmGorHmQFVq1oGyczrntoaLTuqLaa3isFZFhL8qR8pm/5lHVqBqFVteXRiuLq9iKji2s37R72Ai7kkBl6+junVRJ6mp31JQatq26LOowKo72I6hiq1rn6KQvo6o1m/SY0fRpqUhq5Rdy8PAROrp71SooE+1HIBJDazduK1tr4Pq5M9m5+vIRixonosbgYMzKWi9p7+G9yzdU5ZhB2JQIRGKoXOMDuV2EK684m1SJc/uPOByI4T7IhwadJe09XHzHL6IOpaKpa0gkhsrRlQPDV+Pm7lNQbQPR2/e9yVnLN/CuY4+mt69/qBw3pFszf3W+xsxGoxaBSAyVIwk0FFgd39rcwNIFs6tyavFbgz6U4HJ/e0c8vTJZK+2LU4tApAqlaowLz5rB/DWPDu1eduFZM/jJ5pfo64/HgG/YKmkrzLApEYhUoalTaoYVLezt649djX+JDyUCmZBCuyZpzUJ8pGoMjLKWdpDqpzECGbdiW+fds2k3Zy4b31Z/MraO7l5KmdxjBu84egoDIe5sJdVBiUDGbbSt8w57et/XM5Z1al73JJW6huC682fSF5MFX1JZ1DUkYxrvxt6QrhC5pL2Hbzy2PdAiWdWs1IJznc+8RE2Zpp1KsqhFIKOaSBLItX3fmzS2dTLrVrUQJqp+AhsQ5Xr1wICSgJRELQIJ1MCRdAshu8mPWboLQ4PLxelcHhxt4VqYEoGEyjOLe3KnMtbXpVh5xdkqIEZ6oDip8/zDMNo4V5KpDLWMqtSuoVKkatItiFwN9XUsXTA7EUmio7uXZfdvCWXD+kJSNYDZsFlH2VINDZkFaY89v39ogVrju+pG3Q8jrpJ84RF6GWozWwt8GDgE7AT+xt37Chy3C3gDGAQOFwpSkqFQTbPevn5uau+h63evDOtO6ujuZeWDW0e9eq4149rzTxv6uY7uXtZu3DZ0Iotbglm7cVugSWCsk/qmF14dth/2WEl4/ppHA4s1SH39A0NdlXH6/49SYC0CM7sEeNTdD5vZPwG4++cKHLcLaHH334/3tdUiCFeYrYLRTEvV8KWrzgFg6X2bGZjAHMv6uhRvHjo87Gq3LlXL6qvmjHoymEjymGyiOb2tk6Db5/n/5nQr5Bn6C2ThsRYKhhFvkGoNdq5eGHUYoSrWIghs1pC7P5zZsxhgE3BqUO8lwdq1ZmFoezKP5sDAEW6+t4eVD26dUBKA9FVg/kKr/oFBbmrvKTqrKdtV09vXj5NunSy7f0vB4ydybDGnFCgSV279A4Os3bgNyO2KKlxe+p5Nu0eNP4x4g6ThgreFNVj8t0B7kecceNjMHPhf7n5XoYPMbDGwGGDmzPFtKC/ls/6GeXR09w41qaNyxCnrYKqTbl3AyG6ClQ9uHdFVkz2R5h9bqFsn96RbqKWQ34K48KwZ/PDp3sDHCLLrFMbTFbXs/rcrdubH2nfgUKBxSngm1TVkZj8DTirw1HJ3/1HmmOVAC3CVF3gzMzvF3fea2QnAI8Cn3f3fRntfdQ1F78xlnRyuoiuq6dNSTJs6hb19/Rydqil6lZyrIdO3/n93vlKWLhIDzjzhGHbufzPQvYob6ut4vO0D4+7aqUvV8NbAkYruBipm3aKmRI0TBNI15O4XufufFvjKJoGPAx8CriuUBDKvsTfzfR/wAHDeZGKScOxYvZBZJxwTdRhl8+qBgaFunfEkAUh3/zxepiQA6dbJ9n3BJgGApQtmA+Pv2umv0iQAcOv9z0QdQiwENkZgZpcCnwOucPcDRY45xsyOzd4GLgGeDSomKa9Hbr6AXWsWUn1bnFS3Je09NN3+sLp2iOf2m1EIssTEncCxwCNm1mNm34J0V5CZbcgccyLwSzPbDPwK6HT3hwKMSQLw25gMJsv49fUPqFS1DAlssNjdzyzy+F7g8sztF4Bzg4pBwrP+hnnD7q/o2KKNUKQidHT3JmqcoBAVnZNArGqdw641C7l+rmZ4SbytfHBr1CFETolAArWqdQ7rFjVx1BR91CSeVNtJiUBC0NrcwLZVl7FuURP1daWVWBaR4Kj6qISmtbkhr7RBdAXWRLKml7j/QzVRIpBIZBPC2o3b6C1xRy6Rcrjtw2dHHULk1DUkkWltbuDxtg+wblFT1KFIQk0xVSAFJQKJgdbmBq1DkEjsSFj10WKUCCQW1t8wT1NNJVRqib5NO5RJrF18xy/Yvu/NqMOQKmLA1xJWbC4r9B3KRMrhkZsvANKzjG65t0c15KVkNQZ/df7om+0klRKBVIT8qadLf7B5xEYzcTH/jOP41W9fnfDmORKMsXZaEyUCqUC5U0+zG6UcOHSYVw9Ev0I0e9IZz57KEqwTj53Kk8svjjqMiqAxAqkKcW4lHDWlhn/6r+eM6JOOc8yVzIDr1AooSGMEUtWyJ9nbf7y1LC2DWjMGy3SRdPBweq9lGD5nPb9lM96d0aSw+roUK684O5GDwJOlFoFUpTiWwc5uETmWFR1b+N6TLzLoTq0Z155/WtGr247uXm5u7yFJ6WNKjfGVq8/VCb8ExVoESgRStTq6e7mpvaekbRbL2SLIMtKb+AThuruf4PGdrwTy2nGigd/JCb1ryMxWAjcA+zMP3eruGwocdynwdaAW+La7rwkqJkmW7EyjibYOamuMa887jfanXixr//149wguRf7GQB3dvcO6yQyGJcRaM+a+Zzpb974xbEC7vi7FkSNHeP1gvIoBKgEEK7AWQSYR/NHdvzLKMbXAfwAXA3uAp4Br3f03o722WgQyWcW6VI6ZWssXPzKH1uYGOrp7Wf7AlrJs6VhjcMdHq2MRU27XVX6Cmajp01L0HRjglPo6li6YXRW/nzgLvWtonIlgHrDS3Rdk7i8DcPfVo722EoGEaTJdTFB81lC16ujuHTa1Vyf4+Ihq1tCnzOyvgS7gFnd/Ne/5BuDFnPt7gPMLvZCZLQYWA8ycqZo0Ep7cxWwyNv2+Ks+kis6Z2c/M7NkCX1cC3wTOAJqAl4CvFnqJAo8VvPBy97vcvcXdW2bMmDGZsEVEJMekWgTuftF4jjOzu4GfFHhqD3Bazv1Tgb2TiUlERCYmsDLUZnZyzt2PAM8WOOwpYJaZnW5mU4FrgAeDiklEREYKcozgy2bWRLqrZxfwdwBmdgrpaaKXu/thM/sUsJH09NHvuvvWAGMSEZE8gSUCd/9Ykcf3Apfn3N8AjFhfICIi4dAOZSIiCadEICKScEoEIiIJp0QgIpJwSgQiIgmnRCAiknBKBCIiCadEICKScEoEIiIJp0QgIpJwSgQiIgmnRCAiknBKBCIiCadEICKScEoEIiIJp0QgIpJwgW1MY2btwOzM3Xqgz92bChy3C3gDGAQOu3tLUDGJiMhIQe5Qtih728y+Crw2yuEXuvvvg4pFRESKC3LPYgDMzICPAh8I+r1ERGTiwhgj+HPgZXffXuR5Bx42s6fNbHGxFzGzxWbWZWZd+/fvDyRQEZEkmlSLwMx+BpxU4Knl7v6jzO1rge+N8jLz3X2vmZ0APGJmz7v7v+Uf5O53AXcBtLS0+GTiFhGRt00qEbj7RaM9b2ZTgKuA/zzKa+zNfN9nZg8A5wEjEoGIiAQj6K6hi4Dn3X1PoSfN7BgzOzZ7G7gEeDbgmEREJEfQieAa8rqFzOwUM9uQuXsi8Esz2wz8Cuh094cCjklERHIEOmvI3f9bgcf2Apdnbr8AnBtkDCIiMjqtLBYRSTglAhGRhFMiEBFJOCUCEZGEUyIQEUm4wGsNiUj0Gts6Rzx2/dyZrGqdE0E0EjdqEYhUuUJJAOCeTbtZ0bEl5GgkjpQIRKpYsSSQdc+m3SFFInGmRCCScGoViBKBSMKpVSBKBCIiCadEIFKlOrp7ow5BKoQSgUiV+uwPNkcdglQIJQKRKnVoUBv5yfgoEYgk3LpFTVGHIBFTIhCpQuOdEnr93Jm0NjcEHI3E3aQSgZldbWZbzeyImbXkPbfMzHaY2TYzW1Dk548zs0fMbHvm+/TJxCMiaeOdEqoSEwKTbxE8S3pz+mGbzZvZ+0hvU3k2cCnwP82stsDPtwE/d/dZwM8z90VkEjRbSCZqUonA3Z9z920FnroS+L67H3T33wI7gPOKHPcvmdv/ArROJh4RgbUbC/1JjtRQXxdwJFIpghojaABezLm/J/NYvhPd/SWAzPcTir2gmS02sy4z69q/f39ZgxWpJnv7+sd13NIFswOORCrFmInAzH5mZs8W+LpytB8r8Nik5rK5+13u3uLuLTNmzJjMS4lUtVPGeaWvQWLJGnM/Ane/qITX3QOclnP/VGBvgeNeNrOT3f0lMzsZ2FfCe4lIjqULZrOkvSfqMKSCBNU19CBwjZkdZWanA7OAXxU57uOZ2x8HfhRQPCKJoSt9majJTh/9iJntAeYBnWa2EcDdtwL3Ar8BHgI+6e6DmZ/5ds5U0zXAxWa2Hbg4c19EJkmLxGQizL3ylqG3tLR4V1dX1GGIxNZZyzfw1iglJtYtalLLIYHM7Gl3b8l/XCuLRarQaEkA1H0kwykRiIgknBKBiEjCKRGIJMyJx06NOgSJGSUCkYR5cvnFUYcgMaNEIFKFik0f1bRSKWTMlcUiUnmys4LWbtzG3r5+TqmvY+mC2ZotJAUpEYhUqdbmBp34ZVzUNSQiknBKBCIiCadEICKScEoEIiIJp0QgIpJwFVl91Mz2A78r88seD/y+zK8ZBMVZPpUQIyjOcquEOIOK8d3uPmKLx4pMBEEws65C5VnjRnGWTyXECIqz3CohzrBjVNeQiEjCKRGIiCScEsHb7oo6gHFSnOVTCTGC4iy3Sogz1Bg1RiAiknBqEYiIJJwSgYhIwikR5DCzJjPbZGY9ZtZlZudFHVMxZvZpM9tmZlvN7MtRx1OMmf2DmbmZHR91LIWY2Voze97MnjGzB8ysPuqYcpnZpZn/5x1m1hZ1PPnM7DQze8zMnst8Fj8TdUyjMbNaM+s2s59EHUsxZlZvZj/IfC6fM7N5Qb+nEsFwX94EsVQAAAM+SURBVAZud/cm4POZ+7FjZhcCVwLnuPvZwFciDqkgMzsNuBjYHXUso3gE+FN3Pwf4D2BZxPEMMbNa4BvAZcD7gGvN7H3RRjXCYeAWd/8TYC7wyRjGmOszwHNRBzGGrwMPuftZwLmEEK8SwXAOvDNz+z8BeyOMZTSfANa4+0EAd98XcTzFfA34LOnfayy5+8PufjhzdxNwapTx5DkP2OHuL7j7IeD7pC8AYsPdX3L3X2duv0H6pBXLTRDM7FRgIfDtqGMpxszeCfwF8B0Adz/k7n1Bv68SwXBLgLVm9iLpq+zYXB3meS/w52b2pJn9HzP7s6gDymdmVwC97r456lgm4G+Bn0YdRI4G4MWc+3uI6UkWwMwagWbgyWgjKWod6QuTI1EHMor3APuB/53pwvq2mR0T9JsmbocyM/sZcFKBp5YDHwRucvcfmtlHSWfli8KML2uMOKcA00k3xf8MuNfM3uMhzwUeI8ZbgUvCjKeY0eJ09x9ljllOuptjfZixjcEKPBbL1pWZvQP4IbDE3V+POp58ZvYhYJ+7P21mF0QdzyimAO8HPu3uT5rZ14E24B+DfFOtI8hhZq8B9e7uZmbAa+7+zrF+Lmxm9hDprqFfZO7vBOa6+/5IA8swsznAz4EDmYdOJd3Ndp67/7/IAivCzD4O3Ah80N0PjHV8WDKDhCvdfUHm/jIAd18daWB5zCwF/ATY6O53RB1PIWa2GvgY6WR/NOku4Pvd/fpIA8tjZicBm9y9MXP/z4E2d18Y5Puqa2i4vcB/ydz+ALA9wlhG00E6PszsvcBUYlRN0d23uPsJ7t6Y+UDvAd4f0yRwKfA54Io4JYGMp4BZZna6mU0FrgEejDimYTIXTN8BnotrEgBw92Xufmrm83gN8GjckgBA5m/kRTObnXnog8Bvgn7fxHUNjeEG4OtmNgV4C1gccTzFfBf4rpk9CxwCPh52t1AVuRM4CngkfU5jk7vfGG1Iae5+2Mw+BWwEaoHvuvvWiMPKN5/0lfYWM+vJPHaru2+IMKZK92lgfSb5vwD8TdBvqK4hEZGEU9eQiEjCKRGIiCScEoGISMIpEYiIJJwSgYhIwikRiIgknBKBiEjC/X8Z5zBigu5RnAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(umap_embeddings[:,0], umap_embeddings[:,1])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "artist_uri             2495.000000\n",
       "pid_y                 61747.000000\n",
       "num_followers_norm        0.693147\n",
       "Name: 50000, dtype: float64"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "collab_filter_train.to_pandas().iloc[50000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cuml.neighbors import NearestNeighbors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [],
   "source": [
    "nn = NearestNeighbors(n_neighbors=15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NearestNeighbors(n_neighbors=15, n_gpus=1, devices=None, verbose=False, handle=<cuml.common.handle.Handle object at 0x7f07bb27e060>)"
      ]
     },
     "execution_count": 206,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nn.fit(embeds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [],
   "source": [
    "dists, inds = nn.kneighbors(embeds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1215., 5738., 4894., 4700., 4493., 1494.,  512., 5355., 1619., 4744., 5857., 4119., 2784., 4939.,   40.])"
      ]
     },
     "execution_count": 232,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inds[1215]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(993628,)"
      ]
     },
     "execution_count": 245,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "to_np(learn.model.i_bias.weight)[:,0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}