getting_started_notebooks/intro_tutorials/02_Introduction_to_cuDF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"introduction\"></a>\n",
    "## Introduction to cuDF\n",
    "#### By Paul Hendricks\n",
    "-------\n",
    "\n",
    "In this notebook, we will show how to work with cuDF DataFrames in RAPIDS.\n",
    "\n",
    "**Table of Contents**\n",
    "\n",
    "* [Introduction to cuDF](#introduction)\n",
    "* [Setup](#setup)\n",
    "* [cuDF Series Basics](#series)\n",
    "* [cuDF DataFrame Basics](#dataframes)\n",
    "* [Input/Output](#io)\n",
    "* [cuDF API](#cudfapi)\n",
    "* [Conclusion](#conclusion)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"setup\"></a>\n",
    "## Setup\n",
    "\n",
    "This notebook was tested using the following Docker containers:\n",
    "\n",
    "* `rapidsai/rapidsai-dev-nightly:0.12-cuda10.0-runtime-ubuntu16.04-py3.6` container from [DockerHub](https://hub.docker.com/r/rapidsai/rapidsai-nightly)\n",
    "\n",
    "This notebook was run on the NVIDIA GV100 GPU. Please be aware that your system may be different and you may need to modify the code or install packages to run the below examples. \n",
    "\n",
    "If you think you have found a bug or an error, please file an issue here: https://github.com/rapidsai/notebooks-contrib/issues\n",
    "\n",
    "Before we begin, let's check out our hardware setup by running the `nvidia-smi` command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's see what CUDA version we have:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvcc --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"series\"></a>\n",
    "## cuDF Series Basics\n",
    "\n",
    "First, let's load the cuDF library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuDF Version: 0.12.0b+1777.g162097c\n"
     ]
    }
   ],
   "source": [
    "import cudf; print('cuDF Version:', cudf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are two main data structures in cuDF: a `Series` object and a `DataFrame` object. Multiple `Series` objects are used as columns for a `DataFrame`. We'll first explore the `Series` class and build upon that foundation to later introduce how to work with objects of type `DataFrame`.\n",
    "\n",
    "We can create a `Series` object using the `cudf.Series` class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    11\n",
       "2    12\n",
       "3    13\n",
       "dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "column = cudf.Series([10, 11, 12, 13])\n",
    "column"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see from the output that `column` is an object of type `cudf.Series` and has 4 rows.\n",
    "\n",
    "Another way to inspect a `Series` is to use the Python `print` statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    10\n",
      "1    11\n",
      "2    12\n",
      "3    13\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(column)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that our `Series` object has four rows with values 10, 11, 12, and 13. We also see that the type of this data is `int64`. There are several ways to represent data using cuDF. The most common formats are `int8`, `int32`, `int64`, `float32`, and `float64`.\n",
    "\n",
    "We also see a column of values on the left hand side with values 0, 1, 2, 3. These values represent the index of the `Series`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RangeIndex(start=0, stop=4)\n"
     ]
    }
   ],
   "source": [
    "print(column.index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a new column with a different index by using the `set_index` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5    10\n",
      "6    11\n",
      "7    12\n",
      "8    13\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "new_column = column.set_index([5, 6, 7, 8]) \n",
    "print(new_column)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indexes are useful for operations like joins and groupbys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"dataframes\"></a>\n",
    "## cuDF DataFrame Basics\n",
    "\n",
    "As we showed in the previous tutorial, cuDF DataFrames are a tabular structure of data that reside on the GPU. We interface with these cuDF DataFrames in the same way we interface with Pandas DataFrames that reside on the CPU - with a few deviations.\n",
    "\n",
    "In the next several sections, we'll show how to create and manipulate cuDF DataFrames. For more information on using cuDF DataFrames, check out the documentation: https://docs.rapids.ai/api/cudf/stable/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating a cudf DataFrame using lists\n",
    "\n",
    "There are several ways to create a cuDF DataFrame. The easiest of these is to instantiate an empty cuDF DataFrame and then use Python list objects or NumPy arrays to create columns. Below, we create an empty cuDF DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Empty DataFrame\n",
      "Columns: []\n",
      "Index: []\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame()\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can create two columns named `key` and `value` by using the bracket notation with the cuDF DataFrame and storing either a list of Python values or a NumPy array into that column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NumPy Version: 1.17.5\n",
      "   key  value\n",
      "0    0     10\n",
      "1    1     11\n",
      "2    2     12\n",
      "3    3     13\n",
      "4    4     14\n"
     ]
    }
   ],
   "source": [
    "import numpy as np; print('NumPy Version:', np.__version__)\n",
    "\n",
    "\n",
    "# here we create two columns named \"key\" and \"value\"\n",
    "df['key'] = [0, 1, 2, 3, 4]\n",
    "df['value'] = np.arange(10, 15)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating a cudf DataFrame using a list of tuples or a dictionary\n",
    "\n",
    "Another way we can create a cuDF DataFrame is by providing a mapping of column names to column values, either via a list of tuples or by using a dictionary. In the below examples, we create a list of two-value tuples; the first value is the name of the column - for example, `id` or `timestamp` - and the second value is a list of Python objects or Numpy arrays. Note that we don't have to constrain the data stored in our cuDF DataFrames to common data types like integers or floats - we can use more exotic data types such as datetimes or strings. We'll investigate how such data types behave on the GPU a bit later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime, timedelta\n",
    "\n",
    "\n",
    "ids = np.arange(5)\n",
    "t0 = datetime.strptime('2018-10-07 12:00:00', '%Y-%m-%d %H:%M:%S')\n",
    "timestamps = [(t0+ timedelta(seconds=x)) for x in range(5)]\n",
    "timestamps_np = np.array(timestamps, dtype='datetime64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   ids           timestamp\n",
      "0    0 2018-10-07 12:00:00\n",
      "1    1 2018-10-07 12:00:01\n",
      "2    2 2018-10-07 12:00:02\n",
      "3    3 2018-10-07 12:00:03\n",
      "4    4 2018-10-07 12:00:04\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame()\n",
    "df['ids'] = ids\n",
    "df['timestamp'] = timestamps_np\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can create a dictonary of key-value pairs, where each key in the dictionary represents a column name and each value associated with the key represents the values that belong in that column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id           timestamp\n",
      "0   0 2018-10-07 12:00:00\n",
      "1   1 2018-10-07 12:00:01\n",
      "2   2 2018-10-07 12:00:02\n",
      "3   3 2018-10-07 12:00:03\n",
      "4   4 2018-10-07 12:00:04\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame({'id': ids, 'timestamp': timestamps_np})\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating a cudf DataFrame from a Pandas DataFrame\n",
    "\n",
    "Pandas DataFrames are a first class citizen within cuDF - this means that we can create a cuDF DataFrame from a Pandas DataFrame and vice versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pandas Version: 0.24.2\n",
      "     a    b\n",
      "0    0  0.0\n",
      "1    1  0.1\n",
      "2    2  0.2\n",
      "3    3  NaN\n",
      "4    4  0.4\n",
      "5    5  0.5\n",
      "6    6  0.6\n",
      "7    7  0.7\n",
      "8    8  0.8\n",
      "9    9  0.9\n",
      "10  10  1.0\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd; print('Pandas Version:', pd.__version__)\n",
    "\n",
    "\n",
    "pandas_df = pd.DataFrame({'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],\n",
    "                          'b': [0.0, 0.1, 0.2, None, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]})\n",
    "print(pandas_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the `cudf.from_pandas` or `cudf.DataFrame.from_pandas` functions to create a cuDF DataFrame from a Pandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     a     b\n",
      "0    0   0.0\n",
      "1    1   0.1\n",
      "2    2   0.2\n",
      "3    3  null\n",
      "4    4   0.4\n",
      "5    5   0.5\n",
      "6    6   0.6\n",
      "7    7   0.7\n",
      "8    8   0.8\n",
      "9    9   0.9\n",
      "10  10   1.0\n"
     ]
    }
   ],
   "source": [
    "df = cudf.from_pandas(pandas_df)\n",
    "# df = cudf.DataFrame.from_pandas(pandas_df)  # alternative\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating a cuDF DataFrame from cuDF Series\n",
    "\n",
    "We can create a cuDF DataFrame from one or more cuDF Series objects by passing the Series objects in a dictionary mapping each Series object to a column name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b   c\n",
      "0  1  5   9\n",
      "1  2  6  10\n",
      "2  3  7  11\n",
      "3  4  8  12\n"
     ]
    }
   ],
   "source": [
    "column1 = cudf.Series([1, 2, 3, 4])\n",
    "column2 = cudf.Series([5, 6, 7, 8])\n",
    "column3 = cudf.Series([9, 10, 11, 12])\n",
    "df = cudf.DataFrame({'a': column1, 'b': column2, 'c': column3})\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Inspecting a cuDF DataFrame\n",
    "\n",
    "There are several ways to inspect a cuDF DataFrame. The first method is to enter the cuDF DataFrame directly into the REPL. This shows us information about the type of the object, and metadata such as the number of rows or columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.arange(0, 100), 'b': np.arange(100, 0, -1)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>24</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>25</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>26</td>\n",
       "      <td>74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>27</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>28</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>29</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>70</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>71</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>72</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>73</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>74</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>75</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>76</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>77</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>78</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>79</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>80</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>81</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>82</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>83</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>84</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>85</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>86</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>87</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>88</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>89</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>90</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>91</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>92</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>93</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>94</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>95</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>96</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>97</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>98</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     a    b\n",
       "0    0  100\n",
       "1    1   99\n",
       "2    2   98\n",
       "3    3   97\n",
       "4    4   96\n",
       "5    5   95\n",
       "6    6   94\n",
       "7    7   93\n",
       "8    8   92\n",
       "9    9   91\n",
       "10  10   90\n",
       "11  11   89\n",
       "12  12   88\n",
       "13  13   87\n",
       "14  14   86\n",
       "15  15   85\n",
       "16  16   84\n",
       "17  17   83\n",
       "18  18   82\n",
       "19  19   81\n",
       "20  20   80\n",
       "21  21   79\n",
       "22  22   78\n",
       "23  23   77\n",
       "24  24   76\n",
       "25  25   75\n",
       "26  26   74\n",
       "27  27   73\n",
       "28  28   72\n",
       "29  29   71\n",
       "..  ..  ...\n",
       "70  70   30\n",
       "71  71   29\n",
       "72  72   28\n",
       "73  73   27\n",
       "74  74   26\n",
       "75  75   25\n",
       "76  76   24\n",
       "77  77   23\n",
       "78  78   22\n",
       "79  79   21\n",
       "80  80   20\n",
       "81  81   19\n",
       "82  82   18\n",
       "83  83   17\n",
       "84  84   16\n",
       "85  85   15\n",
       "86  86   14\n",
       "87  87   13\n",
       "88  88   12\n",
       "89  89   11\n",
       "90  90   10\n",
       "91  91    9\n",
       "92  92    8\n",
       "93  93    7\n",
       "94  94    6\n",
       "95  95    5\n",
       "96  96    4\n",
       "97  97    3\n",
       "98  98    2\n",
       "99  99    1\n",
       "\n",
       "[100 rows x 2 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A second way to inspect a cuDF DataFrame is to wrap the object in a Python `print` function. This results in showing the rows and columns of the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     a    b\n",
      "0    0  100\n",
      "1    1   99\n",
      "2    2   98\n",
      "3    3   97\n",
      "4    4   96\n",
      "5    5   95\n",
      "6    6   94\n",
      "7    7   93\n",
      "8    8   92\n",
      "9    9   91\n",
      "10  10   90\n",
      "11  11   89\n",
      "12  12   88\n",
      "13  13   87\n",
      "14  14   86\n",
      "15  15   85\n",
      "16  16   84\n",
      "17  17   83\n",
      "18  18   82\n",
      "19  19   81\n",
      "20  20   80\n",
      "21  21   79\n",
      "22  22   78\n",
      "23  23   77\n",
      "24  24   76\n",
      "25  25   75\n",
      "26  26   74\n",
      "27  27   73\n",
      "28  28   72\n",
      "29  29   71\n",
      "..  ..  ...\n",
      "70  70   30\n",
      "71  71   29\n",
      "72  72   28\n",
      "73  73   27\n",
      "74  74   26\n",
      "75  75   25\n",
      "76  76   24\n",
      "77  77   23\n",
      "78  78   22\n",
      "79  79   21\n",
      "80  80   20\n",
      "81  81   19\n",
      "82  82   18\n",
      "83  83   17\n",
      "84  84   16\n",
      "85  85   15\n",
      "86  86   14\n",
      "87  87   13\n",
      "88  88   12\n",
      "89  89   11\n",
      "90  90   10\n",
      "91  91    9\n",
      "92  92    8\n",
      "93  93    7\n",
      "94  94    6\n",
      "95  95    5\n",
      "96  96    4\n",
      "97  97    3\n",
      "98  98    2\n",
      "99  99    1\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For very large dataframes, we often want to see the first couple rows. We can use the `head` method of a cuDF DataFrame to view the first N rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a    b\n",
      "0  0  100\n",
      "1  1   99\n",
      "2  2   98\n",
      "3  3   97\n",
      "4  4   96\n"
     ]
    }
   ],
   "source": [
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Columns\n",
    "\n",
    "cuDF DataFrames store metadata such as information about columns or data types. We can access the columns of a cuDF DataFrame using the `.columns` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['a', 'b'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(df.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can modify the columns of a cuDF DataFrame by modifying the `columns` attribute. We can do this by setting that attribute equal to a list of strings representing the new columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['c', 'd'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "df.columns = ['c', 'd']\n",
    "print(df.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Types\n",
    "\n",
    "We can also inspect the data types of the columns of a cuDF DataFrame using the `dtypes` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c    int64\n",
      "d    int64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can modify the data types of the columns of a cuDF DataFrame by passing in a cuDF Series with a modified data type. Be warned that silent errors may be introduced from nonsensical type conversations - for example, changing a float to an integer or vice versa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c    float32\n",
      "d      int32\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "df['c'] = df['c'].astype(np.float32)\n",
    "df['d'] = df['d'].astype(np.int32)\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Series\n",
    "\n",
    "cuDF DataFrames are composed of rows and columns. Each column is represented using an object of type `Series`. For example, if we subset a cuDF DataFrame using just one column we will be returned an object of type `cudf.dataframe.series.Series`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'cudf.core.series.Series'>\n",
      "0      0.0\n",
      "1      1.0\n",
      "2      2.0\n",
      "3      3.0\n",
      "4      4.0\n",
      "5      5.0\n",
      "6      6.0\n",
      "7      7.0\n",
      "8      8.0\n",
      "9      9.0\n",
      "10    10.0\n",
      "11    11.0\n",
      "12    12.0\n",
      "13    13.0\n",
      "14    14.0\n",
      "15    15.0\n",
      "16    16.0\n",
      "17    17.0\n",
      "18    18.0\n",
      "19    19.0\n",
      "20    20.0\n",
      "21    21.0\n",
      "22    22.0\n",
      "23    23.0\n",
      "24    24.0\n",
      "25    25.0\n",
      "26    26.0\n",
      "27    27.0\n",
      "28    28.0\n",
      "29    29.0\n",
      "      ... \n",
      "70    70.0\n",
      "71    71.0\n",
      "72    72.0\n",
      "73    73.0\n",
      "74    74.0\n",
      "75    75.0\n",
      "76    76.0\n",
      "77    77.0\n",
      "78    78.0\n",
      "79    79.0\n",
      "80    80.0\n",
      "81    81.0\n",
      "82    82.0\n",
      "83    83.0\n",
      "84    84.0\n",
      "85    85.0\n",
      "86    86.0\n",
      "87    87.0\n",
      "88    88.0\n",
      "89    89.0\n",
      "90    90.0\n",
      "91    91.0\n",
      "92    92.0\n",
      "93    93.0\n",
      "94    94.0\n",
      "95    95.0\n",
      "96    96.0\n",
      "97    97.0\n",
      "98    98.0\n",
      "99    99.0\n",
      "Name: c, Length: 100, dtype: float32\n"
     ]
    }
   ],
   "source": [
    "print(type(df['c']))\n",
    "print(df['c'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Index\n",
    "\n",
    "Like `Series` objects, each `DataFrame` has an index attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=100)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use the index values to subset the `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     c   d\n",
      "2  2.0  98\n"
     ]
    }
   ],
   "source": [
    "print(df[df.index == 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting a cudf DataFrame to a Pandas DataFrame\n",
    "\n",
    "We can convert a cuDF DataFrame back to a Pandas DataFrame using the `to_pandas` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n"
     ]
    }
   ],
   "source": [
    "pandas_df = df.to_pandas()\n",
    "print(type(pandas_df))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting a cudf DataFrame to a NumPy Array\n",
    "\n",
    "Often we want to work with NumPy arrays. We can convert a cuDF DataFrame to a NumPy array by first converting it to a Pandas DataFrame using the `to_pandas` method followed by accessing the `values` attribute of the Pandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "numpy_array = df.to_pandas().values\n",
    "print(type(numpy_array))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Converting a cudf DataFrame to Other Data Formats\n",
    "\n",
    "We can also convert a cuDF DataFrame to other data formats. \n",
    "\n",
    "For more information, see the documentation: https://docs.rapids.ai/api/cudf/stable/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"io\"></a>\n",
    "## Input/Output\n",
    "\n",
    "Before we process data and use it in machine learning models, we need to be able to load it into memory and write it after we're done using it. There are several ways to do this using cuDF."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Writing and Loading CSV Files\n",
    "\n",
    "At this time, there is no direct way to use to cuDF to write directly to CSV. However, we can conver the cuDF DataFrame to a Pandas DataFrame and then write it directly to a CSV."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_pandas().to_csv('./dataset.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perhaps one of the most common ways to create cuDF DataFrames is by loading a table that is stored as a file on disk. cuDF provides a lot of functionality for reading in a variety of different data formats. Below, we show how easy it is to read in a CSV file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       c    d\n",
      "0    0.0  100\n",
      "1    1.0   99\n",
      "2    2.0   98\n",
      "3    3.0   97\n",
      "4    4.0   96\n",
      "5    5.0   95\n",
      "6    6.0   94\n",
      "7    7.0   93\n",
      "8    8.0   92\n",
      "9    9.0   91\n",
      "10  10.0   90\n",
      "11  11.0   89\n",
      "12  12.0   88\n",
      "13  13.0   87\n",
      "14  14.0   86\n",
      "15  15.0   85\n",
      "16  16.0   84\n",
      "17  17.0   83\n",
      "18  18.0   82\n",
      "19  19.0   81\n",
      "20  20.0   80\n",
      "21  21.0   79\n",
      "22  22.0   78\n",
      "23  23.0   77\n",
      "24  24.0   76\n",
      "25  25.0   75\n",
      "26  26.0   74\n",
      "27  27.0   73\n",
      "28  28.0   72\n",
      "29  29.0   71\n",
      "..   ...  ...\n",
      "70  70.0   30\n",
      "71  71.0   29\n",
      "72  72.0   28\n",
      "73  73.0   27\n",
      "74  74.0   26\n",
      "75  75.0   25\n",
      "76  76.0   24\n",
      "77  77.0   23\n",
      "78  78.0   22\n",
      "79  79.0   21\n",
      "80  80.0   20\n",
      "81  81.0   19\n",
      "82  82.0   18\n",
      "83  83.0   17\n",
      "84  84.0   16\n",
      "85  85.0   15\n",
      "86  86.0   14\n",
      "87  87.0   13\n",
      "88  88.0   12\n",
      "89  89.0   11\n",
      "90  90.0   10\n",
      "91  91.0    9\n",
      "92  92.0    8\n",
      "93  93.0    7\n",
      "94  94.0    6\n",
      "95  95.0    5\n",
      "96  96.0    4\n",
      "97  97.0    3\n",
      "98  98.0    2\n",
      "99  99.0    1\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "df = cudf.read_csv('./dataset.csv')\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CSV files come in many flavors and cuDF tries to be as flexible as possible, mirroring the Pandas API wherever possible. For more information on possible parameters for working with files, see the cuDF IO documentation: \n",
    "\n",
    "https://rapidsai.github.io/projects/cudf/en/stable/api.html#cudf.io.csv.read_csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"cudfapi\"></a>\n",
    "## cuDF API\n",
    "\n",
    "The cuDF API is pleasantly simple and mirrors the Pandas API as closely as possible. In this section, we will explore the cuDF API and show how to perform common data manipulation operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Selecting Rows or Columns\n",
    "\n",
    "We can select rows from a cuDF DataFrame using slicing syntax. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.arange(0, 100).astype(np.float32), \n",
    "                     'b': np.arange(100, 0, -1).astype(np.float32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     a      b\n",
      "0  0.0  100.0\n",
      "1  1.0   99.0\n",
      "2  2.0   98.0\n",
      "3  3.0   97.0\n",
      "4  4.0   96.0\n"
     ]
    }
   ],
   "source": [
    "print(df[0:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several ways to select a column from a cuDF DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0      0.0\n",
      "1      1.0\n",
      "2      2.0\n",
      "3      3.0\n",
      "4      4.0\n",
      "5      5.0\n",
      "6      6.0\n",
      "7      7.0\n",
      "8      8.0\n",
      "9      9.0\n",
      "10    10.0\n",
      "11    11.0\n",
      "12    12.0\n",
      "13    13.0\n",
      "14    14.0\n",
      "15    15.0\n",
      "16    16.0\n",
      "17    17.0\n",
      "18    18.0\n",
      "19    19.0\n",
      "20    20.0\n",
      "21    21.0\n",
      "22    22.0\n",
      "23    23.0\n",
      "24    24.0\n",
      "25    25.0\n",
      "26    26.0\n",
      "27    27.0\n",
      "28    28.0\n",
      "29    29.0\n",
      "      ... \n",
      "70    70.0\n",
      "71    71.0\n",
      "72    72.0\n",
      "73    73.0\n",
      "74    74.0\n",
      "75    75.0\n",
      "76    76.0\n",
      "77    77.0\n",
      "78    78.0\n",
      "79    79.0\n",
      "80    80.0\n",
      "81    81.0\n",
      "82    82.0\n",
      "83    83.0\n",
      "84    84.0\n",
      "85    85.0\n",
      "86    86.0\n",
      "87    87.0\n",
      "88    88.0\n",
      "89    89.0\n",
      "90    90.0\n",
      "91    91.0\n",
      "92    92.0\n",
      "93    93.0\n",
      "94    94.0\n",
      "95    95.0\n",
      "96    96.0\n",
      "97    97.0\n",
      "98    98.0\n",
      "99    99.0\n",
      "Name: a, Length: 100, dtype: float32\n"
     ]
    }
   ],
   "source": [
    "print(df['a'])\n",
    "# print(df.a)  # alternative"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also select multiple columns by passing in a list of column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       a      b\n",
      "0    0.0  100.0\n",
      "1    1.0   99.0\n",
      "2    2.0   98.0\n",
      "3    3.0   97.0\n",
      "4    4.0   96.0\n",
      "5    5.0   95.0\n",
      "6    6.0   94.0\n",
      "7    7.0   93.0\n",
      "8    8.0   92.0\n",
      "9    9.0   91.0\n",
      "10  10.0   90.0\n",
      "11  11.0   89.0\n",
      "12  12.0   88.0\n",
      "13  13.0   87.0\n",
      "14  14.0   86.0\n",
      "15  15.0   85.0\n",
      "16  16.0   84.0\n",
      "17  17.0   83.0\n",
      "18  18.0   82.0\n",
      "19  19.0   81.0\n",
      "20  20.0   80.0\n",
      "21  21.0   79.0\n",
      "22  22.0   78.0\n",
      "23  23.0   77.0\n",
      "24  24.0   76.0\n",
      "25  25.0   75.0\n",
      "26  26.0   74.0\n",
      "27  27.0   73.0\n",
      "28  28.0   72.0\n",
      "29  29.0   71.0\n",
      "..   ...    ...\n",
      "70  70.0   30.0\n",
      "71  71.0   29.0\n",
      "72  72.0   28.0\n",
      "73  73.0   27.0\n",
      "74  74.0   26.0\n",
      "75  75.0   25.0\n",
      "76  76.0   24.0\n",
      "77  77.0   23.0\n",
      "78  78.0   22.0\n",
      "79  79.0   21.0\n",
      "80  80.0   20.0\n",
      "81  81.0   19.0\n",
      "82  82.0   18.0\n",
      "83  83.0   17.0\n",
      "84  84.0   16.0\n",
      "85  85.0   15.0\n",
      "86  86.0   14.0\n",
      "87  87.0   13.0\n",
      "88  88.0   12.0\n",
      "89  89.0   11.0\n",
      "90  90.0   10.0\n",
      "91  91.0    9.0\n",
      "92  92.0    8.0\n",
      "93  93.0    7.0\n",
      "94  94.0    6.0\n",
      "95  95.0    5.0\n",
      "96  96.0    4.0\n",
      "97  97.0    3.0\n",
      "98  98.0    2.0\n",
      "99  99.0    1.0\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "print(df[['a', 'b']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can select specific rows and columns using the slicing syntax as well as passing in a list of column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    0.0\n",
      "1    1.0\n",
      "2    2.0\n",
      "3    3.0\n",
      "4    4.0\n",
      "5    5.0\n",
      "Name: a, dtype: float32\n"
     ]
    }
   ],
   "source": [
    "print(df.loc[0:5, ['a']])\n",
    "# print(df.loc[0:5, ['a', 'b']])  # to select multiple columns, pass in multiple column names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Defining New Columns\n",
    "\n",
    "We often want to define new columns from existing columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.arange(0, 100).astype(np.float32), \n",
    "                     'b': np.arange(100, 0, -1).astype(np.float32), \n",
    "                     'c': np.arange(100, 200).astype(np.float32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       a      b      c      d\n",
      "0    0.0  100.0  100.0  200.0\n",
      "1    1.0   99.0  101.0  201.0\n",
      "2    2.0   98.0  102.0  202.0\n",
      "3    3.0   97.0  103.0  203.0\n",
      "4    4.0   96.0  104.0  204.0\n",
      "5    5.0   95.0  105.0  205.0\n",
      "6    6.0   94.0  106.0  206.0\n",
      "7    7.0   93.0  107.0  207.0\n",
      "8    8.0   92.0  108.0  208.0\n",
      "9    9.0   91.0  109.0  209.0\n",
      "10  10.0   90.0  110.0  210.0\n",
      "11  11.0   89.0  111.0  211.0\n",
      "12  12.0   88.0  112.0  212.0\n",
      "13  13.0   87.0  113.0  213.0\n",
      "14  14.0   86.0  114.0  214.0\n",
      "15  15.0   85.0  115.0  215.0\n",
      "16  16.0   84.0  116.0  216.0\n",
      "17  17.0   83.0  117.0  217.0\n",
      "18  18.0   82.0  118.0  218.0\n",
      "19  19.0   81.0  119.0  219.0\n",
      "20  20.0   80.0  120.0  220.0\n",
      "21  21.0   79.0  121.0  221.0\n",
      "22  22.0   78.0  122.0  222.0\n",
      "23  23.0   77.0  123.0  223.0\n",
      "24  24.0   76.0  124.0  224.0\n",
      "25  25.0   75.0  125.0  225.0\n",
      "26  26.0   74.0  126.0  226.0\n",
      "27  27.0   73.0  127.0  227.0\n",
      "28  28.0   72.0  128.0  228.0\n",
      "29  29.0   71.0  129.0  229.0\n",
      "..   ...    ...    ...    ...\n",
      "70  70.0   30.0  170.0  270.0\n",
      "71  71.0   29.0  171.0  271.0\n",
      "72  72.0   28.0  172.0  272.0\n",
      "73  73.0   27.0  173.0  273.0\n",
      "74  74.0   26.0  174.0  274.0\n",
      "75  75.0   25.0  175.0  275.0\n",
      "76  76.0   24.0  176.0  276.0\n",
      "77  77.0   23.0  177.0  277.0\n",
      "78  78.0   22.0  178.0  278.0\n",
      "79  79.0   21.0  179.0  279.0\n",
      "80  80.0   20.0  180.0  280.0\n",
      "81  81.0   19.0  181.0  281.0\n",
      "82  82.0   18.0  182.0  282.0\n",
      "83  83.0   17.0  183.0  283.0\n",
      "84  84.0   16.0  184.0  284.0\n",
      "85  85.0   15.0  185.0  285.0\n",
      "86  86.0   14.0  186.0  286.0\n",
      "87  87.0   13.0  187.0  287.0\n",
      "88  88.0   12.0  188.0  288.0\n",
      "89  89.0   11.0  189.0  289.0\n",
      "90  90.0   10.0  190.0  290.0\n",
      "91  91.0    9.0  191.0  291.0\n",
      "92  92.0    8.0  192.0  292.0\n",
      "93  93.0    7.0  193.0  293.0\n",
      "94  94.0    6.0  194.0  294.0\n",
      "95  95.0    5.0  195.0  295.0\n",
      "96  96.0    4.0  196.0  296.0\n",
      "97  97.0    3.0  197.0  297.0\n",
      "98  98.0    2.0  198.0  298.0\n",
      "99  99.0    1.0  199.0  299.0\n",
      "\n",
      "[100 rows x 4 columns]\n"
     ]
    }
   ],
   "source": [
    "df['d'] = np.arange(200, 300).astype(np.float32)\n",
    "\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       a      b      c      d      e\n",
      "0    0.0  100.0  100.0  200.0  300.0\n",
      "1    1.0   99.0  101.0  201.0  301.0\n",
      "2    2.0   98.0  102.0  202.0  302.0\n",
      "3    3.0   97.0  103.0  203.0  303.0\n",
      "4    4.0   96.0  104.0  204.0  304.0\n",
      "5    5.0   95.0  105.0  205.0  305.0\n",
      "6    6.0   94.0  106.0  206.0  306.0\n",
      "7    7.0   93.0  107.0  207.0  307.0\n",
      "8    8.0   92.0  108.0  208.0  308.0\n",
      "9    9.0   91.0  109.0  209.0  309.0\n",
      "10  10.0   90.0  110.0  210.0  310.0\n",
      "11  11.0   89.0  111.0  211.0  311.0\n",
      "12  12.0   88.0  112.0  212.0  312.0\n",
      "13  13.0   87.0  113.0  213.0  313.0\n",
      "14  14.0   86.0  114.0  214.0  314.0\n",
      "15  15.0   85.0  115.0  215.0  315.0\n",
      "16  16.0   84.0  116.0  216.0  316.0\n",
      "17  17.0   83.0  117.0  217.0  317.0\n",
      "18  18.0   82.0  118.0  218.0  318.0\n",
      "19  19.0   81.0  119.0  219.0  319.0\n",
      "20  20.0   80.0  120.0  220.0  320.0\n",
      "21  21.0   79.0  121.0  221.0  321.0\n",
      "22  22.0   78.0  122.0  222.0  322.0\n",
      "23  23.0   77.0  123.0  223.0  323.0\n",
      "24  24.0   76.0  124.0  224.0  324.0\n",
      "25  25.0   75.0  125.0  225.0  325.0\n",
      "26  26.0   74.0  126.0  226.0  326.0\n",
      "27  27.0   73.0  127.0  227.0  327.0\n",
      "28  28.0   72.0  128.0  228.0  328.0\n",
      "29  29.0   71.0  129.0  229.0  329.0\n",
      "..   ...    ...    ...    ...    ...\n",
      "70  70.0   30.0  170.0  270.0  370.0\n",
      "71  71.0   29.0  171.0  271.0  371.0\n",
      "72  72.0   28.0  172.0  272.0  372.0\n",
      "73  73.0   27.0  173.0  273.0  373.0\n",
      "74  74.0   26.0  174.0  274.0  374.0\n",
      "75  75.0   25.0  175.0  275.0  375.0\n",
      "76  76.0   24.0  176.0  276.0  376.0\n",
      "77  77.0   23.0  177.0  277.0  377.0\n",
      "78  78.0   22.0  178.0  278.0  378.0\n",
      "79  79.0   21.0  179.0  279.0  379.0\n",
      "80  80.0   20.0  180.0  280.0  380.0\n",
      "81  81.0   19.0  181.0  281.0  381.0\n",
      "82  82.0   18.0  182.0  282.0  382.0\n",
      "83  83.0   17.0  183.0  283.0  383.0\n",
      "84  84.0   16.0  184.0  284.0  384.0\n",
      "85  85.0   15.0  185.0  285.0  385.0\n",
      "86  86.0   14.0  186.0  286.0  386.0\n",
      "87  87.0   13.0  187.0  287.0  387.0\n",
      "88  88.0   12.0  188.0  288.0  388.0\n",
      "89  89.0   11.0  189.0  289.0  389.0\n",
      "90  90.0   10.0  190.0  290.0  390.0\n",
      "91  91.0    9.0  191.0  291.0  391.0\n",
      "92  92.0    8.0  192.0  292.0  392.0\n",
      "93  93.0    7.0  193.0  293.0  393.0\n",
      "94  94.0    6.0  194.0  294.0  394.0\n",
      "95  95.0    5.0  195.0  295.0  395.0\n",
      "96  96.0    4.0  196.0  296.0  396.0\n",
      "97  97.0    3.0  197.0  297.0  397.0\n",
      "98  98.0    2.0  198.0  298.0  398.0\n",
      "99  99.0    1.0  199.0  299.0  399.0\n",
      "\n",
      "[100 rows x 5 columns]\n"
     ]
    }
   ],
   "source": [
    "data = np.arange(300, 400).astype(np.float32)\n",
    "df.add_column('e', data)\n",
    "\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dropping Columns\n",
    "\n",
    "Alternatively, we may want to remove columns from our `DataFrame`. We can do so using the `drop_column` method. Note that this method removes a column in-place - meaning that the `DataFrame` we act on will be modified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.arange(0, 100).astype(np.float32), \n",
    "                     'b': np.arange(100, 0, -1).astype(np.float32), \n",
    "                     'c': np.arange(100, 200).astype(np.float32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        b      c\n",
      "0   100.0  100.0\n",
      "1    99.0  101.0\n",
      "2    98.0  102.0\n",
      "3    97.0  103.0\n",
      "4    96.0  104.0\n",
      "5    95.0  105.0\n",
      "6    94.0  106.0\n",
      "7    93.0  107.0\n",
      "8    92.0  108.0\n",
      "9    91.0  109.0\n",
      "10   90.0  110.0\n",
      "11   89.0  111.0\n",
      "12   88.0  112.0\n",
      "13   87.0  113.0\n",
      "14   86.0  114.0\n",
      "15   85.0  115.0\n",
      "16   84.0  116.0\n",
      "17   83.0  117.0\n",
      "18   82.0  118.0\n",
      "19   81.0  119.0\n",
      "20   80.0  120.0\n",
      "21   79.0  121.0\n",
      "22   78.0  122.0\n",
      "23   77.0  123.0\n",
      "24   76.0  124.0\n",
      "25   75.0  125.0\n",
      "26   74.0  126.0\n",
      "27   73.0  127.0\n",
      "28   72.0  128.0\n",
      "29   71.0  129.0\n",
      "..    ...    ...\n",
      "70   30.0  170.0\n",
      "71   29.0  171.0\n",
      "72   28.0  172.0\n",
      "73   27.0  173.0\n",
      "74   26.0  174.0\n",
      "75   25.0  175.0\n",
      "76   24.0  176.0\n",
      "77   23.0  177.0\n",
      "78   22.0  178.0\n",
      "79   21.0  179.0\n",
      "80   20.0  180.0\n",
      "81   19.0  181.0\n",
      "82   18.0  182.0\n",
      "83   17.0  183.0\n",
      "84   16.0  184.0\n",
      "85   15.0  185.0\n",
      "86   14.0  186.0\n",
      "87   13.0  187.0\n",
      "88   12.0  188.0\n",
      "89   11.0  189.0\n",
      "90   10.0  190.0\n",
      "91    9.0  191.0\n",
      "92    8.0  192.0\n",
      "93    7.0  193.0\n",
      "94    6.0  194.0\n",
      "95    5.0  195.0\n",
      "96    4.0  196.0\n",
      "97    3.0  197.0\n",
      "98    2.0  198.0\n",
      "99    1.0  199.0\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "df.drop_column('a')\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we want to remove a column without modifying the original DataFrame, we can use the `drop` method. This method will return a new DataFrame without that column (or columns)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.arange(0, 100).astype(np.float32), \n",
    "                     'b': np.arange(100, 0, -1).astype(np.float32), \n",
    "                     'c': np.arange(100, 200).astype(np.float32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original DataFrame:\n",
      "       a      b      c\n",
      "0    0.0  100.0  100.0\n",
      "1    1.0   99.0  101.0\n",
      "2    2.0   98.0  102.0\n",
      "3    3.0   97.0  103.0\n",
      "4    4.0   96.0  104.0\n",
      "5    5.0   95.0  105.0\n",
      "6    6.0   94.0  106.0\n",
      "7    7.0   93.0  107.0\n",
      "8    8.0   92.0  108.0\n",
      "9    9.0   91.0  109.0\n",
      "10  10.0   90.0  110.0\n",
      "11  11.0   89.0  111.0\n",
      "12  12.0   88.0  112.0\n",
      "13  13.0   87.0  113.0\n",
      "14  14.0   86.0  114.0\n",
      "15  15.0   85.0  115.0\n",
      "16  16.0   84.0  116.0\n",
      "17  17.0   83.0  117.0\n",
      "18  18.0   82.0  118.0\n",
      "19  19.0   81.0  119.0\n",
      "20  20.0   80.0  120.0\n",
      "21  21.0   79.0  121.0\n",
      "22  22.0   78.0  122.0\n",
      "23  23.0   77.0  123.0\n",
      "24  24.0   76.0  124.0\n",
      "25  25.0   75.0  125.0\n",
      "26  26.0   74.0  126.0\n",
      "27  27.0   73.0  127.0\n",
      "28  28.0   72.0  128.0\n",
      "29  29.0   71.0  129.0\n",
      "..   ...    ...    ...\n",
      "70  70.0   30.0  170.0\n",
      "71  71.0   29.0  171.0\n",
      "72  72.0   28.0  172.0\n",
      "73  73.0   27.0  173.0\n",
      "74  74.0   26.0  174.0\n",
      "75  75.0   25.0  175.0\n",
      "76  76.0   24.0  176.0\n",
      "77  77.0   23.0  177.0\n",
      "78  78.0   22.0  178.0\n",
      "79  79.0   21.0  179.0\n",
      "80  80.0   20.0  180.0\n",
      "81  81.0   19.0  181.0\n",
      "82  82.0   18.0  182.0\n",
      "83  83.0   17.0  183.0\n",
      "84  84.0   16.0  184.0\n",
      "85  85.0   15.0  185.0\n",
      "86  86.0   14.0  186.0\n",
      "87  87.0   13.0  187.0\n",
      "88  88.0   12.0  188.0\n",
      "89  89.0   11.0  189.0\n",
      "90  90.0   10.0  190.0\n",
      "91  91.0    9.0  191.0\n",
      "92  92.0    8.0  192.0\n",
      "93  93.0    7.0  193.0\n",
      "94  94.0    6.0  194.0\n",
      "95  95.0    5.0  195.0\n",
      "96  96.0    4.0  196.0\n",
      "97  97.0    3.0  197.0\n",
      "98  98.0    2.0  198.0\n",
      "99  99.0    1.0  199.0\n",
      "\n",
      "[100 rows x 3 columns]\n",
      "-------------------------------------------------------------------------------\n",
      "New DataFrame:\n",
      "        b      c\n",
      "0   100.0  100.0\n",
      "1    99.0  101.0\n",
      "2    98.0  102.0\n",
      "3    97.0  103.0\n",
      "4    96.0  104.0\n",
      "5    95.0  105.0\n",
      "6    94.0  106.0\n",
      "7    93.0  107.0\n",
      "8    92.0  108.0\n",
      "9    91.0  109.0\n",
      "10   90.0  110.0\n",
      "11   89.0  111.0\n",
      "12   88.0  112.0\n",
      "13   87.0  113.0\n",
      "14   86.0  114.0\n",
      "15   85.0  115.0\n",
      "16   84.0  116.0\n",
      "17   83.0  117.0\n",
      "18   82.0  118.0\n",
      "19   81.0  119.0\n",
      "20   80.0  120.0\n",
      "21   79.0  121.0\n",
      "22   78.0  122.0\n",
      "23   77.0  123.0\n",
      "24   76.0  124.0\n",
      "25   75.0  125.0\n",
      "26   74.0  126.0\n",
      "27   73.0  127.0\n",
      "28   72.0  128.0\n",
      "29   71.0  129.0\n",
      "..    ...    ...\n",
      "70   30.0  170.0\n",
      "71   29.0  171.0\n",
      "72   28.0  172.0\n",
      "73   27.0  173.0\n",
      "74   26.0  174.0\n",
      "75   25.0  175.0\n",
      "76   24.0  176.0\n",
      "77   23.0  177.0\n",
      "78   22.0  178.0\n",
      "79   21.0  179.0\n",
      "80   20.0  180.0\n",
      "81   19.0  181.0\n",
      "82   18.0  182.0\n",
      "83   17.0  183.0\n",
      "84   16.0  184.0\n",
      "85   15.0  185.0\n",
      "86   14.0  186.0\n",
      "87   13.0  187.0\n",
      "88   12.0  188.0\n",
      "89   11.0  189.0\n",
      "90   10.0  190.0\n",
      "91    9.0  191.0\n",
      "92    8.0  192.0\n",
      "93    7.0  193.0\n",
      "94    6.0  194.0\n",
      "95    5.0  195.0\n",
      "96    4.0  196.0\n",
      "97    3.0  197.0\n",
      "98    2.0  198.0\n",
      "99    1.0  199.0\n",
      "\n",
      "[100 rows x 2 columns]\n"
     ]
    }
   ],
   "source": [
    "new_df = df.drop('a')\n",
    "\n",
    "print('Original DataFrame:')\n",
    "print(df)\n",
    "print(79 * '-')\n",
    "print('New DataFrame:')\n",
    "print(new_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also pass in a list of column names to drop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Original DataFrame:\n",
      "       a      b      c\n",
      "0    0.0  100.0  100.0\n",
      "1    1.0   99.0  101.0\n",
      "2    2.0   98.0  102.0\n",
      "3    3.0   97.0  103.0\n",
      "4    4.0   96.0  104.0\n",
      "5    5.0   95.0  105.0\n",
      "6    6.0   94.0  106.0\n",
      "7    7.0   93.0  107.0\n",
      "8    8.0   92.0  108.0\n",
      "9    9.0   91.0  109.0\n",
      "10  10.0   90.0  110.0\n",
      "11  11.0   89.0  111.0\n",
      "12  12.0   88.0  112.0\n",
      "13  13.0   87.0  113.0\n",
      "14  14.0   86.0  114.0\n",
      "15  15.0   85.0  115.0\n",
      "16  16.0   84.0  116.0\n",
      "17  17.0   83.0  117.0\n",
      "18  18.0   82.0  118.0\n",
      "19  19.0   81.0  119.0\n",
      "20  20.0   80.0  120.0\n",
      "21  21.0   79.0  121.0\n",
      "22  22.0   78.0  122.0\n",
      "23  23.0   77.0  123.0\n",
      "24  24.0   76.0  124.0\n",
      "25  25.0   75.0  125.0\n",
      "26  26.0   74.0  126.0\n",
      "27  27.0   73.0  127.0\n",
      "28  28.0   72.0  128.0\n",
      "29  29.0   71.0  129.0\n",
      "..   ...    ...    ...\n",
      "70  70.0   30.0  170.0\n",
      "71  71.0   29.0  171.0\n",
      "72  72.0   28.0  172.0\n",
      "73  73.0   27.0  173.0\n",
      "74  74.0   26.0  174.0\n",
      "75  75.0   25.0  175.0\n",
      "76  76.0   24.0  176.0\n",
      "77  77.0   23.0  177.0\n",
      "78  78.0   22.0  178.0\n",
      "79  79.0   21.0  179.0\n",
      "80  80.0   20.0  180.0\n",
      "81  81.0   19.0  181.0\n",
      "82  82.0   18.0  182.0\n",
      "83  83.0   17.0  183.0\n",
      "84  84.0   16.0  184.0\n",
      "85  85.0   15.0  185.0\n",
      "86  86.0   14.0  186.0\n",
      "87  87.0   13.0  187.0\n",
      "88  88.0   12.0  188.0\n",
      "89  89.0   11.0  189.0\n",
      "90  90.0   10.0  190.0\n",
      "91  91.0    9.0  191.0\n",
      "92  92.0    8.0  192.0\n",
      "93  93.0    7.0  193.0\n",
      "94  94.0    6.0  194.0\n",
      "95  95.0    5.0  195.0\n",
      "96  96.0    4.0  196.0\n",
      "97  97.0    3.0  197.0\n",
      "98  98.0    2.0  198.0\n",
      "99  99.0    1.0  199.0\n",
      "\n",
      "[100 rows x 3 columns]\n",
      "-------------------------------------------------------------------------------\n",
      "New DataFrame:\n",
      "        c\n",
      "0   100.0\n",
      "1   101.0\n",
      "2   102.0\n",
      "3   103.0\n",
      "4   104.0\n",
      "5   105.0\n",
      "6   106.0\n",
      "7   107.0\n",
      "8   108.0\n",
      "9   109.0\n",
      "10  110.0\n",
      "11  111.0\n",
      "12  112.0\n",
      "13  113.0\n",
      "14  114.0\n",
      "15  115.0\n",
      "16  116.0\n",
      "17  117.0\n",
      "18  118.0\n",
      "19  119.0\n",
      "20  120.0\n",
      "21  121.0\n",
      "22  122.0\n",
      "23  123.0\n",
      "24  124.0\n",
      "25  125.0\n",
      "26  126.0\n",
      "27  127.0\n",
      "28  128.0\n",
      "29  129.0\n",
      "..    ...\n",
      "70  170.0\n",
      "71  171.0\n",
      "72  172.0\n",
      "73  173.0\n",
      "74  174.0\n",
      "75  175.0\n",
      "76  176.0\n",
      "77  177.0\n",
      "78  178.0\n",
      "79  179.0\n",
      "80  180.0\n",
      "81  181.0\n",
      "82  182.0\n",
      "83  183.0\n",
      "84  184.0\n",
      "85  185.0\n",
      "86  186.0\n",
      "87  187.0\n",
      "88  188.0\n",
      "89  189.0\n",
      "90  190.0\n",
      "91  191.0\n",
      "92  192.0\n",
      "93  193.0\n",
      "94  194.0\n",
      "95  195.0\n",
      "96  196.0\n",
      "97  197.0\n",
      "98  198.0\n",
      "99  199.0\n",
      "\n",
      "[100 rows x 1 columns]\n"
     ]
    }
   ],
   "source": [
    "new_df = df.drop(['a', 'b'])\n",
    "\n",
    "print('Original DataFrame:')\n",
    "print(df)\n",
    "print(79 * '-')\n",
    "print('New DataFrame:')\n",
    "print(new_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Missing Data\n",
    "\n",
    "Sometimes data is not as clean as we would like it - often there wrong values or values that are missing entirely. cuDF DataFrames can represent missing values using the Python `None` keyword."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       a     b     c\n",
      "0      0   0.0   0.0\n",
      "1   null   0.1   0.1\n",
      "2      2   0.2  null\n",
      "3      3  null  null\n",
      "4      4   0.4   0.4\n",
      "5      5   0.5   0.5\n",
      "6      6   0.6  null\n",
      "7      7   0.7   0.7\n",
      "8      8   0.8   0.8\n",
      "9   null   0.9   0.9\n",
      "10    10   1.0   1.0\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame({'a': [0, None, 2, 3, 4, 5, 6, 7, 8, None, 10],\n",
    "                     'b': [0.0, 0.1, 0.2, None, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], \n",
    "                     'c': [0.0, 0.1, None, None, 0.4, 0.5, None, 0.7, 0.8, 0.9, 1.0]})\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also fill in these missing values with another value using the `fillna` method. Both `Series` and `DataFrame` objects implement this method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       a     b      c\n",
      "0      0   0.0    0.0\n",
      "1   null   0.1    0.1\n",
      "2      2   0.2  999.0\n",
      "3      3  null  999.0\n",
      "4      4   0.4    0.4\n",
      "5      5   0.5    0.5\n",
      "6      6   0.6  999.0\n",
      "7      7   0.7    0.7\n",
      "8      8   0.8    0.8\n",
      "9   null   0.9    0.9\n",
      "10    10   1.0    1.0\n"
     ]
    }
   ],
   "source": [
    "df['c'] = df['c'].fillna(999)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     a    b      c\n",
      "0    0  0.0    0.0\n",
      "1   -1  0.1    0.1\n",
      "2    2  0.2  999.0\n",
      "3    3 -1.0  999.0\n",
      "4    4  0.4    0.4\n",
      "5    5  0.5    0.5\n",
      "6    6  0.6  999.0\n",
      "7    7  0.7    0.7\n",
      "8    8  0.8    0.8\n",
      "9   -1  0.9    0.9\n",
      "10  10  1.0    1.0\n"
     ]
    }
   ],
   "source": [
    "new_df = df.fillna(-1)\n",
    "print(new_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Boolean Indexing\n",
    "\n",
    "We previously saw how we can select certain rows from our dataset by using the bracket `[]` notation. However, we may want to select rows based on a certain criteria - this is called boolean indexing. We can combine the indexing notation with an array of boolean values to select only certain rows that meet this criteria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>76</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>77</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>78</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>79</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>80</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>81</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>82</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>83</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>84</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>86</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>87</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>88</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>90</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>91</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>92</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>93</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>95</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>96</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>97</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>98</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b   c   d\n",
       "75  3  0  75  25\n",
       "76  3  0  76  24\n",
       "77  3  1  77  23\n",
       "78  3  0  78  22\n",
       "79  3  0  79  21\n",
       "80  3  0  80  20\n",
       "81  3  0  81  19\n",
       "82  3  0  82  18\n",
       "83  3  1  83  17\n",
       "84  3  1  84  16\n",
       "85  3  0  85  15\n",
       "86  3  0  86  14\n",
       "87  3  0  87  13\n",
       "88  3  0  88  12\n",
       "89  3  0  89  11\n",
       "90  3  1  90  10\n",
       "91  3  1  91   9\n",
       "92  3  0  92   8\n",
       "93  3  1  93   7\n",
       "94  3  1  94   6\n",
       "95  3  0  95   5\n",
       "96  3  0  96   4\n",
       "97  3  0  97   3\n",
       "98  3  1  98   2\n",
       "99  3  1  99   1"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mask = df['a'] == 3\n",
    "df[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sorting Data\n",
    "\n",
    "Data is often not sorted before we start to work with it. Sorting data is is very useful for optimizing operations like joins and aggregations, especially when the data is distributed.\n",
    "\n",
    "We can sort data in cuDF using the `sort_values` method and passing in which column we want to sort by. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d\n",
      "0  0  1  0  100\n",
      "1  0  1  1   99\n",
      "2  0  0  2   98\n",
      "3  0  1  3   97\n",
      "4  0  0  4   96\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    a  b   c  d\n",
      "99  3  0  99  1\n",
      "98  3  1  98  2\n",
      "97  3  0  97  3\n",
      "96  3  0  96  4\n",
      "95  3  1  95  5\n"
     ]
    }
   ],
   "source": [
    "print(df.sort_values('d').head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also specify if the column we're sorting should be sorted in ascending or descending order by using the `ascending` argument and passing in `True` or `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    a  b   c  d\n",
      "99  3  0  99  1\n",
      "98  3  1  98  2\n",
      "97  3  0  97  3\n",
      "96  3  0  96  4\n",
      "95  3  1  95  5\n"
     ]
    }
   ],
   "source": [
    "print(df.sort_values('c', ascending=False).head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can sort by multiple columns by passing in a list of column names. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c   d\n",
      "2  0  0  2  98\n",
      "4  0  0  4  96\n",
      "5  0  0  5  95\n",
      "8  0  0  8  92\n",
      "9  0  0  9  91\n"
     ]
    }
   ],
   "source": [
    "print(df.sort_values(['a', 'b']).head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also specify which of those columns should be sorted in ascending or descending order by passing in a list of boolean values, where each boolean value maps to each column, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:When using a sequence of booleans for `ascending`, `na_position` flag is not yet supported and defaults to treating nulls as greater than all numbers\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sort with all columns specified descending:\n",
      "    a  b   c   d\n",
      "76  3  1  76  24\n",
      "77  3  1  77  23\n",
      "79  3  1  79  21\n",
      "81  3  1  81  19\n",
      "84  3  1  84  16\n",
      "-------------------------------------------------------------------------------\n",
      "Sort with both a descending and b ascending:\n",
      "    a  b   c   d\n",
      "75  3  0  75  25\n",
      "78  3  0  78  22\n",
      "80  3  0  80  20\n",
      "82  3  0  82  18\n",
      "83  3  0  83  17\n"
     ]
    }
   ],
   "source": [
    "print('Sort with all columns specified descending:')\n",
    "print(df.sort_values(['a', 'b'], ascending=False).head())\n",
    "print(79 * '-')\n",
    "print('Sort with both a descending and b ascending:')\n",
    "print(df.sort_values(['a', 'b'], ascending=[False, True]).head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Statistical Operations\n",
    "\n",
    "There are several statistical operations we can use to aggregate our data in meaningful ways. These can be applied to both `Series` and `DataFrame` objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "150"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['a'].sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a     150\n",
      "b      48\n",
      "c    4950\n",
      "d    5050\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(df.sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Applymap Operations\n",
    "\n",
    "While cuDF allows us to define new columns in interesting ways, we often want to work with more complex functions. We can define a function and use the `applymap` method to apply this function to each value in a column in element-wise fashion. While the below example is simple, it can be very easily extended to more complex workflows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0      10\n",
      "1      11\n",
      "2      12\n",
      "3      13\n",
      "4      14\n",
      "5      15\n",
      "6      16\n",
      "7      17\n",
      "8      18\n",
      "9      19\n",
      "10     20\n",
      "11     21\n",
      "12     22\n",
      "13     23\n",
      "14     24\n",
      "15     25\n",
      "16     26\n",
      "17     27\n",
      "18     28\n",
      "19     29\n",
      "20     30\n",
      "21     31\n",
      "22     32\n",
      "23     33\n",
      "24     34\n",
      "25     35\n",
      "26     36\n",
      "27     37\n",
      "28     38\n",
      "29     39\n",
      "     ... \n",
      "70     80\n",
      "71     81\n",
      "72     82\n",
      "73     83\n",
      "74     84\n",
      "75     85\n",
      "76     86\n",
      "77     87\n",
      "78     88\n",
      "79     89\n",
      "80     90\n",
      "81     91\n",
      "82     92\n",
      "83     93\n",
      "84     94\n",
      "85     95\n",
      "86     96\n",
      "87     97\n",
      "88     98\n",
      "89     99\n",
      "90    100\n",
      "91    101\n",
      "92    102\n",
      "93    103\n",
      "94    104\n",
      "95    105\n",
      "96    106\n",
      "97    107\n",
      "98    108\n",
      "99    109\n",
      "Name: c, Length: 100, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "def add_ten_to_x(x):\n",
    "    return x + 10\n",
    "\n",
    "print(df['c'].applymap(add_ten_to_x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Histogramming\n",
    "\n",
    "We can access the value counts of a column using the `value_counts` method. Note that this is typically used with columns representing discrete data i.e. integers, strings, categoricals, etc. We may not be as interested in the value counts of numerical data e.g. how often the value 2.1 appears. The results of the `value_counts` method can be used with Python plotting libraries like Matplotlib or Seaborn to generate visualizations such as histograms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    25\n",
      "1    25\n",
      "2    25\n",
      "3    25\n",
      "Name: a, dtype: int32\n"
     ]
    }
   ],
   "source": [
    "result = df['a'].value_counts()\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Concatenations\n",
    "\n",
    "In everyday data science, we typically work with multiple sources of data and wish to combine these data into a single more meaningful representation. These operations are often called concatenations and joins. We can concatenate two or more dataframes together row-wise or column-wise by passing in a list of the dataframes to be concatenated into the `cudf.concat` function and specifying the axis along which to concatenate these dataframes.\n",
    "\n",
    "If we want to concatenate the dataframes row-wise, we can specify `axis=0`. To concatenate column-wise, we can specify `axis=1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'c': np.arange(0, 100).astype(np.int32), \n",
    "                      'd': np.arange(100, 0, -1).astype(np.int32)})\n",
    "df2 = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'c': np.arange(0, 100).astype(np.int32), \n",
    "                      'd': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17</td>\n",
       "      <td>83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>18</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>21</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>28</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>70</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>71</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>72</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>73</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>74</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>76</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>77</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>78</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>79</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>80</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>81</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>82</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>83</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>84</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>86</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>87</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>88</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>90</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>91</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>92</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>93</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>94</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>95</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>96</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>97</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>98</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b   c    d\n",
       "0   0  1   0  100\n",
       "1   0  0   1   99\n",
       "2   0  1   2   98\n",
       "3   0  1   3   97\n",
       "4   0  1   4   96\n",
       "5   0  0   5   95\n",
       "6   0  0   6   94\n",
       "7   0  1   7   93\n",
       "8   0  0   8   92\n",
       "9   0  1   9   91\n",
       "10  0  1  10   90\n",
       "11  0  0  11   89\n",
       "12  0  0  12   88\n",
       "13  0  1  13   87\n",
       "14  0  0  14   86\n",
       "15  0  1  15   85\n",
       "16  0  0  16   84\n",
       "17  0  0  17   83\n",
       "18  0  1  18   82\n",
       "19  0  1  19   81\n",
       "20  0  1  20   80\n",
       "21  0  1  21   79\n",
       "22  0  1  22   78\n",
       "23  0  1  23   77\n",
       "24  0  0  24   76\n",
       "25  1  0  25   75\n",
       "26  1  1  26   74\n",
       "27  1  1  27   73\n",
       "28  1  1  28   72\n",
       "29  1  0  29   71\n",
       ".. .. ..  ..  ...\n",
       "70  2  0  70   30\n",
       "71  2  1  71   29\n",
       "72  2  1  72   28\n",
       "73  2  0  73   27\n",
       "74  2  0  74   26\n",
       "75  3  0  75   25\n",
       "76  3  1  76   24\n",
       "77  3  0  77   23\n",
       "78  3  0  78   22\n",
       "79  3  0  79   21\n",
       "80  3  1  80   20\n",
       "81  3  1  81   19\n",
       "82  3  0  82   18\n",
       "83  3  1  83   17\n",
       "84  3  1  84   16\n",
       "85  3  0  85   15\n",
       "86  3  0  86   14\n",
       "87  3  1  87   13\n",
       "88  3  0  88   12\n",
       "89  3  1  89   11\n",
       "90  3  1  90   10\n",
       "91  3  0  91    9\n",
       "92  3  1  92    8\n",
       "93  3  0  93    7\n",
       "94  3  0  94    6\n",
       "95  3  0  95    5\n",
       "96  3  0  96    4\n",
       "97  3  0  97    3\n",
       "98  3  1  98    2\n",
       "99  3  0  99    1\n",
       "\n",
       "[200 rows x 4 columns]"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = cudf.concat([df1, df2], axis=0)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'c': np.arange(0, 100).astype(np.int32), \n",
    "                      'd': np.arange(100, 0, -1).astype(np.int32)})\n",
    "df2 = cudf.DataFrame({'e': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'f': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'g': np.arange(0, 100).astype(np.int32), \n",
    "                      'h': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "      <th>g</th>\n",
       "      <th>h</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>98</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>98</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>97</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>97</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>96</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>95</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>95</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>94</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>93</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>8</td>\n",
       "      <td>92</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>91</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>90</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>10</td>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>89</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>88</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>87</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>86</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>14</td>\n",
       "      <td>86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>85</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>15</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>16</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>17</td>\n",
       "      <td>83</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>17</td>\n",
       "      <td>83</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>18</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>80</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>79</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "      <td>78</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>78</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>77</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>23</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>76</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>76</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>75</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26</td>\n",
       "      <td>74</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>26</td>\n",
       "      <td>74</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>73</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>72</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>28</td>\n",
       "      <td>72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>29</td>\n",
       "      <td>71</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>70</td>\n",
       "      <td>30</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>70</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>71</td>\n",
       "      <td>29</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>71</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>72</td>\n",
       "      <td>28</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>72</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>73</td>\n",
       "      <td>27</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>73</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>74</td>\n",
       "      <td>26</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>74</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>25</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>76</td>\n",
       "      <td>24</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>76</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>77</td>\n",
       "      <td>23</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>77</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>78</td>\n",
       "      <td>22</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>78</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>79</td>\n",
       "      <td>21</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>79</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>80</td>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>80</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>81</td>\n",
       "      <td>19</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>81</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>82</td>\n",
       "      <td>18</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>82</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>83</td>\n",
       "      <td>17</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>83</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>84</td>\n",
       "      <td>16</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>84</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>15</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>86</td>\n",
       "      <td>14</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>86</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>87</td>\n",
       "      <td>13</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>87</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>88</td>\n",
       "      <td>12</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>88</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>11</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>90</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>90</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>91</td>\n",
       "      <td>9</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>91</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>92</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>92</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>93</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>93</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>94</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>94</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>95</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>95</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>96</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>96</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>97</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>97</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>100 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    a  b   c    d  e  f   g    h\n",
       "0   0  1   0  100  0  1   0  100\n",
       "1   0  1   1   99  0  1   1   99\n",
       "2   0  0   2   98  0  0   2   98\n",
       "3   0  0   3   97  0  0   3   97\n",
       "4   0  1   4   96  0  1   4   96\n",
       "5   0  1   5   95  0  0   5   95\n",
       "6   0  1   6   94  0  1   6   94\n",
       "7   0  0   7   93  0  1   7   93\n",
       "8   0  1   8   92  0  0   8   92\n",
       "9   0  0   9   91  0  0   9   91\n",
       "10  0  0  10   90  0  1  10   90\n",
       "11  0  0  11   89  0  0  11   89\n",
       "12  0  0  12   88  0  1  12   88\n",
       "13  0  0  13   87  0  1  13   87\n",
       "14  0  0  14   86  0  1  14   86\n",
       "15  0  1  15   85  0  1  15   85\n",
       "16  0  1  16   84  0  0  16   84\n",
       "17  0  1  17   83  0  1  17   83\n",
       "18  0  0  18   82  0  1  18   82\n",
       "19  0  1  19   81  0  1  19   81\n",
       "20  0  1  20   80  0  1  20   80\n",
       "21  0  0  21   79  0  0  21   79\n",
       "22  0  0  22   78  0  1  22   78\n",
       "23  0  1  23   77  0  1  23   77\n",
       "24  0  1  24   76  0  1  24   76\n",
       "25  1  0  25   75  1  0  25   75\n",
       "26  1  0  26   74  1  0  26   74\n",
       "27  1  1  27   73  1  0  27   73\n",
       "28  1  0  28   72  1  1  28   72\n",
       "29  1  0  29   71  1  1  29   71\n",
       ".. .. ..  ..  ... .. ..  ..  ...\n",
       "70  2  1  70   30  2  0  70   30\n",
       "71  2  0  71   29  2  1  71   29\n",
       "72  2  0  72   28  2  1  72   28\n",
       "73  2  1  73   27  2  0  73   27\n",
       "74  2  0  74   26  2  1  74   26\n",
       "75  3  0  75   25  3  0  75   25\n",
       "76  3  0  76   24  3  0  76   24\n",
       "77  3  0  77   23  3  0  77   23\n",
       "78  3  1  78   22  3  0  78   22\n",
       "79  3  1  79   21  3  1  79   21\n",
       "80  3  1  80   20  3  0  80   20\n",
       "81  3  1  81   19  3  1  81   19\n",
       "82  3  1  82   18  3  1  82   18\n",
       "83  3  0  83   17  3  0  83   17\n",
       "84  3  1  84   16  3  1  84   16\n",
       "85  3  0  85   15  3  0  85   15\n",
       "86  3  0  86   14  3  0  86   14\n",
       "87  3  0  87   13  3  0  87   13\n",
       "88  3  1  88   12  3  1  88   12\n",
       "89  3  1  89   11  3  0  89   11\n",
       "90  3  1  90   10  3  1  90   10\n",
       "91  3  0  91    9  3  0  91    9\n",
       "92  3  1  92    8  3  1  92    8\n",
       "93  3  1  93    7  3  0  93    7\n",
       "94  3  1  94    6  3  0  94    6\n",
       "95  3  0  95    5  3  1  95    5\n",
       "96  3  0  96    4  3  1  96    4\n",
       "97  3  1  97    3  3  1  97    3\n",
       "98  3  0  98    2  3  0  98    2\n",
       "99  3  0  99    1  3  0  99    1\n",
       "\n",
       "[100 rows x 8 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = cudf.concat([df1, df2], axis=1)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Joins / Merges\n",
    "\n",
    "Multiple dataframes can be joined together using a single (or multiple) column(s). There are two syntaxes for performing joins:\n",
    "\n",
    "* One can use the `DataFrame.merge` method and pass in another dataframe to join, or\n",
    "* One can use the `cudf.merge` function and pass in which dataframes to join.\n",
    "\n",
    "Both syntaxes can also be passed a list of column names to an additional keyword argument `on` - this will specify which columns the dataframes should be joined on. If this keyword is not specified, cuDF will by default join using column names that appear in both dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'c': np.arange(0, 100).astype(np.int32), \n",
    "                      'd': np.arange(100, 0, -1).astype(np.int32)})\n",
    "df2 = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                      'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                      'e': np.arange(0, 100).astype(np.int32), \n",
    "                      'f': np.arange(100, 0, -1).astype(np.int32)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d  e    f\n",
      "0  0  0  0  100  0  100\n",
      "1  0  1  1   99  4   96\n",
      "2  0  0  2   98  0  100\n",
      "3  0  0  3   97  0  100\n",
      "4  0  1  4   96  4   96\n"
     ]
    }
   ],
   "source": [
    "df = df1.merge(df2)\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b_x  c    d  b_y  e    f\n",
      "0  0    0  0  100    0  0  100\n",
      "1  0    1  1   99    0  0  100\n",
      "2  0    0  2   98    0  0  100\n",
      "3  0    0  3   97    0  0  100\n",
      "4  0    1  4   96    0  0  100\n"
     ]
    }
   ],
   "source": [
    "df = df1.merge(df2, on=['a'])\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d  e    f\n",
      "0  0  0  0  100  0  100\n",
      "1  0  1  1   99  4   96\n",
      "2  0  0  2   98  0  100\n",
      "3  0  0  3   97  0  100\n",
      "4  0  1  4   96  4   96\n"
     ]
    }
   ],
   "source": [
    "df = df1.merge(df2, on=['a', 'b'])\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d  e    f\n",
      "0  0  0  0  100  0  100\n",
      "1  0  1  1   99  4   96\n",
      "2  0  0  2   98  0  100\n",
      "3  0  0  3   97  0  100\n",
      "4  0  1  4   96  4   96\n"
     ]
    }
   ],
   "source": [
    "df = cudf.merge(df1, df2)\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b_x  c    d  b_y  e    f\n",
      "0  0    0  0  100    0  0  100\n",
      "1  0    1  1   99    0  0  100\n",
      "2  0    0  2   98    0  0  100\n",
      "3  0    0  3   97    0  0  100\n",
      "4  0    1  4   96    0  0  100\n"
     ]
    }
   ],
   "source": [
    "df = cudf.merge(df1, df2, on=['a'])\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d  e    f\n",
      "0  0  0  0  100  0  100\n",
      "1  0  1  1   99  4   96\n",
      "2  0  0  2   98  0  100\n",
      "3  0  0  3   97  0  100\n",
      "4  0  1  4   96  4   96\n"
     ]
    }
   ],
   "source": [
    "df = cudf.merge(df1, df2, on=['a', 'b'])\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Groupbys\n",
    "\n",
    "A useful operation when working with datasets is to group the data using a specific key and aggregate the values mapping to those keys. For example, we might want to aggregate multiple temperature measurements taken during a day from a specific sensor and average those measurements to find avergage daily temperature at a specific geolocation.\n",
    "\n",
    "cuDF allows us to perform such an operation using the `groupby` method. This will create an object of type `cudf.groupby.groupby.Groupby` that we can operate on using aggregation functions such as `sum`, `var`, or complex aggregation functions defined by the user.\n",
    "\n",
    "We can also specify multiple columns to group on by passing a list of column names to the `groupby` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b  c    d\n",
      "0  0  1  0  100\n",
      "1  0  1  1   99\n",
      "2  0  1  2   98\n",
      "3  0  0  3   97\n",
      "4  0  0  4   96\n"
     ]
    }
   ],
   "source": [
    "df = cudf.DataFrame({'a': np.repeat([0, 1, 2, 3], 25).astype(np.int32), \n",
    "                     'b': np.random.randint(2, size=100).astype(np.int32), \n",
    "                     'c': np.arange(0, 100).astype(np.int32), \n",
    "                     'd': np.arange(100, 0, -1).astype(np.int32)})\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<cudf.core.groupby.groupby.DataFrameGroupBy at 0x7f30a8e334a8>"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grouped_df = df.groupby('a')\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    b     c     d\n",
      "a                \n",
      "0  18   300  2200\n",
      "1  15   925  1575\n",
      "2  12  1550   950\n",
      "3  13  2175   325\n"
     ]
    }
   ],
   "source": [
    "aggregation = grouped_df.sum()\n",
    "print(aggregation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        c     d\n",
      "a b            \n",
      "0 0    81   619\n",
      "  1   219  1581\n",
      "1 0   392   608\n",
      "  1   533   967\n",
      "2 0   782   518\n",
      "  1   768   432\n",
      "3 0  1048   152\n",
      "  1  1127   173\n"
     ]
    }
   ],
   "source": [
    "aggregation = df.groupby(['a', 'b']).sum().to_pandas()\n",
    "print(aggregation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### One Hot Encoding\n",
    "\n",
    "Data scientists often work with discrete data such as integers or categories. However, this data can be represented using a One Hote Encoding format.\n",
    "\n",
    "cuDF allows us to convert these discrete datas to a One Hot Encoding format using the `one_hot_encoding` method. We can pass this method the column name to convert, a prefix with which to prepend to each newly created column, and the categories of data to create new columns for. We can pass in all the categories in the discrete data or a subset - cuDF will flexibly handle both and only create new columns for the categories specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b    c\n",
      "0  0  0  100\n",
      "1  0  1   99\n",
      "2  0  2   98\n",
      "3  0  3   97\n",
      "4  0  4   96\n"
     ]
    }
   ],
   "source": [
    "categories = [0, 1, 2, 3]\n",
    "df = cudf.DataFrame({'a': np.repeat(categories, 25).astype(np.int32), \n",
    "                     'b': np.arange(0, 100).astype(np.int32), \n",
    "                     'c': np.arange(100, 0, -1).astype(np.int32)})\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b    c  a__0  a__1  a__2  a__3\n",
      "0  0  0  100   1.0   0.0   0.0   0.0\n",
      "1  0  1   99   1.0   0.0   0.0   0.0\n",
      "2  0  2   98   1.0   0.0   0.0   0.0\n",
      "3  0  3   97   1.0   0.0   0.0   0.0\n",
      "4  0  4   96   1.0   0.0   0.0   0.0\n",
      "    a   b  c  a__0  a__1  a__2  a__3\n",
      "95  3  95  5   0.0   0.0   0.0   1.0\n",
      "96  3  96  4   0.0   0.0   0.0   1.0\n",
      "97  3  97  3   0.0   0.0   0.0   1.0\n",
      "98  3  98  2   0.0   0.0   0.0   1.0\n",
      "99  3  99  1   0.0   0.0   0.0   1.0\n"
     ]
    }
   ],
   "source": [
    "result = df.one_hot_encoding('a', prefix='a_', cats=categories)\n",
    "print(result.head())\n",
    "print(result.tail())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   a  b    c  a__0  a__1  a__2\n",
      "0  0  0  100   1.0   0.0   0.0\n",
      "1  0  1   99   1.0   0.0   0.0\n",
      "2  0  2   98   1.0   0.0   0.0\n",
      "3  0  3   97   1.0   0.0   0.0\n",
      "4  0  4   96   1.0   0.0   0.0\n",
      "    a   b  c  a__0  a__1  a__2\n",
      "95  3  95  5   0.0   0.0   0.0\n",
      "96  3  96  4   0.0   0.0   0.0\n",
      "97  3  97  3   0.0   0.0   0.0\n",
      "98  3  98  2   0.0   0.0   0.0\n",
      "99  3  99  1   0.0   0.0   0.0\n"
     ]
    }
   ],
   "source": [
    "result = df.one_hot_encoding('a', prefix='a_', cats=[0, 1, 2])\n",
    "print(result.head())\n",
    "print(result.tail())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"conclusion\"></a>\n",
    "## Conclusion\n",
    "\n",
    "In this notebook, we showed how to work with cuDF DataFrames in RAPIDS.\n",
    "\n",
    "To learn more about RAPIDS, be sure to check out: \n",
    "\n",
    "* [Open Source Website](http://rapids.ai)\n",
    "* [GitHub](https://github.com/rapidsai/)\n",
    "* [Press Release](https://nvidianews.nvidia.com/news/nvidia-introduces-rapids-open-source-gpu-acceleration-platform-for-large-scale-data-analytics-and-machine-learning)\n",
    "* [NVIDIA Blog](https://blogs.nvidia.com/blog/2018/10/10/rapids-data-science-open-source-community/)\n",
    "* [Developer Blog](https://devblogs.nvidia.com/gpu-accelerated-analytics-rapids/)\n",
    "* [NVIDIA Data Science Webpage](https://www.nvidia.com/en-us/deep-learning-ai/solutions/data-science/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}