t81_558_class_06_2_chat_gpt.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeffheaton/app_deep_learning/blob/main/t81_558_class_06_2_chat_gpt.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# T81-558: Applications of Deep Neural Networks\n",
        "**Module 6: ChatGPT and Large Language Models**\n",
        "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
        "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Module 6 Material\n",
        "\n",
        "* Part 6.1: Introduction to Transformers [[Video]](https://www.youtube.com/watch?v=mn6r5PYJcu0&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_1_transformers.ipynb)\n",
        "* **Part 6.2: Accessing the ChatGPT API** [[Video]](https://www.youtube.com/watch?v=tcdscXl4o5w&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_2_chat_gpt.ipynb)\n",
        "* Part 6.3: Llama, Alpaca, and LORA [[Video]](https://www.youtube.com/watch?v=oGQ3TQx1Qs8&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_3_alpaca_lora.ipynb)\n",
        "* Part 6.4: Introduction to Embeddings [[Video]](https://www.youtube.com/watch?v=e6kcs9Uj_ps&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_4_embedding.ipynb)\n",
        "* Part 6.5: Prompt Engineering [[Video]](https://www.youtube.com/watch?v=miTpIDR7k6c&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_5_prompt_engineering.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Google CoLab Instructions\n",
        "\n",
        "The following code ensures that Google CoLab is running the correct version of TensorFlow.\n",
        "  Running the following code will map your GDrive to ```/content/drive```."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "try:\n",
        "    from google.colab import drive\n",
        "    COLAB = True\n",
        "    print(\"Note: using Google CoLab\")\n",
        "except:\n",
        "    print(\"Note: not using Google CoLab\")\n",
        "    COLAB = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JyFjANq1v-2z"
      },
      "source": [
        "# Part 6.2: ChatGPT and NLP\n",
        "\n",
        "As the world of deep learning continues to expand and evolve, so do the tools and platforms that developers can utilize to harness its power. In this section, we're going to explore a unique integration - combining the robust capabilities of PyTorch with the computational prowess of OpenAI's ChatGPT.\n",
        "\n",
        "One of the most compelling features of services like ChatGPT is their availability as an API. But before we dive into the depths of coding and integration, let's understand what an API is and its significance in the AI domain.\n",
        "\n",
        "API stands for Application Programming Interface. Think of it as a bridge or a messenger that allows two different software applications to communicate with each other. In the context of AI and machine learning, APIs often allow developers to access a particular model or service without having to house the model on their local machine. This can be especially useful when the model in question, like ChatGPT, is large and resource-intensive.\n",
        "\n",
        "In the realm of AI, APIs have several distinct advantages:\n",
        "\n",
        "* Scalability: Since the actual model runs on external servers, developers don't need to worry about scaling infrastructure.\n",
        "* Maintenance: You get to use the latest and greatest version of the model without constantly updating your local copy.\n",
        "* Cost-Effective: Leveraging external computational resources can be more cost-effective than maintaining high-end infrastructure locally, especially for sporadic or one-off tasks.\n",
        "* Ease of Use: Instead of diving into the nitty-gritty details of model implementation and optimization, developers can directly utilize its capabilities with a few lines of code.\n",
        "\n",
        "In this section, while using PyTorch, we won't be running the neural network computations locally. Instead, our PyTorch code will communicate with the OpenAI API to access and harness the abilities of ChatGPT. The actual execution of the neural network code happens on OpenAI servers, bringing forth a unique synergy of PyTorch's flexibility and ChatGPT's conversational mastery.\n",
        "\n",
        "In this section, we will make use of the OpenAI ChatGPT API. Further information on this API can be found here:\n",
        "\n",
        "* [OpenAI API Login/Registration](https://platform.openai.com/apps)\n",
        "* [OpenAI API Reference](https://platform.openai.com/docs/introduction/overview)\n",
        "* [OpenAI Python API Reference](https://platform.openai.com/docs/api-reference/introduction?lang=python)\n",
        "* [OpenAI Python Library](https://github.com/openai/openai-python)\n",
        "* [OpenAI Cookbook for Python](https://github.com/openai/openai-cookbook/)\n",
        "\n",
        "## Installing the OpenAI Python Library\n",
        "\n",
        "As we delve deeper into the intricacies of deep learning, it's crucial to understand that the tools and platforms we use are as versatile as the concepts themselves. When it comes to accessing ChatGPT, a state-of-the-art conversational AI model developed by OpenAI, there are two predominant pathways:\n",
        "\n",
        "Direct API Access using Python's HTTP Capabilities: Python, with its rich library ecosystem, provides utilities like requests to directly communicate with APIs over HTTP. This method involves crafting the necessary API calls, handling responses, and error checking, giving the developer a granular control over the process.\n",
        "\n",
        "Using the Official OpenAI Python Library: OpenAI offers an official Python library, aptly named openai, that simplifies the process of integrating with ChatGPT and other OpenAI services. This library abstracts many of the intricacies and boilerplate steps of direct API access, offering a streamlined and user-friendly approach to interacting with the model.\n",
        "\n",
        "Each approach has its advantages. Direct API access provides a more hands-on, granular approach, allowing developers to intimately understand the intricacies of each API call. On the other hand, using the openai library can accelerate development, reduce potential errors, and allow for a more straightforward integration, especially for those new to API interactions.\n",
        "\n",
        "In our exploration, we'll opt for the second approach: accessing ChatGPT through the official openai Python library. Our decision is rooted in the following reasons:\n",
        "\n",
        "* Simplicity: The library offers an intuitive interface, making it easier for both beginners and experts to get started.\n",
        "Maintainability: Using an official library ensures that we benefit from the latest updates, improvements, and best practices put forth by OpenAI.\n",
        "* Focus: By leveraging the library, we can concentrate more on the applications and capabilities of ChatGPT rather than the nitty-gritty details of API communication.\n",
        "\n",
        "As we progress, you'll find that this choice allows us to quickly harness the power of ChatGPT while maintaining the flexibility to explore advanced functionalities.\n",
        "\n",
        "The following command installs the **opeanai** library."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FJdVstEjlIJx",
        "outputId": "ff7f59c2-bf31-42aa-8aac-80b778047b9f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: openai in /usr/local/lib/python3.10/dist-packages (0.27.8)\n",
            "Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai) (2.31.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai) (4.66.1)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai) (3.8.5)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (3.2.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai) (2023.7.22)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (23.1.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (4.0.3)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.4.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai) (1.3.1)\n"
          ]
        }
      ],
      "source": [
        "!pip install openai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EBqaIJSJ3BJD"
      },
      "source": [
        "## Obtaining an OpenAI API Key\n",
        "\n",
        "In order to delve into the practical exercises and code demonstrations within this section, students will need to obtain an OpenAI API key. This key grants access to OpenAI's services, including the ChatGPT functionality we'll be exploring. It's important to note that there is a nominal cost associated with the usage of this key, depending on the volume and intensity of requests made to OpenAI's servers. However, securing and using this key is entirely optional for this course. Engaging with this segment is not mandatory, nor will it be a part of any course assignments. The decision to obtain and use an OpenAI key rests solely with the student, allowing for a personalized learning journey tailored to individual interests and resources.\n",
        "\n",
        "To obtain an OpenAI API key, access this [site](https://platform.openai.com/apps)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "NSOgT81AlAUs"
      },
      "outputs": [],
      "source": [
        "import openai\n",
        "\n",
        "openai.api_key = \"[Insert Your Key]\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UyC-6nxu4u_I"
      },
      "source": [
        "\n",
        "\n",
        "typically had three main roles within the context of the messages list:\n",
        "\n",
        "* system role:\n",
        "  * Meaning: This role is usually used for high-level instructions to guide the behavior or context of the model throughout the conversation. Instructions given with the system role often set the tone or establish a framework for how the assistant should respond.\n",
        "  * Example: {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"} would guide the model to provide helpful answers to user queries.\n",
        "* user:\n",
        "  * Meaning: Messages with this role represent the input or questions from the end-user or person interacting with the model.\n",
        "  * Example: {\"role\": \"user\", \"content\": \"Who won the world series in 2020?\"} is a user asking a question about the World Series in 2020.\n",
        "* assistant:\n",
        "  * Meaning: Messages with this role represent the model's responses. When constructing a series of interactions, previous assistant messages can help provide context to the model for future queries.\n",
        "  * Example: {\"role\": \"assistant\", \"content\": \"The Los Angeles Dodgers won the World Series in 2020.\"} is the model's response to the earlier user question.\n",
        "\n",
        "These roles work together in the messages list to create a conversational flow where the model can maintain context over multiple turns of conversation. For instance, by including prior user and assistant messages in the list, one can ensure the model has the necessary context to answer follow-up questions accurately.\n",
        "\n",
        "It's worth noting that while these are the typical roles utilized in the chat-based model interaction, the OpenAI API is versatile. Users are encouraged to experiment with instructions and roles to get the desired behavior. Always refer to the latest OpenAI documentation for any updates or changes to the API and its functionalities.\n",
        "\n",
        "The following code demonstrates how to query the **gpt-3.5-turbo** model, whcih is similar to the engine used by ChatGPT."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BXYrysJ54tA5",
        "outputId": "d708f603-94cb-4ebc-bf5c-d35da249d040"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"id\": \"chatcmpl-7pxm6bNYIaauibKUvPk9uwkS7yo7T\",\n",
            "  \"object\": \"chat.completion\",\n",
            "  \"created\": 1692618858,\n",
            "  \"model\": \"gpt-3.5-turbo-0613\",\n",
            "  \"choices\": [\n",
            "    {\n",
            "      \"index\": 0,\n",
            "      \"message\": {\n",
            "        \"role\": \"assistant\",\n",
            "        \"content\": \"The five largest cities in the USA, ordered by population, are as follows:\\n\\n1. New York City, New York\\n2. Los Angeles, California\\n3. Chicago, Illinois\\n4. Houston, Texas\\n5. Phoenix, Arizona\"\n",
            "      },\n",
            "      \"finish_reason\": \"stop\"\n",
            "    }\n",
            "  ],\n",
            "  \"usage\": {\n",
            "    \"prompt_tokens\": 30,\n",
            "    \"completion_tokens\": 49,\n",
            "    \"total_tokens\": 79\n",
            "  }\n",
            "}\n"
          ]
        }
      ],
      "source": [
        "response = openai.ChatCompletion.create(\n",
        "  model=\"gpt-3.5-turbo\",\n",
        "  messages=[\n",
        "        {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
        "        {\"role\": \"user\", \"content\": \"What are the the 5 largest USA cities ordered by population?\"},\n",
        "        #{\"role\": \"assistant\", \"content\": \"The Los Angeles Dodgers won the World Series in 2020.\"},\n",
        "        #{\"role\": \"user\", \"content\": \"Where was it played?\"}\n",
        "    ]\n",
        ")\n",
        "\n",
        "print(response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SIWbUIv3UIAH"
      },
      "source": [
        "We can extract the needed information from that complex object with the following line of code:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "j6jtipDr80ZE",
        "outputId": "8842d214-83eb-444f-f593-9b7eb1923fea"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'The five largest cities in the USA, ordered by population, are as follows:\\n\\n1. New York City, New York\\n2. Los Angeles, California\\n3. Chicago, Illinois\\n4. Houston, Texas\\n5. Phoenix, Arizona'"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "response.choices[0].message.content"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4oI42e4PqKl"
      },
      "source": [
        "## Application to Text Extraction\n",
        "\n",
        "Language model-based learning, commonly abbreviated as LLM, has numerous applications in the world of business. One prevalent utilization of LLM is in the domain of text extraction. Text extraction focuses on the retrieval of specific pieces of information from a larger body of text. For instance, in scenarios where a dataset contains varied information about individuals—ranging from birthdays to job details—one can employ LLM to zero in on just the birthdays, efficiently filtering out extraneous data. The power of LLM lies in its ability to discern context and extract relevant details based on the user's requirements, as showcased in the code that adeptly identifies and extracts birthday details while disregarding other particulars."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c2ds0AQ9PvmI",
        "outputId": "73c8a3e2-6b3f-4f35-9867-57b619ab3fd5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "14-JUN-1995\n"
          ]
        }
      ],
      "source": [
        "INPUT = \"John was born on June 14, 1995, he was married on May 8, 2015.\"\n",
        "\n",
        "SYSTEM_PROMPT = \"You are to extract any birthdays from input, return the \" \\\n",
        "    \"date in the form 10-FEB-1990, or NONE if no birthday.\"\n",
        "\n",
        "response = openai.ChatCompletion.create(\n",
        "  model=\"gpt-3.5-turbo\",\n",
        "  messages=[\n",
        "        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
        "        {\"role\": \"user\", \"content\": INPUT},\n",
        "    ]\n",
        ")\n",
        "\n",
        "print(response.choices[0].message.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8vXeREOYUgqB"
      },
      "source": [
        "The same code can process a series of text strings. The dates in these strings are in a variety of different formats. The LLM is able to parse and find the needed birthdays and ignore other information. Notice that sometimes the date is not formatted as requested or multiple dates return. Soon we will learn about prompt engineering, wich solves some of these problems."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I2y7WfO_J8Mz",
        "outputId": "49d4f3cf-f4ec-44d6-fefe-2f09e97c20cc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "05-MAR-1990\n",
            "20-JUL-1985\n",
            "22-OCT-1992\n",
            "10-FEB-1990\n",
            "03-DEC-1994\n",
            "Liam's birthday is on 12-MAY-1988.\n",
            "25-APR-1991\n",
            "08-JAN-1984\n",
            "16-SEP-1993\n",
            "11/30/1989\n"
          ]
        }
      ],
      "source": [
        "\n",
        "LIST = [\n",
        "  \"Anna started her first job on 15th January 2012. She was born on March 5, 1990.\",\n",
        "  \"On 04/14/2007, Michael graduated from college. He was born on 20th July 1985.\",\n",
        "  \"Born on 22nd October 1992, Sophia got married on 11.11.2016.\",\n",
        "  \"Graduating from high school on June 5, 2005, was a big moment for Lucas. His birth date is 02/17/1987.\",\n",
        "  \"Isabelle began her professional journey on 01/09/2016, having been born on December 3, 1994.\",\n",
        "  \"Liam was born on May 12, 1988. He celebrated his wedding on 07-15-2014.\",\n",
        "  \"Eva celebrated her college graduation on 20-05-2013. Her birthday falls on April 25, 1991.\",\n",
        "  \"In 2006, specifically on 03.03.2006, Daniel started his first job. He came into this world on January 8, 1984.\",\n",
        "  \"On 05.25.2011, Emily donned her graduation gown. Her birthdate is September 16, 1993.\",\n",
        "  \"Henry marked his birthday on 11/30/1989. He tied the knot on October 10, 2017.\"\n",
        "]\n",
        "\n",
        "for item in LIST:\n",
        "  response = openai.ChatCompletion.create(\n",
        "    model=\"gpt-3.5-turbo\",\n",
        "    messages=[\n",
        "          {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n",
        "          {\"role\": \"user\", \"content\": item},\n",
        "      ]\n",
        "  )\n",
        "\n",
        "  print(response.choices[0].message.content)\n",
        "\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9 (torch)",
      "language": "python",
      "name": "pytorch"
    },
    "language_info": {
      "name": "python",
      "version": "3.9.16"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}