sm2201_manometer_analyzer.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "sm2201_manometer_analyzer.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MossbauerLab/ManometerNNAn/blob/master/sm2201_manometer_analyzer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i5oIySvv66vf",
        "colab_type": "text"
      },
      "source": [
        "Получение списка изображений с гугл диска, изображения представляют собой классификацию положения стрелки манометра\n",
        "шкала (0-1,6 атм.)\n",
        "Изображения рассортированы по директориям с положением стрелки манометра около соответсвующего значения шкалы:\n",
        "\n",
        "*   0,1 атм\n",
        "*   0,2 атм\n",
        "\n",
        "...\n",
        "\n",
        "*   1,6 атм\n",
        "\n",
        "Для поиска варианта при котором наиболее оптимальный результат оценки работы нейросети изображения имеют разную степень свертки (оригинал, 128х128 пикселей, ...)\n",
        "Оригинал имеет имя типа {Value}.{Number}\n",
        "где {Value} - величина давления\n",
        "       {Number} - какое-то число (номер изображения)\n",
        "       \n",
        "Сжатые изображения имеют имя типа  {Value}.{Number}_{WxH}\n",
        "где {WxH} - величина изображения в пикселях ширина х высота\n",
        "\n",
        "ФОРМАТ ИЗОБРАЖЕНИЙ - .jpg\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dQ2NysP8NQb9",
        "colab_type": "text"
      },
      "source": [
        "Монтирование гугл диска"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UbY6-H_C9PN-",
        "colab_type": "code",
        "outputId": "d1e04561-5376-4e18-bcdd-bfc0aa6d59c1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount(u'/content/gdrive')"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/gdrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M2qiPpFcPFgo",
        "colab_type": "code",
        "outputId": "5c21d187-bd96-495b-a116-7c7ae7423ef0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1902
        }
      },
      "source": [
        "import os\n",
        "from tensorflow.python.keras.preprocessing.image import ImageDataGenerator\n",
        "from tensorflow.python.keras.models import Sequential\n",
        "from tensorflow.python.keras.layers import Conv2D, MaxPooling2D\n",
        "from tensorflow.python.keras.layers import Activation, Dropout, Flatten, Dense\n",
        "from tensorflow.python.keras.applications import VGG16\n",
        "\n",
        "\n",
        "# list root dir\n",
        "# for item in os.listdir(u'/content/gdrive/My Drive'):\n",
        "#    print(u'sub folder item: ' + item)\n",
        "\n",
        "manometer_nn_data_path = os.path.join(u'/content/gdrive', 'My Drive/MessbauerSoft/NeuralNetworks_Data/Manometer')\n",
        "print (u'path to root folder with manometer neural network data is:' + os.path.abspath(manometer_nn_data_path))\n",
        "\n",
        "# manometer train data = test (it is not good, but we are going to use this case)\n",
        "train_data_path = os.path.join(manometer_nn_data_path, u'train')\n",
        "test_data_path = os.path.join(manometer_nn_data_path, u'test')\n",
        "validate_data_path = os.path.join(manometer_nn_data_path, u'validate')\n",
        "\n",
        "# list sub folder (classification items)\n",
        "'''\n",
        "for item in os.listdir(train_data_path):\n",
        "    print(u'sub folder item: ' + item)\n",
        "'''\n",
        "img_width, img_height = 128, 128\n",
        "\n",
        "input_shape = (img_width, img_height, 3)\n",
        "\n",
        "epochs = 30\n",
        "\n",
        "batch_size = 6\n",
        "\n",
        "nb_classes = 33\n",
        "\n",
        "# todo: umv: count images\n",
        "nb_train_samples = 35\n",
        "\n",
        "vgg16_net = VGG16(include_top=False, input_shape=input_shape, pooling='max')\n",
        "# vgg16_net.summary()\n",
        "\n",
        "model = Sequential()\n",
        "model.add(vgg16_net)\n",
        "#model.add(Flatten())\n",
        "model.add(Dense(1024))\n",
        "model.add(Activation('relu'))\n",
        "model.add(Dense(nb_classes))\n",
        "model.add(Activation('softmax'))\n",
        "model.summary()\n",
        "\n",
        "# optimizer SGD - Stochastic gradient descent optimizer, has a lot of params\n",
        "# see optimizers here: https://keras.io/optimizers/\n",
        "model.compile(loss='categorical_crossentropy', optimizer='SGD', metrics=['accuracy'])\n",
        "\n",
        "datagen = ImageDataGenerator(rescale=1. / 255)\n",
        "train_generator = datagen.flow_from_directory(train_data_path,\n",
        "                                              target_size=(img_width, img_height),\n",
        "                                              batch_size=batch_size,\n",
        "                                              class_mode='categorical')\n",
        "\n",
        "# i don't know would we use validation or not ...\n",
        "validation_generator = datagen.flow_from_directory(validate_data_path, \n",
        "                                                   target_size=(img_width, img_height),\n",
        "                                                   batch_size=batch_size,\n",
        "                                                   class_mode='categorical')\n",
        "\n",
        "\n",
        "model.fit_generator(train_generator, \n",
        "                    steps_per_epoch=1,\n",
        "                    epochs=epochs,\n",
        "                    validation_data=None\n",
        "                    #validation_steps=nb_validation_samples // batch_size\n",
        "                   )\n",
        "\n",
        "nb_test_samples = 5\n",
        "test_generator = datagen.flow_from_directory(test_data_path,\n",
        "                                             target_size=(img_width, img_height),\n",
        "                                             batch_size=batch_size,\n",
        "                                             class_mode='categorical')\n",
        "\n",
        "# TEST generated neural network\n",
        "prediction = model.predict_generator(test_generator, nb_test_samples)\n",
        "print(prediction)\n"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "path to root folder with manometer neural network data is:/content/gdrive/My Drive/MessbauerSoft/NeuralNetworks_Data/Manometer\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "vgg16 (Model)                (None, 512)               14714688  \n",
            "_________________________________________________________________\n",
            "dense_66 (Dense)             (None, 1024)              525312    \n",
            "_________________________________________________________________\n",
            "activation_66 (Activation)   (None, 1024)              0         \n",
            "_________________________________________________________________\n",
            "dense_67 (Dense)             (None, 33)                33825     \n",
            "_________________________________________________________________\n",
            "activation_67 (Activation)   (None, 33)                0         \n",
            "=================================================================\n",
            "Total params: 15,273,825\n",
            "Trainable params: 15,273,825\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Found 33 images belonging to 33 classes.\n",
            "Found 0 images belonging to 0 classes.\n",
            "Epoch 1/30\n",
            "6/6 [==============================] - 1s 205ms/step - loss: 3.9454 - acc: 0.0000e+00\n",
            "Epoch 2/30\n",
            "6/6 [==============================] - 0s 55ms/step - loss: 3.7504 - acc: 0.0000e+00\n",
            "Epoch 3/30\n",
            "6/6 [==============================] - 0s 51ms/step - loss: 3.6664 - acc: 0.0000e+00\n",
            "Epoch 4/30\n",
            "6/6 [==============================] - 0s 63ms/step - loss: 3.6485 - acc: 0.0000e+00\n",
            "Epoch 5/30\n",
            "6/6 [==============================] - 0s 52ms/step - loss: 3.6282 - acc: 0.0000e+00\n",
            "Epoch 6/30\n",
            "6/6 [==============================] - 0s 63ms/step - loss: 3.5937 - acc: 0.0303\n",
            "Epoch 7/30\n",
            "6/6 [==============================] - 0s 51ms/step - loss: 3.6169 - acc: 0.0000e+00\n",
            "Epoch 8/30\n",
            "6/6 [==============================] - 0s 64ms/step - loss: 3.5625 - acc: 0.0000e+00\n",
            "Epoch 9/30\n",
            "6/6 [==============================] - 0s 50ms/step - loss: 3.5600 - acc: 0.0000e+00\n",
            "Epoch 10/30\n",
            "6/6 [==============================] - 0s 64ms/step - loss: 3.5223 - acc: 0.0000e+00\n",
            "Epoch 11/30\n",
            "6/6 [==============================] - 0s 50ms/step - loss: 3.4714 - acc: 0.0606\n",
            "Epoch 12/30\n",
            "6/6 [==============================] - 0s 66ms/step - loss: 3.3785 - acc: 0.0303\n",
            "Epoch 13/30\n",
            "6/6 [==============================] - 0s 51ms/step - loss: 3.3837 - acc: 0.0909\n",
            "Epoch 14/30\n",
            "6/6 [==============================] - 0s 62ms/step - loss: 3.2580 - acc: 0.0909\n",
            "Epoch 15/30\n",
            "6/6 [==============================] - 0s 54ms/step - loss: 3.2233 - acc: 0.0606\n",
            "Epoch 16/30\n",
            "6/6 [==============================] - 0s 60ms/step - loss: 3.2285 - acc: 0.0909\n",
            "Epoch 17/30\n",
            "6/6 [==============================] - 0s 53ms/step - loss: 3.5004 - acc: 0.0909\n",
            "Epoch 18/30\n",
            "6/6 [==============================] - 0s 62ms/step - loss: 3.1683 - acc: 0.1212\n",
            "Epoch 19/30\n",
            "6/6 [==============================] - 0s 54ms/step - loss: 3.0066 - acc: 0.1515\n",
            "Epoch 20/30\n",
            "6/6 [==============================] - 0s 62ms/step - loss: 2.7801 - acc: 0.0909\n",
            "Epoch 21/30\n",
            "6/6 [==============================] - 0s 53ms/step - loss: 3.1527 - acc: 0.1515\n",
            "Epoch 22/30\n",
            "6/6 [==============================] - 0s 60ms/step - loss: 2.7997 - acc: 0.1212\n",
            "Epoch 23/30\n",
            "6/6 [==============================] - 0s 53ms/step - loss: 2.8598 - acc: 0.1212\n",
            "Epoch 24/30\n",
            "6/6 [==============================] - 0s 61ms/step - loss: 2.4328 - acc: 0.1818\n",
            "Epoch 25/30\n",
            "6/6 [==============================] - 0s 53ms/step - loss: 2.4195 - acc: 0.3030\n",
            "Epoch 26/30\n",
            "6/6 [==============================] - 0s 62ms/step - loss: 2.2110 - acc: 0.3636\n",
            "Epoch 27/30\n",
            "6/6 [==============================] - 0s 54ms/step - loss: 1.5725 - acc: 0.3636\n",
            "Epoch 28/30\n",
            "6/6 [==============================] - 0s 61ms/step - loss: 1.7635 - acc: 0.4848\n",
            "Epoch 29/30\n",
            "6/6 [==============================] - 0s 52ms/step - loss: 2.5044 - acc: 0.2727\n",
            "Epoch 30/30\n",
            "6/6 [==============================] - 0s 63ms/step - loss: 1.5783 - acc: 0.6364\n",
            "Found 5 images belonging to 1 classes.\n",
            "[[0.03624989 0.0104452  0.01842199 0.03863705 0.00926134 0.00776412\n",
            "  0.04386811 0.03877321 0.03276113 0.09763247 0.0034531  0.03452554\n",
            "  0.00598234 0.00627609 0.01351872 0.04916716 0.02479527 0.03606032\n",
            "  0.05744633 0.03529778 0.01632055 0.02174292 0.03004006 0.02807529\n",
            "  0.04852898 0.05030997 0.03651778 0.0408971  0.02527314 0.02201227\n",
            "  0.02405757 0.03921041 0.01667671]\n",
            " [0.05737844 0.00796003 0.0476581  0.0785146  0.00923128 0.01241602\n",
            "  0.03336006 0.03849171 0.02874148 0.10759817 0.00470418 0.02899349\n",
            "  0.00714727 0.00571896 0.01689826 0.05400101 0.0209258  0.01883465\n",
            "  0.05788897 0.02891535 0.00814958 0.01236013 0.04940263 0.01628321\n",
            "  0.02341289 0.02270923 0.03976995 0.02336926 0.04016955 0.01576107\n",
            "  0.04018004 0.02719106 0.01586363]\n",
            " [0.06614926 0.00935175 0.02212459 0.05328618 0.00644669 0.01059096\n",
            "  0.03569921 0.02658899 0.02761524 0.09716625 0.00395676 0.02643806\n",
            "  0.00983507 0.00435065 0.01391126 0.05724528 0.03000821 0.02317563\n",
            "  0.04119319 0.0301652  0.01423867 0.01648325 0.07825228 0.02220909\n",
            "  0.02759613 0.03183759 0.03697866 0.02103511 0.034855   0.0268477\n",
            "  0.03060842 0.03998523 0.02377437]\n",
            " [0.048753   0.01358478 0.02946464 0.04416144 0.00649902 0.0154683\n",
            "  0.04214516 0.03393388 0.0348365  0.10041659 0.00622296 0.06153427\n",
            "  0.01066105 0.0079382  0.01558437 0.06644469 0.02042123 0.02065528\n",
            "  0.01963185 0.04356598 0.01285314 0.02544992 0.05005761 0.01569097\n",
            "  0.02615274 0.03721485 0.02954831 0.01515709 0.02019032 0.02622958\n",
            "  0.03079294 0.04928944 0.01944996]\n",
            " [0.0433542  0.01752217 0.02982989 0.05952336 0.00699285 0.00668217\n",
            "  0.03891253 0.02334117 0.00732629 0.08515152 0.0043116  0.05374608\n",
            "  0.00566748 0.0114318  0.02273621 0.07511409 0.02070301 0.01375116\n",
            "  0.0416393  0.03412486 0.01279855 0.02732053 0.05352048 0.01471297\n",
            "  0.03768894 0.04217323 0.05282048 0.00944298 0.02593746 0.03403711\n",
            "  0.01325956 0.04886628 0.02555968]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ObG0td0qBxY6"
      },
      "source": [
        ""
      ]
    }
  ]
}