Table of Contents
Deep Dream is a computer vision tool, created by Google's engineer Alexander Mordvintsev, to help us understand how neural networks work. It uses the convolutional layers of Neural Networks to find and enhance patterns in images, thus intentionally creating dream-like, psychedelic, over-processed images.This work is mainly based on the tiled implementation of the Deep Dream Tensorflow tutorial found here. Subtle details of the implementation have been changed and it is currently under work. In addition, a wrapper and a script have been constructed to choose from various models and run experiments on the algorithm with different configurations.
The examples below have been generated from a starting input image:
- These examples were created with lower network layers and smaller octave range in order to create more subtle dreamifications, preserving the shapes and structures of the original image.
- The following examples on the other hand, were created with deeper layers and more steps per octave. In general, we can observe that the images got a deeper modification, with new shapes and structures appearing in the image.
The examples below have been generated from random noise, allowing the network get 'creative' with its representations:
- We can observe how the noise gets interpreted as leafs and branches of trees, with some shapes recalling animals and birds.
“Google Dream (...) is a tool, not the product, so calling it art would be a little like an artist raising their hand and declaring their paintbrush art because they were so happy with the way they used it lay paint on a canvas.”
- Paddy Johnson
Although Deep Dream cannot exactly be considered art, its applications are intriguing and representative of our brain as explained by Memo Akten from the citation below.
"It might look like Deep Dream is generating say, sparrow’s faces in clouds, but what it is actually doing is generating patterned noise, which our brains try to find meaning in. It creates just enough of a sparrow’s head in a cloud, so that our brains find the rest. Visually, our minds and Deep Dream are doing exactly the same thing. It’s such a perfect mirror. I love that conceptual aspect."
- Memo Akten
Python3 version 3.8 ~ 3.10 is required, with the following packages installed:
numpyPillowMatplotlibTensorflow>= 2.8.0
CUDA(to run Tensorflow on GPU)
The main file to run experiments is the deep_dream.py file in the main directiry. An accurate description of all the arguments is available below.
The script deep_dream.sh in the main directory is also available as an example to set parameters and run experiments.
-img: Path of the input image to use for the experiment. It can be set to 'random_noise' (set as default) in order to create a random normal distributed noise to pass to the deep dream model.-img_out: Name for the output image. Default is 'output'.-img_max_dim: Resize input image to a maximum dimension, which will be used to create the output image. Aspect ration will be kept. Default value is 1024.-save_input: When used, the flag will save the input image resized to the maximum dimension defined with the previous flag -img_max_dim.-ts: Size of the tile for the image roll. Should be a number between 0 and img_max_dim. Default is 512.-ss: Step size to update our image at every iteration. Some good values for this parameter are between 0.001 and 0.1 . Default value is 0.01 .-osc: Octave scale. Scale parameter that determines the size of the image at every octave. Its default value is 1.2 .-or: Octaves range. This parameter takes as input two values, representing the starting and ending indexes of the octave range. Each value of the octave is multiplied by osc to get the size of the image for the current octave. Choosing negative values will make the image smaller than the original, resulting in filters that have more impact to the global structure and shapes. Higher octave values on the other hand, will make the image bigger than the original, preserving the overall structure and shapes of the image while making the amplifications appear more like adding details. The default values are [-4, 2] creating the octave range [-4, -3, -2, -1, 0, 1] .-os: Number of iterations to run for each octave. The default value is 80.-m: String parameter representing the base model to use for the deep dream. Possible values for this parameters is represented by the models currently implemented. For now this parameter can be set to either 'inceptionV3' or 'inceptionResNet' .-ml: Range of the model layers to use for the deep dream. If left blank or wrongly chosen, the application will ask you to complete this parameter during runtime, after informing the user of the available range of layers. For 'inceptionResNet' model the available range is from 0 to 43, while for the 'inceptionV3' it is from 0 to 15 .
Add more args for more detailed controlCreate bash script to run experiemnts.- Anneal value of tile size and other params between steps.
The following links were used as reference to build the framework: