VC ("virtual content") is a tiny JEMSCI app that generates explorable dream worlds. It's built on top of the following machine vision libraries:
- VQGAN: https://github.com/CompVis/taming-transformers
- CLIP: https://github.com/openai/CLIP
- DPT (via MiDaS): https://github.com/isl-org/MiDaS
- 3D Photo Inpainting: https://github.com/vt-vl-lab/3d-photo-inpainting
- ESRGAN: https://github.com/xinntao/Real-ESRGAN
- RIFE: https://github.com/hzwer/arXiv2020-RIFE
- A few things you're going to need to install if you haven't yet:
sudo snap install aws-cli --classic
sudo apt install jq postgresql-client redis-tools
curl -sL https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.0/install.sh -o install_nvm.sh
bash install_nvm.sh
source ~/.bash_profile
nvm install 16.13.0 # (yes, not v17 - it has a problem with OpenSSL)
npm install -g npx
- Spin up a
g4dn.xlarge
EC2 instance from the Deep Learning Base AMI (Ubuntu 18):
aws ec2 run-instances \
--image-id ami-0bdd0109e841ac9fc \
--count 1 \
--instance-type g4dn.xlarge \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=vc}]' \
--key-name MyKeyPair
Bear in mind this costs more than 50c an hour, i.e. £9 a day or £270 per month, as long as the instance is running. All you need to do is stop the instance when you're not using it. It's a good idea to associate an Elastic IP to your instance.
To stop/start your instance, simply:
make start
make stop
- Create an S3 bucket for your results (the bucket name must be universally unique):
aws s3api create-bucket \
--bucket unique-bucket-name \
--region eu-west-1 \
--create-bucket-configuration LocationConstraint=eu-west-1
- Add your new box to
~/.ssh/config
:
Host vc
HostName static.ip.goes.here
User ubuntu
IdentityFile path/to/your/key.pem
- SSH into the box:
ssh vc
- Set up your work dir:
sudo mkdir /opt/vc
sudo chown ubuntu:ubuntu /opt/vc
echo >> ~/.bashrc
echo "cd /opt/vc" >> ~/.bashrc
echo "source venv/bin/activate" >> ~/.bashrc
echo "export FLASK_APP=vc.app:app" >> ~/.bashrc
echo >> ~/.bashrc
cd /opt/vc
git clone https://github.com/alex-moon/vc.git .
-
cp .env.example .env
and add your AWS credentials and unique bucket name to.env
-
If you don't want to run a worker on this machine, remove the following line from your
.env
:
ROLE=worker
- Build:
./build.sh
- Move your nginx conf into place:
sudo cp nginx.unsecure.conf /etc/nginx/sites-enabled/vc.conf
sudo service nginx restart
- Move your supervisor conf into place:
sudo cp supervisor.api.conf /etc/supervisor/conf.d/vc.api.conf
sudo cp supervisor.worker.conf /etc/supervisor/conf.d/vc.worker.conf # if you want to run the worker
sudo service supervisor restart
- Visit http://static.ip.goes.here in your browser
It's advisable to set up SSL certificates for your server. This requires you to have DNS set up and some familiarity with configuring nginx.
- Set up your SSL certs:
sudo snap install core
sudo snap refresh core
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot certonly --nginx
- Copy the secure conf into place:
sudo cp nginx.conf /etc/nginx/sites-enabled/vc.conf
-
Modify
/etc/nginx/sites-enabled/vc.conf
to match your hostname and certificate filepaths. This should be as straightforward as replacingvc.ajmoon.uk
with your domain name. -
Restart nginx:
sudo service nginx restart
If you want to work on the front-end locally, simply install node:
brew update
brew install node
npm install
And then run webpack:
make serve
And visit http://localhost:8000/ in your browser.
If you want to run the API on docker locally, create a copy of your .env without ROLE=worker
in it and then simply:
make build run