Scouter

Scouter: A human face detector which displays your Github contribution statistics.

Check Slides for GDG DevFest 2019 Kaohsiung

Google Slides

Check Presentation Stream on COSCUP 2018

Presentation On COSCUP 2018

Brief

• Fetch data with Github API
  • Github user data
  • Github user avatar
  • Parse Github contribution HTML to get user contribution statistics
• Convert avatar to identity with Face Recognition API. Encoding avatar identity with userId.
• Track face and crop face image from camera streaming with OpenCV
• Send face image to Flask API server
  • Convert unknown face image to identity.
  • Get userId and contribution statistics with identity.
• Send user contribution to App and display.

MongoDB

1. Have a local running mongoDB using docker

make db

2. (Optional) Import dump user data

make migrate

# Check user data in mongodb
docker exec -it mongo mongo scouter --eval "printjson(db.users.findOne())"
docker exec -it mongo mongo scouter --eval "printjson(db.users.count())"

Skip 'Fetching data with Github API' if using dump data

(Optional) Fetching data with Github API

Generate Github access token

1. Github -> User -> settings -> Developer settings -> Personal access tokens
2. Keep your token safe.

Run fetchers with token

0. Install go

1. Fetch user data with Github Search API

GITHUB_ACCESS_TOKEN=
go build ./cmd/user_fetcher && ./user_fetcher -token ${GITHUB_ACCESS_TOKEN}

2. Fetch user detail information like follwers and repos with Github User API

go build ./cmd/user_detail_fetcher && ./user_detail_fetcher -token ${GITHUB_ACCESS_TOKEN}

3. Fetch users' avatar with user.url from data in mongodb

go build ./cmd/avatar_downloader && ./avatar_downloader

4. Fetch users' contribution statics by parsing html response of user.url from data in mongodb

go build ./cmd/contribution_fetcher && ./contribution_fetcher

5. Make sure user avatar are good to go

ls data/avatars

Face detection and Face recognition

Face Recognition API

0. Prepare a python virtual env

python3 -m venv .venv
source ./.venv/bin/activate

1. Install python dependency

pip3 install dlib flask face_recognition pymongo bson

2. Try some face recognition API

face_recognition --show-distance true --tolerance 0.54 ./pictures_of_people_i_know/ ./unknown_pictures/

3. Prepare face identity file with encoding generator

# Filter data/avatars image. Save images with human faces to data/human_face.
# Generate face_recognition/encodings and face_recognition/index with data/human_face
python ./face_recognition/encoding_file_generator.py

4. Run apiserver to serve face recognition API

python ./face_recognition/apiserver.py

5. Send imageBytes to api server

PostRequest("http://"+apiserverip+":5000/face_detection", imageBytes)

Unity

1. Download latest Unity with iOS Build tools

2. Create a new project

3. Download and import a free face tracker example from unity asset store Face Tracker Example

4. Download and import another priced asset is required: OpenCV for Unity NOTE: This is a priced asset.

5. Copy unity scenes and scripts

cp -r unity/Assets/Scouter/* <unity-workspace>/<your-project>/Assets/FaceTrackerExample

6. Open Unity and setup. File -> Build Settings -> Player Settings

Mac App Store Options -> Bundle Identifier: com.chechiachang.scouter
Configuration -> Scriptin Runtime Version: .NET 4.x Equivalent -> restart

7. Open Scenes/WebCamTextureFaceTrackerExample

Inspector -> Web Cam Texture Face Tracker Example -> Apiserverip: 
- 127.0.0.1 on your localhost mac
- 172.20.10.3 with personal hotspot of Iphone

Run

Project Configuration

1. Open unity build for iphone project with xcode. Open another project.
2. Xcode developer account:

• Xcode - Preferences - Accounts: Add and login your 'apple developer ID'.
• The team of your developer account will show up.
• In my case, A personal team show up with my username as team name.

3. Signing:

• Click my-project. The project configure page will show up.
• General - Identity: Change your display name and Bundle Identifier. Any reasonable identifier other than the example identifier will work.
• General - Signing: Check 'Automatically manage signing'.
• Choose your team. A signing certificates will show up.
• If you stuck here, check your bundle identifier.

Build project

1. Attach your device (your iphone). Unlock your iphone.
2. Click 'Build and Run Current Schema'.

TODOs

• Github API crawler
  • Add an API to search user in Taiwan
    • Order by joined asc
    • Implement a API call with narrowed search condition
  • Add an API to fetch user Data
    • Get user with userUrl
    • order by most follower
    • order by most commit. Might need query by username.
  • Save user data to mongodb
    • username
    • avatar
    • # of follower
    • # of contributions
• Google Search API Face downloader
  • Search Avatar with Github username and login
  • Google Custom Search API Google Custom Search API
• Face Recognizer
  • Face detector
    • Generate face encoding and save to Python Pickle file
    • Face Recognition Face recognition
• Front-End
  • Unity ios app
  • API portal
  • AR GUI
• Readme