Pani is a heavily modified version of the camera2basic app, built to simultaneously record 12MP RAW image data, pretty much all exposed camera/image metadata, and gyroscope + accelerometer measurements.
Warning: I am still actively working on this app, it crashes sometimes for reasons beyond my understanding, and sometimes for reasons beyond human understanding. Handle with care.
This code is built/tested for Pixel 6(a-Pro), 7(a-Pro), 8Pro.
- If your device does not lock installing applications from unknown sources, you can just directly install
Pani.apk
(see: releases) - Otherwise, you should enable developer mode on your device, open this project in android studio, build, and install
Alternatively, thanks to IzzySoft, you can also find this app on IzzyOnDroid:
- Swipe from the left of the screen to access the settings menu. This menu contains:
- Physical/ISP camera selection
Lock AE
→ Lock autoexposure during captureManual E
→ Set manual exposure, auto-populates ISO and exposure time to the most recent autoexposure valuesLock AF
→ Lock autofocus during capture, unlock this to set the app to continuous video-mode focusManual F
→ Manual set focus distance, auto-populates to most recent autofocus valueLock OIS
→ Turn off optical image stabilizationMax FPS
→ Max framerate at which data will be recorded, most camera streams only support a max of 30 or 60 depending on the device
- Below the viewfinder is a live readout of ISO, exposure time, and focus distance
Descriptor
→ suffix for saved data, recorded data is written toDocuments/{current-date-time}_{Descriptor}
Frames
→ number of frames recorded in captureBurn
→ if checked, will delete recorded data after capture is complete, useful for debugging and de-hiccuping (see later notes)Big Button
→ Hold down to start capturing of data, let go to end capture; wait for viewfinder to reset before recording the next capture
-
If the capture is succesful data is written to your devices
Documents
folder, this can be found in the File browser underInternal Storage
. A succesful capture contains a set of RAW images in.dng
format with associated metadata in binary format, as well asMOTION.bin
andCHARACTERISTICS.bin
, which are the recorded IMU measurements and physical camera properties (e.g., focal length), respectively -
Use
convert_files.py
to convert captured data into a.npz
numpy dictionary file viapython3 convert_files.py -d {path_to_capture_folder}
This can also be used to batch convert a folder of capture folders by pointing
-d
to the parent path -
To understand the format of the output
frame_bundle.npz
, which contains combined image, IMU measurements, and metadata, see the tutorial indata_format.ipynb
- App sometimes hangs and/or crashes. If a folder does not contain
CHARACTERISTICS.bin
, the last save file, you can assume data recording was interrupted and the capture is incomplete - App sometimes 'hiccups' on the very first capture after frame 40, this is what the
Burn
button was added for. Recommend burning a 60 frame capture when you first open the app, before recording any actual data to get rid of this hiccup - When
Manual E
is set, the viewfinder slightly dims. I think this is a bug with the camera2 preview stream, as the recorded data will appear brighter than the viewfinder - Don't select a camera that your phone does not have (e.g., telephoto on a Pixel 6a), app will crash
- Physical camera IDs are hard-coded. They should be correct for most Pixel devices, but I have not tested any other Android phones. Due to the non-trivial problem of determining which camera ID corresponds to which camera, these values are set in the radio button selector section of
CameraFragment.kt
(approx. lines 500-550). If they are wrong I recommend just manually testing numbers 1-10, it's easier than trying to dig through camera characteristics - To avoid accidental rotations, the app assumes you are recording in portrait mode, and .dng files will always be in portrait orientation
- If it's behaving weird try turning the app off and on again
This app was initially developed during my (ilya) summer 2023 Google internship, with a lot of support from my managers (Francois Bleibel and Kiran Murphy) and the wider Gcam team. A big thanks to them for making this project possible.
The original code this is based on is from https://github.com/android/camera-sample and is licensed under Apache 2.0 by The Android Open Source Project, see NOTICE
for a copy of their license.
Good luck,
Ilya