extract the android screen as opencv Mat or numpy array

[mrnth]

Hello bro,
If you know any open source or python library help extract the android screen as opencv Mat or numpy array, please share it to me. Thank you so much

[imneonizer]

Hey @mrnth,
This repository does exactly the same, except the codebase is very old.

[mrnth]

Hey @mrnth,
This repository does exactly the same, except the codebase is very old.

i want to hide all application window when running my python script. So, I can not capture pc screen.
but I found a android sw help stream android screen to HTTP then iI can get realtime frame by opencv.
it's very useful even though the latency is quite high
https://play.google.com/store/apps/details?id=info.dvkr.screenstream&hl=en&gl=US
Thanks for your reply

[imneonizer]

Wow!! Thanks for sharing the application.
Indeed you can capture the screen using OpenCV.

On top of that, if you are facing high latency I would suggest to use USB Debugging + ADB and do port forward via USB for a low latency streaming.

[mrnth]

Thanks for the suggestion.
I found the command to cast android screen using ffplay on Linux: "adb shell screenrecord --output-format = h264 - | ffplay -framerate 60 -probesize 32 -sync video -"
But I don't know how to get frame using opencv in my python script. If you know more information. please give me some keywords or suggestions, thanks a lot

[imneonizer]

I will suggest sticking with the screen stream application,
You can directly access the stream using opencv by passing the url into VideoCapture object.

Example:

import cv2

device_ip = "192.168.0.44"
cap = cv2.VideoCapture(f"http://{device_ip}:8080/stream.mjpeg")

while True:
    success, frame = cap.read()
    if not success:
        break

    cv2.imshow("frame", frame)
    if cv2.waitKey(1) == ord("q"): 
        break

If you want to cast screen via port forwarding using USB, open the screen stream aplication go to settings>advanced and enable locahost and start streaming.
Now connect your device to the system in USB Debugging mode and do a port forwarding

adb forward tcp:8080 tcp:8080

After this, you can visit locahost:8080 to view the screen cast on your system

[mrnth]

Thank you so much