problem with adaptive streaming

[Rguzman2022]

PLEASE FOLLOW THESE INSTRUCTIONS BEFORE POSTING

1. Please thoroughly read README.md, INSTALL.md, GETTING_STARTED.md, and FAQ.md
2. Please search existing open and closed issues in case your issue has already been reported
3. Please try to debug the issue in case you can solve it on your own before posting

After following steps 1-3 above and agreeing to provide the detailed information requested below, you may continue with posting your issue

Video Support Issue

Expected results

We may do inference on a video file utilizing infer Simple video.py, and you can save the resulting video alongside the bounding boxes and/or masks.

Actual results

The infer simple.py utility may be used to execute inference on a directory of picture files (demo/*.jpg in this example). In this example, a ResNet-101-FPN foundation from the models zoo is used with a final generated Mask R-CNN model. The models should be downloaded by Detectron immediately from the URL supplied by the —wts parameter. The directory supplied by —output-dir will include PDF visualisations of the observations produced by this programme. Here is an illustration of the results you might anticipate seeing (for details on the copyright of the photos used in the demonstration, check demo/NOTICE).

What did you observe instead?

Mask R-CNN may be sluggish when performing inferences using your own high-quality photos since a lot of effort is required up sampling the projected filters to the actual picture quality . If the misc mask duration given by tools/infer simple.py is excessive, you can identify this problem . The fix is to first enlarge your photos so that the shorter side is somewhere between 600 and 800 pixels, and then perform inference on the smaller image.

E.g.:

def main(args):
                'rest (caches and auto-tuning need to warm up)'
            )

        image_ret = vis_utils.vis_one_image_opencv_copy(
        image_ret = vis_utils.vis_one_image_opencv(
            im,
            cls_boxes,
            cls_segms,

System information

• Operating system: window 11
• Compiler version: Jupyter Notebook
• CUDA version: cuda-10.2
• cuDNN version: cuDNN v8.7.0
• NVIDIA driver version: 285.27
• GPU models (for all devices if they are not all the same): Nvidia Quadro RTX 4000
• PYTHONPATH environment variable: c:\ user \ Administrator \ Desktop
• python --version output: Python 3.7.0
• Anything else that seems relevant: - Revise the description if our APIs have changed

[Aryan-Mishra24]

how do you want my response to be as a text response or a pull request

[Aryan-Mishra24]

here 's the text version let me know if you'd like a pull request
Thank you for providing detailed information about your issue. Based on the description, it seems that you would like to perform inference on a video file using Detectron and save the resulting video with bounding boxes and/or masks.

The infer_simple.py utility provided by Detectron is designed to work on image files, not video files. However, you can modify the script to process video files frame by frame, draw the bounding boxes and masks on the frames, and save them as a new video.

Here's an example of how you can modify the infer_simple.py script to work with video files:

First, you need to install OpenCV Python package (if you haven't already) to handle video processing:

pip install opencv-python
Then, import the required libraries in the infer_simple.py script:
python

import cv2
import numpy as np
Replace the main function in the infer_simple.py script with the following code:
python
def main(args):
cfg.merge_from_file(args.cfg)
cfg.merge_from_list(args.opts)
cfg.freeze()
model = infer_engine.initialize_model_from_cfg(args.weights)
dummy_coco_dataset = dummy_datasets.get_coco_dataset()

# Read video file
video = cv2.VideoCapture(args.video_file)
frame_width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
frame_height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = int(video.get(cv2.CAP_PROP_FPS))

# Define the codec and create VideoWriter object to save the output video
fourcc = cv2.VideoWriter_fourcc(*'XVID')
out = cv2.VideoWriter('output.avi', fourcc, fps, (frame_width, frame_height))

while video.isOpened():
    ret, frame = video.read()
    if not ret:
        break

    cls_boxes, cls_segms, _ = infer_engine.im_detect_all(model, frame, None)

    vis_frame = vis_utils.vis_one_image_opencv(
        frame,
        cls_boxes,
        cls_segms,
        thresh=0.7,
        dataset=dummy_coco_dataset,
        show_box=True,
        show_class=True
    )

    # Write the frame with the detections into the output video
    out.write(vis_frame)

# Release everything when the job is finished
video.release()
out.release()
cv2.destroyAllWindows()

Modify the argument parser section to accept video files as input:
python

parser.add_argument(
'--video-file',
dest='video_file',
help='Path to the video file you want to process',
default=None,
type=str
)
Finally, run the modified infer_simple.py script with the required arguments, including the path to the video file:
bash

python tools/infer_simple.py
--cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml
--weights https://dl.fbaipublicfiles.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl
--video-file path/to/your/video_file.mp4
After running the script, you should get an output.avi video file with the bounding boxes and masks drawn on each frame.

[0x00021]

nice