Correct inclination of image based on gyroscope

[hflemmen]

Details

To make the human pose estimation more robust to rotations, we can transform the image to what will in the image frame be zero-rotation. We can get the rotation from the ardupilot EKF2.

Tasks

• Find the correct mavros topic to get IMU data
• Apply correction
• Find best way to handle corners
• Make the tests

Criteria

• The image is rotated responding to the drone inclination.

[hflemmen]

Use topic: /mavros/imu/data

[tenfoldpaper]

https://docs.px4.io/v1.9.0/en/config/flight_controller_orientation.html
https://docs.px4.io/v1.9.0/en/advanced_config/advanced_flight_controller_orientation_leveling.html
These links show that there should be some output from PX4 that directly tells us the yaw, pitch and roll angles. If we can modify the mavros interface to output those angles on top of the quaternion orientation, we could greatly simplify the computation.

[tenfoldpaper]

Basic algorithm is ready. Depending on the image type that we feed into the neural network, if we can use typical image types, we could integrate Python Image Library which has an easy built-in image rotation function.

[hflemmen]

https://docs.px4.io/v1.9.0/en/config/flight_controller_orientation.html
https://docs.px4.io/v1.9.0/en/advanced_config/advanced_flight_controller_orientation_leveling.html
These links show that there should be some output from PX4 that directly tells us the yaw, pitch and roll angles. If we can modify the mavros interface to output those angles on top of the quaternion orientation, we could greatly simplify the computation.

I am not sure that this means that the PX4 outputs yaw, pitch and roll. These pages discusses how to configure the orientation of the FC, not output.

Basic algorithm is ready. Depending on the image type that we feed into the neural network, if we can use typical image types, we could integrate Python Image Library which has an easy built-in image rotation function.

Nice! The image is initially a ros image message, but can easily be converted to a opencv type with cv bridge.

[FabianBell]

I will do it from now on.

[hflemmen]

Use topic: /mavros/imu/data

Perhaps a note on how to find relevant mavros topics, since this might not be exactly the one we want. The docs are out of date. I.e. the topics there still exist, but others exist too. The best way to find the topic you are looking for is to run mavros with either a simulator or the FC on the pi and look at the available topics with rostopic list. Then you can also look at the output it gives with rostopic echo <topic_name> or see the message type with rostopic info <topic_name>. More help from rostopic -h.

[FabianBell]

We have the following problems:

1. None of the IMU topics is publishing anything
2. I get "No handlers could be found for logger "rosout"" which seems to be no problem in many cases but http://wiki.ros.org/rospy_tutorials/Tutorials/Logging says that it is due to an incorrect initialisation

The Mavros state is

seq: 0
  stamp: 
    secs: 1595431290
    nsecs: 145989841
  frame_id: ''
connected: False
armed: False
guided: False
manual_input: False
mode: ''
system_status: 0

@hflemmen Did I do something wrong?

[hflemmen]

1. Did you test them all? It seems like a lot of them are inactive, might be they are used with px4 and not with ardupilot. There might be some things you can try:

First of all, make sure that there is an connection to the FC, i.e. that it is connected(and powered) and receiving mavlink hearbeats.

If not, then you can check if LOG_DISARMED is set, that LOG_BACKEND_TYPE is at least set to mavlink and the other logging parameters.

2. I first saw that on one of the workshops we had with drive in January. I have only seen it on the pi, but drive also got it. I don't know what it is, but I have had great succsess until now with just ignoring the warning and everything has worked as normal.

[FabianBell]

1. Did you test them all? It seems like a lot of them are inactive, might be they are used with px4 and not with ardupilot. There might be some things you can try:

First of all, make sure that there is an connection to the FC, i.e. that it is connected(and powered) and receiving mavlink hearbeats.

If not, then you can check if LOG_DISARMED is set, that LOG_BACKEND_TYPE is at least set to mavlink and the other logging parameters.

1. I first saw that on one of the workshops we had with drive in January. I have only seen it on the pi, but drive also got it. I don't know what it is, but I have had great succsess until now with just ignoring the warning and everything has worked as normal.

Okay so it is connected properly but I still do not receive any signal. Where are the log files located?

[hflemmen]

Which log files do you mean? I think you can get the one from the FC through QGC. Not sure about the ROS ones. Perhaps somewhere in ~/.ros.

The log parameters have to be set so that the FC will send the relevant data. Sending data over the serial link is implemented in the same way as logging.

[hflemmen]

In the simulator I get some data on(amongs others):

• mavros/global_position/local
• mavros/local_position/pose
• mavros/imu/data
• mavros/imu/data_raw

I guess first thing to try is to see if one of the first two can be decomposed into giving the inclination in roll direction. If not we may need to calculate it from one of the last two.