Detect and decode bursts that straddle multiple work() calls

[mhostetter]

No description provided.

[potto216]

Assuming I understand the problem I thought one way to test if a fix worked would be to capture the RF data from the SDR to a file, then feed the file through the ADS demod blocks with different amounts of delay inserted at the beginning of the file to cause bursts to straddle the work calls. If the fix worked then no matter the amount of shift at the beginning the same number of bursts would be detected.

As a quick test I captured my SDR's raw data to a file while the ADS was processing live. I then ran the collected file through the same ADS processing blocks without any delay. I expected the same detects as the live processing. However, when I ran the file though and compared the output in the GRC debugging console with the live capture I found that the file capture had lat/longs for flights which were blank in the live collect. Additionally the "Msgs", "Age" values were different.

What was especially interesting is that when I repeated running the file with no delay through the ADS processing blocks and compared it to the previous run with the file through the processing blocks I expected the exact same output. Instead what I noticed is that the "Msgs", "Age" often differed and in the second run I saw the runtime warnings:
/usr/lib/python2.7/dist-packages/numpy/core/fromnumeric.py:2909: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/lib/python2.7/dist-packages/numpy/core/_methods.py:80: RuntimeWarning: invalid value encountered in true_divide
ret = ret.dtype.type(ret / rcount)

But in the live run and the first run of the file there were not any runtime warnings in the output.

Do you know if this is expected behavior or would you like to see the console outputs? Also is this a reasonable way to test if a fix for this issue worked?

[mhostetter]

You might be onto something here. Yes, the console outputs would be helpful.

One thing to note is that the decoder is stateful. The decoding of particular messages are dependent on previously decoded ones. I wouldn't expect the decoder to work correctly if previous data was reprocessed through it (like a repeated file) in one flowgraph run. I think you'd need to stop and restart the flowgraph to avoid this issue. Are you stopping and re-running the flowgraph?

[mhostetter]

Regarding testing correct operation with bursts spanning multiple work calls, I think you're on the right track. However, I think you can do what you mention even more precisely using the GNU Radio's python interface. This is what's used for the QA unit tests.

Below is an example (untested) of what this could look like. This could (and should) eventually go into gr-adsb/python/qa_framer.py.

The idea is to create a test vector (that's short and hopefully "clean", i.e. high SNR) containing only 1 burst. That complex signal would be saved in in_sig. You can vary work_noutput_items to force the work call size (I think this should work, but needs testing). If the framer detects a burst, it will add a tag to the output stream. You can check for the presence of the tag and verify its sample offset.

class qa_framer(gr_unittest.TestCase):

    def setUp(self):
        self.tb = gr.top_block()

    def tearDown(self):
        self.tb = None

    def test_burst_span_multiple_work_calls(self):
        fs = 2e6 # samples/second
        threshold = 0.01
        in_sig = np.asarray([...])
        out_sig = np.abs(in_sig)**2
        tag_key = "burst"
        tag_offset = 123
        work_noutput_items = 256

        source = blocks.vector_source_c(in_sig, False, 1)
        mag2 = blocks.complex_to_mag_squared(1)
        framer = adsb.framer(fs, threshold)
        sink = blocks.vector_sink_c(1)

        # Force framer to require a minimum of `work_noutput_items` samples
        framer.set_min_noutput_items(work_noutput_items)

        self.tb.connect(source, mag2)
        self.tb.connect(mag2, framer)
        self.tb.connect(framer, sink)

        # Force scheduler to set a maximum of `work_noutput_items` samples per work call
        self.tb.run(work_noutput_items)

        self.assertEqual(len(sink.tags()), 1)
        self.assertEqual(pmt.to_python(sink.tags()[0].key), tag_key)
        self.assertEqual(pmt.to_python(sink.tags()[0].offset), tag_offset)
        self.assertEqual(sink.data(), sink_data)

[potto216]

Okay this is great because I wanted an excuse to learn how to do QA tests for GNURadio:). I'll work on it this weekend and also try to extract a high quality burst from my collect. For the issue I described previously:

My live run console output is here:
ads-b-live-run.txt

I restarted the flowgraph and fed in the captured I/Q data from the live collect and its output is here:
ads-b-file-run1.txt

I restarted the flowgraph and repeated the file processing so I believe the state should have been reset. This has the Python warnings.
ads-b-file-run2.txt