MP4InputStream async

[stefc]

It would be very helpful if the MP4InputStream working also async so I can iterate async over track.ReadNextFrame() like

frame = await track.ReadNextFrameAsync(cancelationToken)

public async IAsyncEnumerable<Memory<short>> DecodeAsync(string fileName, [EnumeratorCancellation] CancellationToken cancellationToken = default)
    {   
        Decoder dec = new(track.GetDecoderSpecificInfo());
        Frame frame;
        SampleBuffer buf = new();

        while (track.HasMoreFrames())
        {
            frame = track.ReadNextFrame();
            dec.DecodeFrame(frame.GetData(), buf);

            var memory = buf.Data.AsMemory();

            var resizeFactor = sizeof(short) / sizeof(byte);
            var shortMemory = Unsafe.As<Memory<byte>, Memory<short>>(ref memory)[..(memory.Length / resizeFactor)];


            await Task.Yield();  // => this I wish can avoid if a ReadNextFrameAsync would exists ;) 

            yield return shortMemory;
        }
    }

[jimm98y]

This library is just a port of the Java code base with some fixes and enhancements. The problem is the original library was not very well prepared for async and rewriting it completely would be a significant effort. And I don't think it's necessary, because in order to accomplish what you are asking for, you could simply treat the library as a synchronous "black box" and supply your own thread to run it. Something like:

public Task<Memory<short>> DecodeAsync(string fileName, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{   
    return Task.Run(() => 
    {
           Decoder dec = new(track.GetDecoderSpecificInfo());
           Frame frame;
           SampleBuffer buf = new();

           while (track.HasMoreFrames())
           {
               frame = track.ReadNextFrame();
               dec.DecodeFrame(frame.GetData(), buf);

               var memory = buf.Data.AsMemory();

               var resizeFactor = sizeof(short) / sizeof(byte);
               var shortMemory = Unsafe.As<Memory<byte>, Memory<short>>(ref memory)[..(memory.Length / resizeFactor)];

               return shortMemory;
           }
   });
}

Of course you must ensure that you are not accessing one parser instance from multiple threads because the library is not thread safe. If you want partial results, you can supply an Action<Memory<short>> delegate and call it from the loop.

[stefc]

Also this is now working on may side pretty well. I iterate now with an IAsyncEnumerable over the frames and accessing the lib with an Task.FromResult so I can avoid the Task.Yield call. I also improve the iterating itself so I can cut out only a range instead looping over the whole track.

private static Task<Memory<short>> DecodeBufferAsync(SampleBuffer buf)
{
    var memory = new Memory<byte>(new byte[buf.Data.Length]);
    buf.Data.CopyTo(memory);
    var resizeFactor = sizeof(short) / sizeof(byte);
    var shortMemory = Unsafe.As<Memory<byte>, Memory<short>>(ref memory)[..(memory.Length / resizeFactor)];
    return Task.FromResult(shortMemory);
}

public async IAsyncEnumerable<FrameData> DecodeTimeSpanAsync(TimeSpan start, TimeSpan end)
{
    Decoder dec = new(track.GetDecoderSpecificInfo());
    SampleBuffer buf = new();

    var durationQuotient = 1000.0 / SampleRate; // in ms

    var startTime = start.TotalSeconds - durationQuotient;
    var endTime = end.TotalSeconds - durationQuotient;

    var cursor  = track.Seek(startTime);  // one frames before start
    while (track.HasMoreFrames() && cursor < endTime)
    {
        var frame = track.ReadNextFrame();
        dec.DecodeFrame(frame.GetData(), buf);

        var shortMemory = await DecodeBufferAsync(buf);

        cursor = frame.GetTime();
        var position = (byte)(cursor / TrackLength.TotalSeconds * 100);

        yield return new FrameData { Data = shortMemory, Time = cursor, Position = position };
    }
}

[jimm98y]

Nice, thanks for sharing! However, if you were to run this from a UI thread, you'd still block it until you read the frame, which might be perfectly fine depending upon your use case.

[stefc]

For me is fine because all I do is use it from a CLI tool for ML purposes I scan a 1h podcast in under a second which is super ! Lukas Volf ***@***.***> schrieb am Do. 14. Dez. 2023 um 11:12:

…

Nice, thanks for sharing! However, if you were to run this from a UI thread, you'd still block it until you read the frame, which might be perfectly fine depending upon your use case. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADROOBFMLAAP7LXRNEF7YTYJLGKVAVCNFSM6AAAAABAOULBOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGU2TSMRUGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>