retdec-fileinfo runs 40 seconds and consumes 8 GB of RAM on 28 KB file

[s3rvac]

retdec-fileinfo runs 40 seconds and consumes 8 GB of RAM on 28 KB file

Steps to reproduce

$ retdec-fileinfo FILE

where FILE is E556E9A67BBED5B6C32BE434A148841FEFEE81EB39432D427CB08FEFB0098173.

Expected output

retdec-fileinfo analyzes the file in a reasonable time, using a reasonable amount of memory.

Actual output

$ /usr/bin/time -v retdec-fileinfo E556E9A67BBED5B6C32BE434A148841FEFEE81EB39432D427CB08FEFB0098173.dat
Input file               : E556E9A67BBED5B6C32BE434A148841FEFEE81EB39432D427CB08FEFB0098173.dat
[..]
Overlay size             : 0x6a6c
        [..]
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:39.25
        [..]
        Maximum resident set size (kbytes): 8375140
        [..]
        Exit status: 0

That is, it ran for almost 40 seconds and consumed 8 GB of memory.

Notes

Before the image-loader changes, retdec-fileinfo ran for 2 seconds and consumed 2 GB of RAM.

My configuration

• Current master (commit 4e31e3a)
• 64b Debian, GCC 9.2, Debug build

[ladislav-zezula]

#873
avast/retdec-regression-tests#78

[s3rvac]

Fixed by @ladislav-zezula in #873.