Getting VIPS_LEAK output

[kipcole9]

For an upcoming meetup I'd like to be able to show the memory benefits that come from libvips. I'm using export VIX_COMPILATION_MODE=PLATFORM_PROVIDED_LIBVIPS for the build process but I see the same results using the precompile libvips and NIF.

A follow-up question: Is there a function I can call to get the memory high-water mark at runtime? (and thread count too if possible).

My example code is:

kip@Kips-MacBook-Pro image % VIPS_LEAK=1 iex -S mix
Erlang/OTP 26 [erts-14.2] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] [jit] [dtrace]

Interactive Elixir (1.16.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> i = Image.thumbnail! "./test/support/images/Kamchatka-2019-8754.jpg", 200
%Vix.Vips.Image{ref: #Reference<0.3808751778.2887122972.162784>}
iex(2)> Image.write i, "/tmp/nothing.jpg"
{:ok, %Vix.Vips.Image{ref: #Reference<0.3808751778.2887122972.162784>}}
iex(3)> %
kip@Kips-MacBook-Pro image %

I'm expecting to see the VIP_LEAK debug message after a clean shutdown which I am doing with CTRL-\

I know VIPS_LEAK is set because if I use the vips command line I see the expected debug output:

kip@Kips-MacBook-Pro image % vips --version
vips-8.15.2
vips_threadset_free: peak of 0 threads
memory: high-water mark 0 bytes

Any help or suggestions welcome!

[kipcole9]

I tried just running a script to see if that might avoid an iex interactions. Also produced no output.

I wonder if there is a possibility to add a NIF api call to vips_leak()

Additionally I wonder if somehow vips_shutdown() isn't being called?

How would I go about compiling libvips with DEBUG set to true?

[akash-akya]

@kipcole9 did you nuke the _build artifacts before compiling? If prebuilt artifact is already present then NIF won't be compiled.

You can check if vix is using prebuilt libvips or not using otool, if you are on mac.

$ otool -L ./_build/dev/lib/vix/priv/vix.so | grep libvips.42.dylib
	@rpath/libvips.42.dylib (compatibility version 60.0.0, current version 60.0.0)

When compiled against platform libvips then it would look like this

$ otool -L ./_build/dev/lib/vix/priv/vix.so | grep libvips.42.dylib
	/usr/local/lib/libvips.42.dylib (compatibility version 61.0.0, current version 61.0.0)

[kipcole9]

@akash-akya, all good on that part. I was just trying to say that I can't get any output when I invoke with VIPS_LEAK=1 iex -S mix and shut down the VM cleaning with System.stop/0.

Basically, I'm just trying to get the memory high water mark for libvips. And I'd be super happy if there was a NIF interface to vips_leak().

[akash-akya]

@kipcole9 can you try the latest dev branch?

example.exs

:ok = Vix.Vips.set_vips_leak_checking(true)

## Code
{:ok, _im} = Vix.Vips.Image.new_from_file("test/images/puppies.jpg")

:ok = Vix.Nif.nif_vips_shutdown()

Run the script

$ mix example.exs
vips_threadset_free: peak of 0 threads
3 objects alive:
0) VipsImage (0x7fd9979042c0) 1032 bytes, count=1
VipsImage (image), image class, 518x389 uchar, 3 bands, srgb, jpegload
1) VipsForeignLoadJpegFile (0x600002d04000), count=1
VipsForeignLoadJpegFile (jpegload), load jpeg from file (.jpg, .jpeg, .jpe, .jfif), priority=50, is_a, get_flags, get_flags_filename, header, load, jpegload filename="/Users/akash/repo/elixir/vix/test/images/puppies.jpg" out=((VipsImage*) 0x7fd9979042c0) flags=((VipsForeignFlags) VIPS_FOREIGN_SEQUENTIAL) -
2) VipsSource (0x600002d040c0), count=2
VipsSource (source), input source,
15 VipsArea alive
	0x600000d3cd80 count = 1, bytes = 54
	0x600000d3c740 count = 1, bytes = 54
	0x600000d3cc80 count = 1, bytes = 75
	0x600000d3cc00 count = 1, bytes = 69
	0x600000d3cb80 count = 1, bytes = 38
	0x600000d3cb00 count = 1, bytes = 38
	0x600000d3ca80 count = 1, bytes = 38
	0x600000d3ca00 count = 1, bytes = 38
	0x600000d3c500 count = 1, bytes = 42
	0x600000d3c940 count = 1, bytes = 42
	0x600000d3c8c0 count = 1, bytes = 42
	0x600000d3c780 count = 1, bytes = 2
	0x600000d3c080 count = 1, bytes = 138
	0x600000d3c4c0 count = 1, bytes = 5
	0x600000d3c240 count = 1, bytes = 8
memory: high-water mark 0 bytes

Unfortunately it is hard to make it work with iex properly (output does not look pretty)

[kipcole9]

@akash-akya, that's extremely kind of you and I'm sorry to cause more work for you on something that probably only I care about. I will give it a spin. I'm still not sure why VIPS_LEAK=1 ...... doesn't trigger this but it's time to move on.

I was looking at the libvips code and I see

size_t
vips_tracked_get_mem_highwater(void)

size_t
vips_tracked_get_mem(void)

That would be great to have those as NIF calls if it's at all possible because then I can graph memory usage over time in a Livebook. What do you think?

[akash-akya]

Hey @kipcole9, sorry for the delay. I added those two function under dev branch

I'm still not sure why VIPS_LEAK=1 ...... doesn't trigger

I haven't checked yet, but I think that logic only exists in vips binary and not libvips library. I'll check it.

I'm sorry to cause more work for you on something that probably only I care about.

No It is not like that. I am happy that you actively raise issues/discussions, which is needed to improve the library.
Feel free to start discussions :)