Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GIF palette optimization #104

Merged
merged 1 commit into from
Sep 21, 2023

Conversation

rkfg
Copy link
Contributor

@rkfg rkfg commented Sep 20, 2023

The default GIF saving algorithm lacks dithering and palette optimization which results in noticeable banding artifacts. In this PR I add a new option (turned off by default) that enables these features. The new imageio v3 API is used for that because it supports pyav plugin which creates the resulting GIF with ffmpeg and a filter graph that precalculates the palette from all frames in the sequence. If the av library is missing it installs it automatically. As a result, the GIF looks much better but the file might be bigger (not always), and it also requires a dependency to install.

There's a weird quirk that I couldn't explain. If the filter graph is used, the image is always resized to 640x480 no matter what its source size is. It's probably a bug somewhere in the pyav plugin or the av library because ffmpeg with the same graph doesn't do that. So to mitigate it I added a scale filter to the graph in the beginning.

Comparison

Palette Size Image
No optimization 3.8 Mb 02078-3855594682
Optimized 4.5 Mb 02077-3855594682_opt

Banding is visible on the ground in the first GIF, it's even worse in dynamic.

Note that these images are not perfectly in sync. Turns out the Pillow GIF creator is a bit less smart than ffmpeg. The default 8 FPS setting means that every frame should be shown for 125 ms (1000/8), however, the GIF spec says that the frame time is specified in hundredths of a second which would be 12.5. Since 0.5 can't be represented, Pillow uses just 12 which gives us 8.(3) FPS instead of 8.

Ffmpeg does an interesting trick, it alternates the frame delays between 130 and 120 ms which, on average, gives us 125 ms and 8 FPS. So the optimized version is technically more correct in that regard too.

@zappityzap
Copy link
Collaborator

Worked for me on Ubuntu 22.04.

I like the improvement in quality for the size. Gifsicle seems to make the smallest file, and claims to be more compatible, so it might be worth keeping. There may be better options for gifsicle instead of just --optimize.

@rkfg
Copy link
Contributor Author

rkfg commented Sep 21, 2023

Gifsicle should work with this optimization too, it simply launches that program after making the gif using whatever method.

@continue-revolution
Copy link
Owner

This is great. Tomorrow I will move the original optimization setting to Settings/AnimateDiff like this one and merge.

@ahvonenj
Copy link

Been using this optimization for around a day now. Working beautifully!

@rkfg
Copy link
Contributor Author

rkfg commented Sep 21, 2023

Glad it works fine. I think GIF isn't going away any time soon so we have to make it look as good as possible. Webp animation support is atrocious, I couldn't find any viewers in Linux (except Chrome-based browsers) that can play it. It's not supported on iOS at all. So even though webp is technically superior it's practically so inferior I don't think it's worth supporting until the situation improves. Playing mp4 in the web UI would be nice though.

@continue-revolution continue-revolution changed the base branch from master to dev September 21, 2023 20:59
@continue-revolution continue-revolution merged commit 8bd07e3 into continue-revolution:dev Sep 21, 2023
@continue-revolution continue-revolution mentioned this pull request Sep 21, 2023
5 tasks
continue-revolution added a commit that referenced this pull request Sep 21, 2023
* convert params dict to AnimateDiffProcess for API (#110)

* Add GIF palette optimization (#104)

* move gifsicle to settings

* re-format with isort

* xformers

* fix radio

* readme

---------

Co-authored-by: zappityzap <[email protected]>
Co-authored-by: rkfg <[email protected]>
@rkfg rkfg deleted the gifpal branch September 22, 2023 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants