-
Notifications
You must be signed in to change notification settings - Fork 253
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Warning should be given first thing in the GitHub page that this library is not interoperable with other LZ4 tooling #222
Comments
Is it true, is there some version that produces output not interoperable with previous versions of other tools too? |
I see that this is among the custom (not interoperable) implementations in https://lz4.org/ homepage
just it's unexpected because they come from the same official GitHub account. |
It’s not about older versions of other tools, it’s all down to the block
format. For our use case we ended up just switching to the Apache Java
library which uses the Frame format instead, now I can use the lz4 cli to
work with the files.
In case of disaster I need access to these files from the cli, I don’t have
time to use any Java script in the middle, if it’s not present on the
server then it’s useless to me.
I do agree with you though, it is very much unexpected this is not
mentioned at all on this project’s readme, it should be made clearer.
…On Sat, 17 Aug 2024 at 00:15, Emanuele Sabellico ***@***.***> wrote:
I see that this is among the custom (not interoperable) implementations in
https://lz4.org/ homepage
Because
They use the block
<https://github.com/lz4/lz4/blob/master/doc/lz4_Block_format.md>
compression format, but add their own frame / header logic (or none at all)
Consequently, they are not interoperable with LZ4 command line utility, nor
(generally) between themselves.
just it's unexpected because they come from the same official GitHub
account.
—
Reply to this email directly, view it on GitHub
<#222 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVLI3DW7TIQWUYUT6NSF73ZR2B2PAVCNFSM6AAAAABLACE54OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJUGQZTGOJUG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
If you only ever intend on using Java to work with the compress files this
project will work for you. If you want to use the cli too then look
elsewhere.
…On Sat, 17 Aug 2024, 00:40 Andrei Vasile, ***@***.***> wrote:
It’s not about older versions of other tools, it’s all down to the block
format. For our use case we ended up just switching to the Apache Java
library which uses the Frame format instead, now I can use the lz4 cli to
work with the files.
In case of disaster I need access to these files from the cli, I don’t
have time to use any Java script in the middle, if it’s not present on the
server then it’s useless to me.
I do agree with you though, it is very much unexpected this is not
mentioned at all on this project’s readme, it should be made clearer.
On Sat, 17 Aug 2024 at 00:15, Emanuele Sabellico ***@***.***>
wrote:
> I see that this is among the custom (not interoperable) implementations
> in https://lz4.org/ homepage
> Because
>
> They use the block
> <https://github.com/lz4/lz4/blob/master/doc/lz4_Block_format.md>
> compression format, but add their own frame / header logic (or none at all)
> Consequently, they are not interoperable with LZ4 command line utility, nor
> (generally) between themselves.
>
> just it's unexpected because they come from the same official GitHub
> account.
>
> —
> Reply to this email directly, view it on GitHub
> <#222 (comment)>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AEVLI3DW7TIQWUYUT6NSF73ZR2B2PAVCNFSM6AAAAABLACE54OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJUGQZTGOJUG4>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
I was checking these comments and seems like the official frame format was implemented in 1.4.0 |
Looking at the documentation, The The other API likely builds on top of the inner Maybe this distinction should be better documented. |
I didn't catch that, that's a very useful find.
Thank you for finding it both. I stand corrected
…On Fri, 23 Aug 2024, 23:42 Yann Collet, ***@***.***> wrote:
Looking at the documentation,
this java implementation seems to offer 2 sets of API,
one is LZ4Compressor/LZ4SafeDecompressor,
the other is LZ4FrameOutputStream/LZ4FrameInputStream.
The LZ4Frame API should be directly compatible with lz4 CLI.
The other API likely builds on top of the inner lz4 block format, and may
or may not add some metadata of its own, making it incompatible with lz4.
Maybe this distinction should be better documented.
—
Reply to this email directly, view it on GitHub
<#222 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEVLI3D4FFQ5FMBNSBOIAHDZS63EBAVCNFSM6AAAAABLACE54OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBXHA4TIMBTGU>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
As the title suggests. A warning is given on the LZ4 wiki page that this library does not conform the LZ4 standard and will not work with standard tooling. This looks to be a common issue (#184, #194 ) with people attempting to decompress files created with this library in other languages (or vice versa) and failing.
A deviation from the standard should be mentioned, especially as this repo is under the lz4 account.
The text was updated successfully, but these errors were encountered: