AVRO-3939: [Rust] Make it possible to use custom schema equality comparators #2739
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WIP Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Implement compare_fields() Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Add unit tests for the primitives Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g
changed the title
StructFieldEq Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
|
@ZENOTME The PR is ready for review and test! |
Fix clippy error with Rust nightly, e.g.: ``` error: the item `TryFrom` is imported redundantly --> avro/src/types.rs:34:5 | 34 | convert::TryFrom, | ^^^^^^^^^^^^^^^^ --> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13 | = note: the item `TryFrom` is already defined here error: could not compile `apache-avro` (lib) due to 9 previous errors ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
fix more Rust nightly clippy errors in tests Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g
force-pushed
the
avro-3939-custom-schemata-comparators
branch
from
ae14966
to
8d40852
Compare
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
```
error: the item `RecordField` is imported redundantly
--> avro/src/types.rs:1153:18
|
1150 | use super::*;
| -------- the item `RecordField` is already imported here
...
1153 | schema::{RecordField, RecordFieldOrder},
| ^^^^^^^^^^^
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
```
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
…ogged messages Not just against the last logged message Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g
force-pushed
the
avro-3939-custom-schemata-comparators
branch
from
fd8689d
to
a1ab3e1
Compare
```
error: the item `FromIterator` is imported redundantly
--> avro/benches/serde_json.rs:20:33
|
20 | use std::{collections::HashMap, iter::FromIterator};
| ^^^^^^^^^^^^^^^^^^
--> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13
|
= note: the item `FromIterator` is already defined here
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
error: could not compile `apache-avro` (bench "serde_json") due to 1 previous error
warning: build failed, waiting for other jobs to finish...
Error: Process completed with exit code 101.
```
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
It is much faster than SpecificationEq which serializes the schemas to JSON and then compares them as strings Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g
added a commit
that referenced
this pull request
Feb 20, 2024
…arators (#2739) * AVRO-3939: [Rust] Make it possible to use custom schema comparators Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Temporarily use StructFieldComparator as default Implement compare_fields() Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Fix formatting and clippy issues Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Rename the trait and its impls from Comparator to Eq Add unit tests for the primitives Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Add support for comparing the custom attributes to StructFieldEq Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Add more unit tests Fix clippy error with Rust nightly, e.g.: ``` error: the item `TryFrom` is imported redundantly --> avro/src/types.rs:34:5 | 34 | convert::TryFrom, | ^^^^^^^^^^^^^^^^ --> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13 | = note: the item `TryFrom` is already defined here error: could not compile `apache-avro` (lib) due to 9 previous errors ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRo-3939: Add more unit tests fix more Rust nightly clippy errors in tests Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: More unit tests and better logging Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Fix rust nightly clippy error ``` error: the item `RecordField` is imported redundantly --> avro/src/types.rs:1153:18 | 1150 | use super::*; | -------- the item `RecordField` is already imported here ... 1153 | schema::{RecordField, RecordFieldOrder}, | ^^^^^^^^^^^ | = note: `-D unused-imports` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(unused_imports)]` ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Improve TestLogger's assert_logged to assert against all logged messages Not just against the last logged message Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * Fix rust nightly clippy error ``` error: the item `FromIterator` is imported redundantly --> avro/benches/serde_json.rs:20:33 | 20 | use std::{collections::HashMap, iter::FromIterator}; | ^^^^^^^^^^^^^^^^^^ --> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13 | = note: the item `FromIterator` is already defined here | = note: `-D unused-imports` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(unused_imports)]` error: could not compile `apache-avro` (bench "serde_json") due to 1 previous error warning: build failed, waiting for other jobs to finish... Error: Process completed with exit code 101. ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: More test assertions and better logging Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Use StructFieldEq impl by default It is much faster than SpecificationEq which serializes the schemas to JSON and then compares them as strings Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVro-3939: Format the code Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> --------- Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> (cherry picked from commit e038004)
|
Sorry for being late. Thanks for this nice job! @martin-g |
RanbirK
pushed a commit
to RanbirK/avro
that referenced
this pull request
May 13, 2024
…arators (apache#2739) * AVRO-3939: [Rust] Make it possible to use custom schema comparators Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Temporarily use StructFieldComparator as default Implement compare_fields() Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Fix formatting and clippy issues Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Rename the trait and its impls from Comparator to Eq Add unit tests for the primitives Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Add support for comparing the custom attributes to StructFieldEq Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Add more unit tests Fix clippy error with Rust nightly, e.g.: ``` error: the item `TryFrom` is imported redundantly --> avro/src/types.rs:34:5 | 34 | convert::TryFrom, | ^^^^^^^^^^^^^^^^ --> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13 | = note: the item `TryFrom` is already defined here error: could not compile `apache-avro` (lib) due to 9 previous errors ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRo-3939: Add more unit tests fix more Rust nightly clippy errors in tests Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: More unit tests and better logging Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Fix rust nightly clippy error ``` error: the item `RecordField` is imported redundantly --> avro/src/types.rs:1153:18 | 1150 | use super::*; | -------- the item `RecordField` is already imported here ... 1153 | schema::{RecordField, RecordFieldOrder}, | ^^^^^^^^^^^ | = note: `-D unused-imports` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(unused_imports)]` ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Improve TestLogger's assert_logged to assert against all logged messages Not just against the last logged message Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * Fix rust nightly clippy error ``` error: the item `FromIterator` is imported redundantly --> avro/benches/serde_json.rs:20:33 | 20 | use std::{collections::HashMap, iter::FromIterator}; | ^^^^^^^^^^^^^^^^^^ --> /rustc/2bf78d12d33ae02d10010309a0d85dd04e7cff72/library/std/src/prelude/mod.rs:129:13 | = note: the item `FromIterator` is already defined here | = note: `-D unused-imports` implied by `-D warnings` = help: to override `-D warnings` add `#[allow(unused_imports)]` error: could not compile `apache-avro` (bench "serde_json") due to 1 previous error warning: build failed, waiting for other jobs to finish... Error: Process completed with exit code 101. ``` Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: More test assertions and better logging Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVRO-3939: Use StructFieldEq impl by default It is much faster than SpecificationEq which serializes the schemas to JSON and then compares them as strings Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> * AVro-3939: Format the code Signed-off-by: Martin Tzvetanov Grigorov <[email protected]> --------- Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
AVRO-3939
What is the purpose of the change
Verifying this change
Documentation