Upgrade parquet dependencies to 1.14.1 #1107
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1107 +/- ##
============================================
- Coverage 52.02% 52.00% -0.02%
+ Complexity 655 654 -1
============================================
Files 89 89
Lines 5388 5388
Branches 708 708
============================================
- Hits 2803 2802 -1
Misses 2323 2323
- Partials 262 263 +1 ☔ View full report in Codecov by Sentry. |
chandrashekar-s
force-pushed
the
upgrade-parquet
branch
from
59dd2c8
to
1b7e691
Compare
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.parquet</groupId> |
There was a problem hiding this comment.
These dependencies are transitive dependencies of beam-sdks-java-io-parquet and were resolving to a different version than the parquet-avro. So, locking all the dependencies to the same version at the root package.
There was a problem hiding this comment.
I think this is okay but one problem that I see is that Beam's code may start throwing exceptions if we use a difference Parquet lib version. Although this issue is not unique here, it might be worth trying to fix avoid it. One option is to remove these new deps and only change parquet.version when we change Beam's version and keep them consistent; WDTY?
There was a problem hiding this comment.
Thanks @chandrashekar-s for looking into version upgrade issues; please see my comment below:
| </dependency> | ||
|
|
||
| <dependency> | ||
| <groupId>org.apache.parquet</groupId> |
There was a problem hiding this comment.
I think this is okay but one problem that I see is that Beam's code may start throwing exceptions if we use a difference Parquet lib version. Although this issue is not unique here, it might be worth trying to fix avoid it. One option is to remove these new deps and only change parquet.version when we change Beam's version and keep them consistent; WDTY?
Description of what I changed
This PR upgrades
parquet-avroand its transitive dependencies to the same version, which fixes the error introduced by this PR.E2E test
Relied on automated e2e tests.
TESTED:
NA
Checklist: I completed these to help reviewers :)
I have read and will follow the review process.
I am familiar with Google Style Guides for the language I have coded in.
No? Please take some time and review Java and Python style guides.
My IDE is configured to follow the Google code styles.
No? Unsure? -> configure your IDE.
I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)
I ran
mvn clean packageright before creating this pull request and added all formatting changes to my commit.All new and existing tests passed.
My pull request is based on the latest changes of the master branch.
No? Unsure? -> execute command
git pull --rebase upstream master