Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add singingbush changes #2906

Closed
wants to merge 1,042 commits into from
Closed

Conversation

RanbirK
Copy link

@RanbirK RanbirK commented May 13, 2024

What is the purpose of the change

(For example: This pull request improves file read performance by buffering data, fixing AVRO-XXXX.)

Verifying this change

(Please pick one of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Extended interop tests to verify consistent valid schema names between SDKs
  • Added test that validates that Java throws an AvroRuntimeException on invalid binary data
  • Manually verified the change by building the website and checking the new redirect

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

dependabot bot and others added 30 commits May 13, 2024 15:25
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.50 to 1.0.51.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.50...1.0.51)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ssCastException (apache#1527)

* AVRO-3377: Mangle class identifier if required when initializing class

Use same mangling code that existed in the compiler (now moved to
the core library) to mangle type identifer so that the correct class
can be found more often.

* Cleaning up after merge
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.75 to 1.0.76.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.75...1.0.76)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.41 to 2.0.42.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.41...2.0.42)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.70...1.0.71)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) from 2.16.0 to 2.16.1.
- [Commits](FasterXML/jackson-bom@jackson-bom-2.16.0...jackson-bom-2.16.1)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
With an example as we have with the other types as well.
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.76 to 1.0.79.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.76...1.0.79)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.33 to 1.0.35.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](dtolnay/quote@1.0.33...1.0.35)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.51 to 1.0.55.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.51...1.0.55)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.193 to 1.0.194.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.193...v1.0.194)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
…che#2564)

Check if the IDL source file has changed before regenerating the Java
classes to prevent unnecessary recompilation when using maven.

Co-authored-by: Brian Cullen <[email protected]>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.55 to 1.0.56.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.55...1.0.56)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
a
Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
* AVRO-3920: [Rust] * serialize custom attr in record field

* AVRO-3920: [Rust] Do not parse "logicalType" as a custom RecordField attribute

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.110 to 1.0.111.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](serde-rs/json@v1.0.110...v1.0.111)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.74 to 1.0.75.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.74...1.0.75)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.46 to 2.0.48.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.46...2.0.48)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.194 to 1.0.195.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.194...v1.0.195)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serial_test](https://github.com/palfrey/serial_test) from 2.0.0 to 3.0.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](palfrey/serial_test@v2.0.0...v3.0.0)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.75 to 1.0.76.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.75...1.0.76)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* support map and array schema

* AVRO-3927: [Rust] Introduce factory methods for Map & Array schema

The user facing API is shorter than using the From trait

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

---------

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
Co-authored-by: ZENOTME <[email protected]>
Co-authored-by: Martin Tzvetanov Grigorov <[email protected]>
…ypes::Value::Int when possible (apache#2682)

* AVRO-3928: [Rust] Convert serde_json::Value::Number to apache_avro::types::Value::Int when possible

If the number is bigger than i32::MIN and smaller than i32::MAX then
convert to types::Value::Int, otherwise to types::Value::Long

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

* AVRO-3928: [Rust] Add more unit tests for types::Value::from(serde_json::Value)

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>

---------

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
martin-g and others added 10 commits May 13, 2024 15:25
Bumps [num-bigint](https://github.com/rust-num/num-bigint) from 0.4.4 to 0.4.5.
- [Changelog](https://github.com/rust-num/num-bigint/blob/master/RELEASES.md)
- [Commits](rust-num/num-bigint@num-bigint-0.4.4...num-bigint-0.4.5)

---
updated-dependencies:
- dependency-name: num-bigint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.82 to 1.0.83.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](dtolnay/anyhow@1.0.82...1.0.83)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.59 to 1.0.60.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.59...1.0.60)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [paste](https://github.com/dtolnay/paste) from 1.0.14 to 1.0.15.
- [Release notes](https://github.com/dtolnay/paste/releases)
- [Commits](dtolnay/paste@1.0.14...1.0.15)

---
updated-dependencies:
- dependency-name: paste
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.60 to 2.0.61.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.60...2.0.61)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](dtolnay/proc-macro2@1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.116 to 1.0.117.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](serde-rs/json@v1.0.116...v1.0.117)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.200 to 1.0.201.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](serde-rs/serde@v1.0.200...v1.0.201)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ache#2904)

Bumps [org.mockito:mockito-core](https://github.com/mockito/mockito) from 5.11.0 to 5.12.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](mockito/mockito@v5.11.0...v5.12.0)

---
updated-dependencies:
- dependency-name: org.mockito:mockito-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@github-actions github-actions bot added C++ Pull Requests for C++ binding Java Pull Requests for Java binding build Ruby Python C# PHP C JS Perl website Rust labels May 13, 2024
@KalleOlaviNiemitalo
Copy link
Contributor

This pull request contains the same commits as #2310, followed by hundreds of commits rebased from the "main" branch. Specifically, the 521 commits 3bc92f3..fccd6eb in this pull request have been rebased from the 521 commits 50e2db1..f99ed98 on the "main" branch.

$ git ls-remote https://github.com/apache/avro refs/pull/{2310,2906}/{head,merge}
3bc92f3ee7b1894e1b50f57d74779d88d02d7bcc        refs/pull/2310/head
e4914d3429ba87bf1bf04b0072be40424658b1c7        refs/pull/2310/merge
fccd6eb57b0bb1e5c8606d0eada3ceec64da7bad        refs/pull/2906/head
fd612d88c6876c889e70cfbb16d056b65809dd00        refs/pull/2906/merge

At the moment, both pull requests would result in the same tree if merged: https://github.com/apache/avro/compare/refs/pull/2310/merge..refs/pull/2906/merge

$ git rev-parse refs/pull/2310/merge^{tree} refs/pull/2906/merge^{tree}
d08186dcbf28a841f6b6774f0f03077c091dd34c
d08186dcbf28a841f6b6774f0f03077c091dd34c

This pull request doesn't seem to have any advantages over #2310, but the rebased commits make this pull request harder to review.

@martin-g martin-g closed this May 14, 2024
@RanbirK
Copy link
Author

RanbirK commented May 14, 2024

This pull request contains the same commits as #2310, followed by hundreds of commits rebased from the "main" branch. Specifically, the 521 commits 3bc92f3..fccd6eb in this pull request have been rebased from the 521 commits 50e2db1..f99ed98 on the "main" branch.

$ git ls-remote https://github.com/apache/avro refs/pull/{2310,2906}/{head,merge}
3bc92f3ee7b1894e1b50f57d74779d88d02d7bcc        refs/pull/2310/head
e4914d3429ba87bf1bf04b0072be40424658b1c7        refs/pull/2310/merge
fccd6eb57b0bb1e5c8606d0eada3ceec64da7bad        refs/pull/2906/head
fd612d88c6876c889e70cfbb16d056b65809dd00        refs/pull/2906/merge

At the moment, both pull requests would result in the same tree if merged: https://github.com/apache/avro/compare/refs/pull/2310/merge..refs/pull/2906/merge

$ git rev-parse refs/pull/2310/merge^{tree} refs/pull/2906/merge^{tree}
d08186dcbf28a841f6b6774f0f03077c091dd34c
d08186dcbf28a841f6b6774f0f03077c091dd34c

This pull request doesn't seem to have any advantages over #2310, but the rebased commits make this pull request harder to review.

Given that https://github.com/davidmc24/gradle-avro-plugin was archived in Oct 2023 and the other PR is open for almost a year and not much progress was happening on that, so we wanted to take it up to resolve the issues to expedite the availability of this avro gradle plugin. Please let me know if we need to approach volunteering for this in some other way.

@martin-g
Copy link
Member

Here are hints:

  1. You need to explain your ideas better next time by using properly the title and the description fields of the PR!
    By ignoring them you ask to be ignored.

  2. You might ask someone to help you with Git!
    A fresh new PR is expected to have a few commits, not hundreds.

@KalleOlaviNiemitalo
Copy link
Contributor

As I understand it, the problem with #2310 is that AVRO-3733 "Migrate package names and artifacts to org.apache.avro namespace" and AVRO-3734 "Integrate build with Apache Avro github actions / local build" have not been done yet. So I would suggest working on those on top of the commits that are in #2310.

It might make sense to merge the current changes from #2310 to a new branch in this repository. That way, a separate pull request for the AVRO-3733 changes could then target that branch and be reviewed here, without looking at the code-drop commits again. Then another PR for AVRO-3734, and finally merge the branch to main. OTOH, that would give multiple opportunities to mistakenly use a squash merge and lose the history.

@RanbirK
Copy link
Author

RanbirK commented May 17, 2024

As I understand it, the problem with #2310 is that AVRO-3733 "Migrate package names and artifacts to org.apache.avro namespace" and AVRO-3734 "Integrate build with Apache Avro github actions / local build" have not been done yet. So I would suggest working on those on top of the commits that are in #2310.

It might make sense to merge the current changes from #2310 to a new branch in this repository. That way, a separate pull request for the AVRO-3733 changes could then target that branch and be reviewed here, without looking at the code-drop commits again. Then another PR for AVRO-3734, and finally merge the branch to main. OTOH, that would give multiple opportunities to mistakenly use a squash merge and lose the history.

I am unable to push commits to https://github.com/SingingBush/avro/tree/migration/import_gradle_plugin and that is why I thought to create a latest fork from https://github.com/apache/avro and the pull in the changes from SingingBush/avro/tree/migration/import_gradle_plugin to fix the pipeline issues. I am new to contributing in this space to apologies if I am missing any process.

@martin-g
Copy link
Member

IMO any new work on this/these task(s) should be done against a temporary branch.
We could not merge to main branch PRs which do not pass the CI checks.
#2310 introduces 130 new CI checks - this is almost 10 times more than what we have now for all SDKs! This setup should be re-considered !
Once this temporary branch is stable enough we could merge it to main.

@martin-g
Copy link
Member

See #2310 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build C++ Pull Requests for C++ binding C# C Java Pull Requests for Java binding JS Perl PHP Python Ruby Rust website
Projects
None yet
Development

Successfully merging this pull request may close these issues.