upgrade executor to non-duplicating incremental delivery format

[yaacovCR]

Description

GraphQL Incremental Delivery is moving to a new response format without duplication.

• The corresponding changes are being integrated into graphql-js with the test changes live on main and within the latest (but old v17 alpha).
• The spec changes are in process, for example, my latest effort (on defer only) is at Alternative spec implementation of deduplicated incremental delivery graphql/graphql-spec#1077. The implementation changes on main are meant to match those spec changes.

This PR updates the executor within graphql-tools to follow the new format, with my best effort to integrate the changes made within the graphql-tools executor, largely around abortion.

Notes:

• This version of incremental delivery has a new response format. The loader package has been updates to work with the new response format (mergeIncrementalResults()), and Yoga tests that rely on it pass, but there should probably eventually be new unit tests new unit tests have been added within the existing mergeIncrementalResults() tests. The function still work with the old response format.
• The new response format itself is a BREAKING CHANGE, clients expecting an executor that emits the old format will no longer have path references where they expect, instead they will get ids and be required to lookup the path based on the information from pending.
• This version is a straight-copy from graphql-js where a decision has been made to disable early execution by default. Some believe that despite theoretical causes for concern, a better default would be to enable early execution. This version enables early execution by default, with a flag to disable, differing from graphql-js where early execution will probably be disabled by default, at least initially.
• The abortion logic for stream has been modified to be somewhat similar and to avoid Promise.race, although this may mean that abortion might get stuck while waiting for the next payload if the underlying iterator does not properly handle a return(). The rationale for avoiding Promise.race is the memory leak that may occur with long-running promises. This change should probably be discussed further before merging. See the comment below for how this is accomplished.
• My work on Incremental Delivery during 2023 was sponsored by the Guild, presented at GraphQL Conf 2023. The main blocker to release has been the spec changes, where the best prose has been elusive.
• It would be amazing for the community to get the new format more widely tested. That's what this PR is about.
• Hopefully, this will get us closer to incremental delivery being released. Until, then, we have Fitzgerald: "So we beat on, boats against the current, borne back ceaselessly into the past."

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as
  expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can
reproduce. Please also list any relevant details for your test configuration

• graphql-js tests have been transferred.
• Existing graphql-tools tests have been updated.

Test Environment:

• OS:
• @graphql-tools/...: latest
• NodeJS: 22

Checklist:

• I have followed the
  CONTRIBUTING doc and the
  style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests and linter rules pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[changeset-bot]

🦋 Changeset detected

Latest commit: 006da37

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@graphql-tools/utils	Minor
@graphql-tools/executor	Major
@graphql-tools/delegate	Patch
@graphql-tools/stitch	Patch
federation-benchmark	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

✅ Benchmark Results

     ✓ no_errors
     ✓ expected_result

     checks.........................: 100.00% ✓ 334       ✗ 0  
     data_received..................: 39 MB   3.9 MB/s
     data_sent......................: 143 kB  14 kB/s
     http_req_blocked...............: avg=3.88µs   min=2.09µs  med=2.71µs   max=155.7µs  p(90)=3.83µs  p(95)=4.28µs  
     http_req_connecting............: avg=606ns    min=0s      med=0s       max=101.32µs p(90)=0s      p(95)=0s      
     http_req_duration..............: avg=55.58ms  min=47.74ms med=51.43ms  max=154.1ms  p(90)=60.81ms p(95)=89.38ms 
       { expected_response:true }...: avg=55.58ms  min=47.74ms med=51.43ms  max=154.1ms  p(90)=60.81ms p(95)=89.38ms 
     http_req_failed................: 0.00%   ✓ 0         ✗ 167
     http_req_receiving.............: avg=131.43µs min=103.4µs med=126.15µs max=431.28µs p(90)=149.2µs p(95)=155.06µs
     http_req_sending...............: avg=25.04µs  min=19.37µs med=25.08µs  max=58.94µs  p(90)=28.8µs  p(95)=34.64µs 
     http_req_tls_handshaking.......: avg=0s       min=0s      med=0s       max=0s       p(90)=0s      p(95)=0s      
     http_req_waiting...............: avg=55.42ms  min=47.56ms med=51.25ms  max=153.61ms p(90)=60.63ms p(95)=89.23ms 
     http_reqs......................: 167     16.660336/s
     iteration_duration.............: avg=59.99ms  min=51.55ms med=55.86ms  max=158.67ms p(90)=67.9ms  p(95)=93.4ms  
     iterations.....................: 167     16.660336/s
     vus............................: 1       min=1       max=1
     vus_max........................: 1       min=1       max=1

[github-actions]

🚀 Snapshot Release (alpha)

The latest changes of this PR are available as alpha on npm (based on the declared changesets):

Package	Version	Info
@graphql-tools/delegate	10.0.14-alpha-20240709213044-006da37e240c6ccb1d305e19ade18dd70384cb22	npm ↗︎ unpkg ↗︎
@graphql-tools/executor	2.0.0-alpha-20240709213044-006da37e240c6ccb1d305e19ade18dd70384cb22	npm ↗︎ unpkg ↗︎
@graphql-tools/stitch	9.2.11-alpha-20240709213044-006da37e240c6ccb1d305e19ade18dd70384cb22	npm ↗︎ unpkg ↗︎
@graphql-tools/utils	10.4.0-alpha-20240709213044-006da37e240c6ccb1d305e19ade18dd70384cb22	npm ↗︎ unpkg ↗︎

[github-actions]

💻 Website Preview

The latest changes are available as preview in: https://e83da3f3.graphql-tools.pages.dev

[yaacovCR]

Some additional issues:

• The existing changeset in this PR was added automatically, I guess. The appropriate changeset would be BREAKING for the executor, and I guess for the loader? And a MINOR for utils as processing the new format is I guess a new feature. (EDIT: this has been done!)
• The executor package gets its own new version of collectFields. Perhaps that could be integrated into the existing collectFields that is in utils, but presumably after incremental delivery is finalized.

[EmrysMyrddin]

For changeset, you can run yarn changeset to add new changesets with the desired level and message :-)

[yaacovCR]

We can use the approach from graphql/graphql-js#4043 which improves code readability and includes a significant performance benefit. At graphql-js, we have not yet adopted this approach, because we will be following the spec more closely, at least during the initial adoption period.

[yaacovCR]

I forgot to mention that another BREAKING CHANGE is that incremental delivery no longer is supported within subscriptions.

In theory, we can remote flattenAsyncIterable() but the error handling in subscriptions becomes a bit different than in current tests if it is simply removed, which I have not gotten to the bottom of which I have now done.

[yaacovCR]

I've updated this PR to re-enable early execution by default.

[yaacovCR]

Is it possible that we would want this executor to emit the old duplicated format by default and enable the new format under a flag?

[yaacovCR]

I believe I've fixed the issue with aborting from incremental delivery despite pending long-running promises, still avoiding the use of Promise.race().

The key was to realize that the IncrementalPublisher awaits the next result from IncrementalGraph which uses a hand-crafted AsyncGenerator rather than a "true" AsyncGenerator. A hand-crafted AsyncGenerator can -- and in this case does -- cause the Promises returned by .next() methods to return early if .return() is called in parallel. This allows a way to abort the await early without using Promise.race().

I've added a test demonstrating this.

[yaacovCR]

I've added the new unit tests for mergeIncrementalResults.

[yaacovCR]

Is it possible that we would want this executor to emit the old duplicated format by default and enable the new format under a flag?

Thinking about this more, this is definitely possible in terms of functionality, but there would be a breaking change in terms of the types -- or we would have to introduce a new set of types for the new format with maybe a prefix or suffix.

I am not unsure how to proceed, definitely easier just to make it a BREAKING CHANGE, so at this point I will leave this PR as is => From my perspective, this is now ready for review! 🚀

[n1ru4l]

@yaacovCR Now catching up with your work and rewatching your video from GraphQL Conf was very helpful! 🙏

I am fine with shipping this as a breaking change. But, we should have a middleware/wrapper function that takes the execute function and emits the old format. That way we can at least provide people with a migration path within Yoga that can not upgrade all their client's code immediately!

[yaacovCR]

Sounds good, I can work on that.

[yaacovCR]

I added an argument to toggle whether incremental is disabled for subscriptions so that should cover that breaking change. I'm trying to decide whether to add an argument for deduplication or whether to add a wrapper, I'm thinking of going with the former.

As an aside, I think tests are broken separate from this PR.

[ardatan]

I left a few comments. And some of them would probably fix the type checking issues.

[yaacovCR]

I ran yarn upgrade and I got my local tests passing, too, that's what threw me off :) oneOf got backported to the executor, but I hadn't upgraded graphql locally :(

[yaacovCR]

This is ready for re-review. You can now get the old format by setting a few execution arguments, to disable duplication, send path/label within incremental entries, and send completion errors as null incremental entries.

[yaacovCR]

@ardatan @n1ru4l in the above checks — can I check to see the performance change with this PR? Do you know which spots to best look at?