ENG-2572: VectorClockIds are a tuple. Editing changesets are gone #3988

Previously, vector clock ids were the same as change set ids. And, we generated a "editing change set" anytime we mutated the graph. This changeset was ephemeral, and not connected to a real "change set" in the system. In addition, for conflict detection to work correctly, the node write clocks have to store every vector clock write that has *ever* happened to them. This meant these clocks would grow indefinitely, since they have to store every ephemeral "editing change set" in the node, forever. This change transforms the vector clock id into a tuple of the real ChangeSetId and the UserPk/ActorId of the current user. In the context of system actors, like Pinga and the Rebaser, the WorkspacePk is used in place of the UserPk and removes editing change sets entirely. Now the bound on the vector clock write clocks in node weights is the number of change sets and users in the system, which will grow much more slowly than the editing change sets. This is a breaking change, since it changes both Node and Edge weight data strucutres. Migration must be in place before this can be deployed.

Whenever we don't have HistoryActor, generate an actor id that lasts for the current DalContext and use that for the vector clock id's actor id. But, when the rebaser writes out the final snapshot, use the workspace pk for the actor id. Co-Authored-By: Jacob Helwig <[email protected]>

Co-Authored-By: Jacob Helwig <[email protected]>

On SDF boot, attempt to automatically migrate all snapshots for a deployment, beginning with the builtin workspace's snapshot. Follows the "based_on_change_set_id" paths, treating the snapshots as a dependency graph, so that shared clock ids are migrated to the new clock ids correctly. Once this code is deployed, SDF will panic if it encounters a 'legacy' snapshot. Co-Authored-By: Jacob Helwig <[email protected]>

If a table's structure changes, cached query plans against that table need to be invalidated, or postgresql will return an error. This change prevents that error after migrating the database in a production system running pb_bouncer, which holds on to connections and reuses them even if our services are restarted. We could avoid needing to discard plans by selecting exactly the columns we need instead of SELECT * (unless the column type changes!) This issue never hit us before because we haven't changed table structures much since adding pg_bouncer to the stack. Co-Authored-By: Jacob Helwig <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENG-2572: VectorClockIds are a tuple. Editing changesets are gone #3988

ENG-2572: VectorClockIds are a tuple. Editing changesets are gone #3988

Commits on Jul 11, 2024