Explicit Split of Replace Op | Null Json Node handling #120

isopropylcyanide · 2020-06-21T13:32:10Z

Summary

PR Status	Type	Impact level
Ready	Feature	Medium

Description

This PR does two things

Adds a new flag ADD_EXPLICIT_REMOVE_ADD_ON_REPLACE which splits a replace operation into a remove followed by an add operation
Updates the contract for asJson to handle null values for the input source / target nodes which currently blows up in an ugly NPE.
Bumps up version to 0.4.11

Motivation & Context

New Flag

New flag is required because of use cases in audit systems where the semantic of a remove and add is not intuitively similar to a replace operation.

For e,g consider a system where users can add and remove their favourite brands. If in a single API operation, removal of existing brand B1 and addition of a new brand B2 is allowed for user ease, the patch contains one replace event.

"op": "replace", "fromValue": "F1", "value": "F2"

This, however, needs to be processed into two events

"op": "remove",  "value": "F2"

"op": "add",  "value": "F2"

It's not exactly clear how a new addition of a brand can replace the deletion of a previous brand. Both are separate events which need to be captured accordingly. Hence we should provide a flag which lets client make this decision. It is off by default.

Updating contract
The asJson doesn't handle null inputs well. Following conditions can be true

source is null
target is null
both are null
both are non null, which works

Expectation:

If source is null and target is not, a patch needs to be provided with all add events by considering source to be an empty node
If target is null and source is not, a patch needs to be provided with all remove events by considering source to be an empty node
If both are null, the patch should be empty

Reality:
Currently it blows up with an NPE in all 3 cases, even when both are null

To see why the inputs can be null, consider the use case of a change data capture system like Debezium or Maxwell that streams database changes in json. If a row is newly added or deleted, the source and target are null respectively.

Imagine a client trying to compute the diff between before and after. We shouldn't expect them to create an empty node before calling the differ. Hence, the contract has been updated to automatically handle null as there was no formal specification for the same earlier.

How was this tested

100% coverage on new code
All units run fine.

holograph

A few small changes, especially wrt asJson and whether or not asJsonNullSafe is an appropriate choice; I think this is worth consideration.

Also, and this isn't why I'm requesting changes on the PR, I'm still trying to wrap my head around the use case; JSON documents are bare data and do not carry any semantic information, I simply can't figure out why you'd want the diff algorithm to do this instead of reasoning about the content in external code. Further, since this is strictly an optional postprocessing step (that relies on a non-RFC extension already present in the codebase), I'm of the opinion that a special use case should remain outside the library, but @vishwakarma might have a different view...

src/main/java/com/flipkart/zjsonpatch/DiffFlags.java

src/main/java/com/flipkart/zjsonpatch/JsonDiff.java

holograph · 2020-06-21T13:54:32Z

src/main/java/com/flipkart/zjsonpatch/JsonDiff.java

+
+    public static JsonNode asJsonNullSafe(final JsonNode source, final JsonNode target, EnumSet<DiffFlags> flags) {
+        if (source == null && target == null) {
+            return asJson(MAPPER.createObjectNode(), MAPPER.createObjectNode(), flags);


This seems wasteful, might as well return null (since the contract is extended anyway) or an empty array node.

Agreed. Good point. Will optimize for add/remove cases as well

src/main/java/com/flipkart/zjsonpatch/JsonDiff.java

isopropylcyanide · 2020-06-22T00:53:27Z

src/main/java/com/flipkart/zjsonpatch/JsonDiff.java

+            // return add node at root pointing to the target
+            diff.diffs.add(Diff.generateDiff(Operation.ADD, JsonPointer.ROOT, target));
+        }
+        if (source != null && target == null) {


Not using else ifs for readability.

isopropylcyanide · 2020-06-22T00:54:35Z

src/test/java/com/flipkart/zjsonpatch/JsonDiffTest.java

@@ -127,44 +127,33 @@ public void testPath() throws Exception {
        Assert.assertEquals(target, expected);
    }

-


Tests have been updated. Removed the asJsonNullSafe method. Since the existing asJson is enriched with the new contract, it's tests have too.

holograph

Code looks fine, now we're just down to "does this make sense in the library". @vishwakarma care to weigh in?

vishwakarma · 2020-06-22T08:22:22Z

Thanks @isopropylcyanide for raising the diff.
Thanks @holograph sir for reviewing it, really appreciate it.

My opinion on this diff -
As we are not changing the default behaviour of this library which is still compliant with the RFC to produce all possible set of operations to minimize the patch size thus its fine with me.
Further, Historically we have started supporting few extensions of this library via DiffFlags.java thus i think we should allow this one more variant on allowed set of operations.

isopropylcyanide · 2020-06-23T09:07:03Z

In that case, are we good to merge this? @holograph @vishwakarma

isopropylcyanide added 6 commits June 21, 2020 01:14

Comment release plugin

1db5973

Support: Replace instruction into add remove and add

29adb8d

Add tests and ensure full coverage

d74fd51

Introduce null safety for null target/src nodes

2292f0b

Self review: Undo basic formatting

9de28d4

Add missing test: No replace split in absence of flag

ddc4b9e

holograph requested changes Jun 21, 2020

View reviewed changes

isopropylcyanide added 2 commits June 22, 2020 05:33

RC: Update JavaDoc

246fec5

RC: Optimize early returns and enrich existing contract for asJson

b3744c0

isopropylcyanide commented Jun 22, 2020

View reviewed changes

holograph approved these changes Jun 22, 2020

View reviewed changes

vishwakarma merged commit 88793f4 into flipkart-incubator:master Jun 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Explicit Split of Replace Op | Null Json Node handling #120

Explicit Split of Replace Op | Null Json Node handling #120

isopropylcyanide commented Jun 21, 2020 •

edited

Loading

holograph left a comment

holograph Jun 21, 2020

isopropylcyanide Jun 22, 2020

isopropylcyanide Jun 22, 2020

isopropylcyanide Jun 22, 2020 •

edited

Loading

holograph left a comment

vishwakarma commented Jun 22, 2020

isopropylcyanide commented Jun 23, 2020

		@@ -127,44 +127,33 @@ public void testPath() throws Exception {
		Assert.assertEquals(target, expected);
		}

Explicit Split of Replace Op | Null Json Node handling #120

Explicit Split of Replace Op | Null Json Node handling #120

Conversation

isopropylcyanide commented Jun 21, 2020 • edited Loading

Summary

Description

Motivation & Context

How was this tested

holograph left a comment

Choose a reason for hiding this comment

holograph Jun 21, 2020

Choose a reason for hiding this comment

isopropylcyanide Jun 22, 2020

Choose a reason for hiding this comment

isopropylcyanide Jun 22, 2020

Choose a reason for hiding this comment

isopropylcyanide Jun 22, 2020 • edited Loading

Choose a reason for hiding this comment

holograph left a comment

Choose a reason for hiding this comment

vishwakarma commented Jun 22, 2020

isopropylcyanide commented Jun 23, 2020

isopropylcyanide commented Jun 21, 2020 •

edited

Loading

isopropylcyanide Jun 22, 2020 •

edited

Loading