Add unit tests for the layout engine #424

Adlai-Holler · 2017-07-07T00:56:43Z

This adds a testing platform and some unit tests to the layout engine. Right now there are flags to toggle between the current behavior and the desired behavior, and they're set to "current behavior."

Issues reproduced (need to merge with GitHub issues):

Extraneous calculatedLayoutDidChange calls. Not a biggie.
If a layout pass comes in mid-transition, we will calculateLayout: for some nodes against their current bounds on main, but discard the result. https://github.com/TextureGroup/Texture/pull/424/files#diff-b1441b7e61c34b6f132da6c94bb10c0cR125
If we do setNeedsLayout and get a layout pass while a transition is measuring, the transition's "to-layout" will win, but the setNeedsLayout+layoutPass should actually win since its data is newer. https://github.com/TextureGroup/Texture/pull/424/files#diff-b1441b7e61c34b6f132da6c94bb10c0cR204
If we setNeedsLayout on a leaf node that is subject to multipass measurement (e.g. flexShrink) we will not update the layout for the leaf or the stack. We will remeasure it only against the fixed (second-pass) constrained size, and inevitably get a result confirming that the current size is correct, but we should have offered the node another unconstrained measurement pass. In my estimation this is the most serious issue. https://github.com/TextureGroup/Texture/pull/424/files#diff-b1441b7e61c34b6f132da6c94bb10c0cR284

I'd like to reproduce the orphan-node issue but I'm not sure what conditions would trigger it @nguyenhuy. We've got enough flexibility in the testing platform – I believe – to test an incredible variety of different scenarios.

PROTIP: You can take any expectation and add .andDebugBreak() to get a breakpoint when that call happens. So if you expect an async layout: OCMExpect([nodeA.mock calculateLayoutThatFits:]).offMainThread() you can change that to OCMExpect([nodeA.mock calculateLayoutThatFits:]).offMainThread().andDebugBreak() and when that calculateLayoutThatFits invocation happens, the debugger will pause for you. ✨Magic✨

nguyenhuy

This is incredibly cool! We need more tests like this before fixing any existing problems in the layout system. Thank you for taking the first stab!

Regarding the orphan-node issue, I think you can reproduce by first start a layout transition with animation, then while the animation is running, trigger another layout pass on the root node via -setNeedsLayout. Depending on the differences between the original layout and the first transition's layout, expect both orphaned subnode(s) as well as subnode(s) that should be inserted but never was. More information can be found in facebookarchive/AsyncDisplayKit#2862.

Following the same steps above, but instead of triggering the second/last layout pass via -setNeedsLayout, trigger another async layout transition. Even if the second transition is queued up correctly and starts applying its result on main after the animation of the first transition finished, what will happen is that the second transition will run the ASM diff-ing algorithm against the original layout, instead of the first transition's layout. That is because layout transition contexts capture the from/original layout at initialization time, as opposed to at commit time. And that can also cause orphaned or never-inserted subnodes.

nguyenhuy · 2017-07-07T15:21:11Z

Tests/ASLayoutEngineTests.mm

+  fixture.layout = layoutA;
+
+  ASLayoutSpecBlock specBlockA = ^ASLayoutSpec * _Nonnull(__kindof ASDisplayNode * _Nonnull node, ASSizeRange constrainedSize) {
+    return [ASStackLayoutSpec stackLayoutSpecWithDirection:ASStackLayoutDirectionHorizontal spacing:0 justifyContent:ASStackLayoutJustifyContentSpaceBetween alignItems:ASStackLayoutAlignItemsStart children:@[ nodeB, nodeC, nodeE ]];


Shouldn't the children contains nodes B, D, E instead of B, C, E?

nguyenhuy · 2017-07-07T15:22:15Z

Tests/ASLayoutEngineTests.mm

+ * From fixture 2:
+ *   B survives with same layout
+ *   C is removed
+ *   D is removed


Should be "D joins the first layout" since it's a new subnode.

nguyenhuy · 2017-07-07T15:25:52Z

Tests/ASLayoutEngineTests.mm

+  // The code in this section is designed to move in time order, all on the main thread:
+
+  OCMExpect([nodeA.mock animateLayoutTransition:OCMOCK_ANY]).onMainThread();
+  OCMExpect([nodeA.mock didCompleteLayoutTransition:OCMOCK_ANY]).onMainThread();


These method calls are only expected when the correct behavior is not enforced. The correct behavior is that the layout transition will be cancelled and this it won't animate and complete.

nguyenhuy · 2017-07-07T15:31:36Z

Tests/ASLayoutEngineTests.mm

+ * While it's measuring, on main switch to fixture 4 (setNeedsLayout A, D) and run a CA layout pass.
+ *
+ * Correct behavior, we end up at fixture 4 since it's newer.
+ * Current incorrect behavior, we end up at fixture 2 and we remeasure surviving node C.


I believe this unexpected behavior is caused by this early check in __layout, which is introduced in #2657.

nguyenhuy · 2017-07-07T15:36:46Z

Tests/Common/debugbreak.h

+//  You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//


Do we really need this license? Or added just to make Danger happy?

Just to make Danger happy.

That's sad :(

nguyenhuy · 2017-07-07T15:40:55Z

Tests/ASLayoutEngineTests.mm

+     * The returned value (8) is clamped to the fixed with (7), and then compared to the previous
+     * width (7) and we decide not to propagate up the invalidation, and we stay stuck on the old
+     * layout (fixture3).
+     */


Is this because nodes only store their last-used constraint?

Yes exactly!

nguyenhuy · 2017-07-07T16:02:21Z

Btw, I don't think this diff is trivial. I'd suggest to add something to the change log. It's your call though.

ghost · 2017-07-07T23:02:14Z

🚫 CI failed with log

Adlai-Holler · 2017-07-08T02:29:23Z

The CI failure this time was interesting! I'll put this info somewhere else in the future, but I want to make sure not to lose it:

This is an extremely idealized version of a 3 hour slog to diagnose a very tricky (for me) race condition. I believe next time the slog can be way shorter with these learnings. To diagnose, I override -invokeTest like below, and told Xcode just to run the single test method (right click it, or use Ctrl+Alt+Cmd+u).

- (void)invokeTest
{
  for (int i = 0; i < 10000; i++) {
    NSLog(@"***Invocation #%d***", i);
    [super invokeTest];
    [NSRunLoop.mainRunLoop runMode:NSDefaultRunLoopMode beforeDate:NSDate.distantPast];
  }
}

Note: I also had to change the dispatch_once_ts away from static so that they would fire again on next iteration.

I then turned on verbose logging ASEnableVerboseLogging.

Then I set a breakpoint where OCMReportFailure is called. This is important, because the "Test failure breakpoint" Xcode provides only handles the failure macro, not explicitly invoking recordFailureAtLine:. We could modify/fork OCMock to use the _XCTFailureHandler macro to cover this base in the future.

Then I walked away and got a coffee while the 10000 iterations ran.

When I came back, I was at the failure breakpoint (the extra layout pass wasn't running). So I dumped the log for that iteration, plus the previous iteration. I use Sublime text to strip out all timestamps and pointer values from the log, then I ran diff on the two log chunks. The diff showed that in the test fail case, the node still had a pending layout that it wanted to use in __layout and in the common case, the node had null pending layout.

I added some more log statements to cover when _pendingDisplayNodeLayout changed and ran the process again. I grabbed the good & bad log chunks, stripped them down, ran the diff and got this diff:

Conclusion: If CA runs a layout pass on the main thread before the transition can get a hold of nodeC's lock on the background thread, then no extra layout happens because the pending layout that gets established can be used in the layout pass for the new fixture (C's layout between the two target layouts is the same, D's is not and that's why its behavior is more predictable). If instead CA runs its layout pass after the bg layout gets set as pending, nodeC will discard it as part of the layout pass and have to regenerate it. Note that it's really nodeA's lock that matters here, since nodeA's lock is held throughout the calculation process, and nodeA needs layout so it's a race for the two threads to lock node A.

Not sure yet what I'll do with this knowledge, but there it is.

nguyenhuy · 2017-07-17T10:59:52Z

@Adlai-Holler Please fix the file conflicts and merge away.

maicki

Let's get that in!

ghost · 2017-08-03T10:55:10Z

🚫 CI failed with log

ghost · 2017-11-28T13:17:32Z

🚫 CI failed with log

ghost · 2017-11-28T15:19:04Z

🚫 CI failed with log

ghost · 2017-11-28T15:26:48Z

🚫 CI failed with log

ghost · 2017-11-30T17:16:32Z

🚫 CI failed with log

Adlai-Holler

Thanks for picking this up man. Empty test failure log. wth? cc @garrettmoon any ideas here? It took an hour which suggests a timeout?

Adlai-Holler · 2017-11-30T19:21:26Z

Source/Layout/ASLayout.mm

-  }
-  return NO;
-}
-


Oh wow. Very very scary that this never warned of duplicate method implementation.

nguyenhuy · 2017-11-30T19:57:32Z

FYI, these new tests pass locally if I apply this diff on top of #695.

And yes, we should make sure failure logs have enough info.

Before: - Even if a pending layout was applied before, it'll be unnecessarily applied again in next layout passes and cause `-calculatedLayoutDidChange` being called multiple times. After: - If a pending layout was applied, the calculated layout will not be ignored but reused, if possible, in next layout passes. Test plan: testSetNeedsLayoutAndNormalLayoutPass in #424.

ghost · 2017-12-01T14:31:50Z

🚫 CI failed with log

ghost · 2017-12-01T16:10:16Z

	1 Warning
⚠️	This is a big PR, please consider splitting it up to ease code review.

Generated by 🚫 Danger

…reGroup#695) Before: - Even if a pending layout was applied before, it'll be unnecessarily applied again in next layout passes and cause `-calculatedLayoutDidChange` being called multiple times. After: - If a pending layout was applied, the calculated layout will not be ignored but reused, if possible, in next layout passes. Test plan: testSetNeedsLayoutAndNormalLayoutPass in TextureGroup#424.

* Build testing platform & tests for the layout engine * Add our license header to debugbreak. * Remove thing * Address review comments * Beef up the logging * Update -[ASLayout isEqual:] * testLayoutTransitionWithAsyncMeasurement passes now * Disable testASetNeedsLayoutInterferingWithTheCurrentTransition * Fix build errors

Adlai-Holler added the Layout label Jul 7, 2017

Adlai-Holler requested review from maicki, garrettmoon, appleguy and nguyenhuy July 7, 2017 00:56

Adlai-Holler force-pushed the AHLayoutEngineTests branch from 2efb7b6 to c894eb3 Compare July 7, 2017 00:59

Adlai-Holler changed the title ~~Add unit tests for the layout engine~~ Add unit tests for the layout engine #trivial Jul 7, 2017

nguyenhuy approved these changes Jul 7, 2017

View reviewed changes

Adlai-Holler changed the title ~~Add unit tests for the layout engine #trivial~~ Add unit tests for the layout engine Jul 7, 2017

maicki approved these changes Jul 17, 2017

View reviewed changes

remirobert approved these changes Aug 13, 2017

View reviewed changes

nguyenhuy force-pushed the AHLayoutEngineTests branch from 9552195 to 5d38b33 Compare November 30, 2017 17:00

nguyenhuy mentioned this pull request Nov 30, 2017

[ASDisplayNode+Layout] Ensure a pending layout is applied once #695

Merged

Adlai-Holler commented Nov 30, 2017

View reviewed changes

Adlai Holler added 5 commits December 1, 2017 14:24

Build testing platform & tests for the layout engine

e2ce0b0

Add our license header to debugbreak.

94c8e55

Remove thing

5a3e135

Address review comments

0c6cbaf

Beef up the logging

e312125

nguyenhuy and others added 3 commits December 1, 2017 14:27

Update -[ASLayout isEqual:]

616bf00

testLayoutTransitionWithAsyncMeasurement passes now

94d7c80

Disable testASetNeedsLayoutInterferingWithTheCurrentTransition

ceddc77

nguyenhuy force-pushed the AHLayoutEngineTests branch from 5d38b33 to ceddc77 Compare December 1, 2017 14:29

nguyenhuy and others added 2 commits December 1, 2017 14:35

Fix build errors

1e5c959

Merge branch 'master' into AHLayoutEngineTests

6345417

nguyenhuy merged commit 0dc7002 into master Dec 1, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add unit tests for the layout engine #424

Add unit tests for the layout engine #424

Adlai-Holler commented Jul 7, 2017 •

edited

Loading

nguyenhuy left a comment •

edited

Loading

nguyenhuy Jul 7, 2017

nguyenhuy Jul 7, 2017

nguyenhuy Jul 7, 2017

nguyenhuy Jul 7, 2017

nguyenhuy Jul 7, 2017

Adlai-Holler Jul 7, 2017

nguyenhuy Jul 8, 2017

nguyenhuy Jul 7, 2017

Adlai-Holler Jul 7, 2017

nguyenhuy commented Jul 7, 2017

ghost commented Jul 7, 2017

Adlai-Holler commented Jul 8, 2017

nguyenhuy commented Jul 17, 2017

maicki left a comment

ghost commented Aug 3, 2017

ghost commented Nov 28, 2017

ghost commented Nov 28, 2017

ghost commented Nov 28, 2017

ghost commented Nov 30, 2017

Adlai-Holler left a comment •

edited

Loading

Adlai-Holler Nov 30, 2017

nguyenhuy commented Nov 30, 2017 •

edited

Loading

ghost commented Dec 1, 2017

ghost commented Dec 1, 2017

Add unit tests for the layout engine #424

Add unit tests for the layout engine #424

Conversation

Adlai-Holler commented Jul 7, 2017 • edited Loading

nguyenhuy left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nguyenhuy commented Jul 7, 2017

ghost commented Jul 7, 2017

Adlai-Holler commented Jul 8, 2017

nguyenhuy commented Jul 17, 2017

maicki left a comment

Choose a reason for hiding this comment

ghost commented Aug 3, 2017

ghost commented Nov 28, 2017

ghost commented Nov 28, 2017

ghost commented Nov 28, 2017

ghost commented Nov 30, 2017

Adlai-Holler left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nguyenhuy commented Nov 30, 2017 • edited Loading

ghost commented Dec 1, 2017

ghost commented Dec 1, 2017

Adlai-Holler commented Jul 7, 2017 •

edited

Loading

nguyenhuy left a comment •

edited

Loading

Adlai-Holler left a comment •

edited

Loading

nguyenhuy commented Nov 30, 2017 •

edited

Loading