Explicit parameter from xunit.v3 not working in Visual Studio Test Explorer

[vbreuss]

Explicit tests are never run in the Visual Studio Test Explorer, regardless if a single test, a test class or all tests are selected to run. This affects both Fact and Theory attributed tests.

I created a repository to reproduce the issue here, but basically it's adding the following test and trying to execute it in VS:

[Fact(Explicit = true)]
public void ExplicitFact()
{
    Assert.True(false);
}

[bradwilson]

Thanks! I'll take a look.

[bradwilson]

I've been thinking about how to interpret the "gesture" the user makes here, since Test Explorer doesn't have any knowledge of explicit tests, so there's no way to know exactly what the user has asked to do here.

I believe that where I landed is this, and let me know what you think.

If the user has asked to run specific tests, that may mean they did something like "please run this one test as it's explicit", or it could also mean they said "please run all the tests in this one assembly" and Test Explorer decided to hand back the full set of tests.

Therefore: if the requested test list is entirely explicit tests, then we will run them; if the test list includes both explicit and non-explicit tests (aka, the "run this class" or "run this assembly" version), then we will only run the non-explicit tests.

What do you think of this interpretation @vbreuss?

[vbreuss]

I think this is a legitimate interpretation, @bradwilson and also similar to the implementation in NUnit, as expressed in this comment:

the simple heuristic of dropping any tests that contain the Explicit trait if there are more than one test being invoked and any other test does not contain the Explicit trait

In my project, I currently use NUnit for some special case tests to activate/deactivate certain features and had to introduce a dummy test without the Explicit attribute in the test class, so that the explicit tests are not run automatically in visual studio.

[bradwilson]

Fixed in 3.0.0-pre.26 https://xunit.net/docs/using-ci-builds

Note you will also need to pick up 0.2.0-pre.87 or later of xunit.v3 for this to work. We previously weren't reporting Explicit during test case discovery, which we need to make this feature work. Without this upgrade you'll still have the broken behavior, as the default value for a missing Explicit during test case discovery deserialization is false.

[vbreuss]

Thanks for your effort, @bradwilson. I tested the updated packages and it worked as expected!