Unit Tests

In ILSpy we use five different kinds of unit tests:

• Correctness Tests
• Pretty Tests
• IL-Pretty Tests
• Roundtrip Tests
• Ugly Tests

It is important for each language feature to have at least correctness tests. Pretty tests are preferred, because they do test the code output of the decompiler as well, but sometimes it is not possible to create a working pretty test. If that's the case, try to create correctness tests instead. See below for more details.

Test Configurations

Each test is run in four different configurations:

• Debug + Legacy CSC
• Optimize + Legacy CSC
• Debug + Roslyn CSC
• Optimize + Roslyn CSC

Of course, language features that are not supported by the legacy C# compiler should only be tested using a Roslyn configuration.

Correctness Tests

The test runner is /ICSharpCode.Decompiler.Tests/CorrectnessTestRunner.cs. The test cases are located in /ICSharpCode.Decompiler.Tests/TestCases/Correctness/. Each test case consists of either a cs or il file. One test consists of the following steps:

• compile/assemble a test case (call the result "executable 1")
• decompile "executable 1" to C# ("decompiled.cs")
• compile decompiled.cs, resulting in "executable 2"
• run both executable and compare their output (exit code, stdout, stderr)

We repeat the steps above with each test configuration.

The tests pass if the code compiles without error and produces the same output. The tests do not care at all if the resulting code is pretty, or if any high-level constructs like closures were detected.

Adding New Tests

When adding new tests, please make sure that you only check in the test case in CorrectnessTestRunner.cs and the test case files (i.e. source files) in the TestCases/Correctness directory. Any executables that are generated in the test run must not be added. Please note further that the executables are compiled to the TestCases directory, so you can open them in ILSpy for debugging and testing purposes more easily.

Pretty Tests

The test runner is /ICSharpCode.Decompiler.Tests/PrettyTestRunner.cs. The test cases are located in /ICSharpCode.Decompiler.Tests/TestCases/Pretty/. Each test case consists of a cs file and a set of IL files, one for each test configuration. Only the cs file should be checked into the repository.

One test consists of the following steps:

• If no IL file is present for the given configuration, generate it.
• Assemble the generated IL file for the configuration.
• Decompile the generated assembly.
• Compare the source code generated by the decompiler with the original source code and output a diff, if there are any differences.

These steps are repeated with each test configuration.

The test passes, if the output generated by the decompiler matches the code in the cs file. Comments, preprocessor directives and whitespace lines are ignored by the diffing algorithm.

IL-Pretty Tests

TODO

Roundtrip Tests

TODO

Ugly Tests

The test runner is /ICSharpCode.Decompiler.Tests/UglyTestRunner.cs. The test cases are located in /ICSharpCode.Decompiler.Tests/TestCases/Ugly/. Each test case consists of two cs files and a set of IL files, one for each configuration. One cs file contains the test case and the other (named .Expected.cs) contains the expected output, which of course may differ if language features are disabled. It may be necessary to exclude the .Expected.cs from compilation.

Note that the IL files should be checked into source control. Make sure to recreate them when adding a new test case, otherwise the test will succeed regardless of the changes you made!

One test consists of the following steps:

• If no IL file is present for the given configuration, generate it.
• Assemble the generated IL file for the configuration.
• Decompile the generated assembly.
• Compare the source code generated by the decompiler with the .Expected.cs source code and output a diff, if there are any differences.

These steps are repeated with each test configuration.

The test passes, if the output generated by the decompiler matches the code in the .Expected.cs file. Comments, preprocessor directives and whitespace lines are ignored by the diffing algorithm.