@samitouri / QOSAMI-WSL-2
Code Commits Issues Pulls Sessions CI/CD
master
master
md 189 lines 8.04 KB
Rendered Raw
1 # Testing
2
3 ## Setup
4
5 Tests are created and executed using the [Test Authoring and Execution Framework (TAEF)](https://docs.microsoft.com/windows-hardware/drivers/taef/). Once you have successfully built and deployed the WSL application, all you need are the TAEF binaries to begin authoring and running tests. It is best practice to use taef binaries included in the Microsoft.Taef nuget package used to compile the tests. For example: `packages\Microsoft.Taef.10.77.230207002\build\Binaries`
6
7 ## Executing Tests
8
9 Executing tests with TAEF is done by invoking the `TE.exe` binary:
10
11 1. Open a command prompt with administrative privileges.
12 2. Navigate to the subdirectory containing the built test binaries (`bin/<X64|Arm64>/<Debug|Release>/`)
13 3. Execute the binaries via invoking TE and passing the test dll/s as arguments: `TE.exe test1.dll test2.dll test3.dll`
14
15 ## test.bat Options
16
17 The following options are handled by `test.bat` / `run-tests.ps1` before invoking TE.exe:
18
19 ### **/attachdebugger**
20
21 Automatically launches WinDbgX and attaches it to the test host process. Requires [WinDbg](https://aka.ms/windbg) to be installed (`winget install Microsoft.WinDbg`). Under the hood it passes `/waitfordebugger /inproc` to TE.exe so tests run in-process, then attaches WinDbgX directly to `TE.exe`.
22
23 `test.bat /attachdebugger /name:*MyTest*`
24
25 ## Useful **TE.exe** Command Line Parameters for Debugging/Executing Tests
26
27 Command Line parameters are passed to `TE.exe` after supplying the target `.dll`:
28
29 ### **/list**
30
31 Lists the individual tests loaded from the test `.dll` passed in:
32
33 `TE.exe test1.dll test2.dll /list`
34
35 ### **/name:\<testname\>**
36
37 Specifies a specific test or group of tests, supporting wildcards `*` and `?` to execute (without this, every test will be run on invoke):
38
39 `TE.exe test1.dll /name:*HelloWorldTest*`
40
41 ### **/inproc**
42
43 Very useful for debugging via WinDbg, executes tests within the TE.exe process and not the TE.ProcessHost.exe child process:
44
45 `TE.exe test1.dll /inproc`
46
47 ### **/breakOnCreate /breakOnError /breakOnInvoke**
48
49 Especially useful for WinDbg debugging when coupled with `/inproc`. They break into the debugger if/on: before instantiating a test class, if a error or test failure is logged, and prior to test method invoking, respectively.
50
51 `TE.exe test1.dll /inproc /breakOnCreate /breakOnError /breakOnInvoke`
52
53 ### **/p:\<paramName\>=\<paramName\>**
54
55 Used for passing runtime parameters to test methods, as well as to setup and cleanup methods. Be mindful of the use of quotation marks.
56
57 `TE.exe test1.dll /p:"foo=hello" /p:"bar=2"`
58
59 These variables can be retrieved in test source code using the following example:
60
61 ```cpp
62 using namespace WEX::Common;
63 using namespace WEX::TestExecution;
64
65 String runtimeParamString;
66 DWORD fooBar;
67
68 VERIFY_SUCCEEDED(RuntimeParameters::TryGetValue(L"foo", runtimeParamString));
69 VERIFY_SUCCEEDED(RuntimeParameters::TryGetValue(L"bar", fooBar));
70 ```
71
72 ### **/runas:<\RunAsType\>**
73
74 Specifies the environment to run the tests in:
75
76 `TE.exe *.dll /runas:<System|Elevated|Restricted|LowIL|AppContainer|etc>`
77
78 ### **/sessionTime:<\value\>**
79
80 Specify a timeout for the **TE.exe** execution, which aborts on timeout.
81
82 `TE.exe test1.dll /sessionTimeout:0:0:0.5 // [Day.]Hour[:Minute[:Second[.FractionalSeconds]]`
83
84 ## Creating Tests
85
86 A good example for [how to create tests with TAEF](https://docs.microsoft.com/windows-hardware/drivers/taef/authoring-tests-in-c--) can be found in the `/test/SimpleTests.cpp`, `/test/MountTests.cpp`, and `/test/CMakeLists.txt`.
87
88 Make sure to locate the TAEF header file the files at `%\Program Files (x86)\Windows Kits\10\Testing\Development\inc\WexTestClass.h`.
89
90 Below is a brief overview:
91
92 ### Writing the Test
93
94 For tests that only apply to a specific WSL version, use the version-specific test method macros instead of `TEST_METHOD`:
95
96 - `WSL2_TEST_METHOD(Name)` — test only runs on WSL2
97 - `WSL1_TEST_METHOD(Name)` — test only runs on WSL1
98 - `WSLC_TEST_METHOD(Name)` — test only runs on WSL2 (for use in WSLC test classes)
99 - `TEST_METHOD(Name)` — test runs on both WSL1 and WSL2
100
101 These macros use TAEF metadata properties to tag tests with their required WSL version.
102 When tests are run via `run-tests.ps1` or CloudTest, a `/select:` query automatically
103 filters out tests that don't match the target version—so they don't appear in results at all
104 (no "skipped" noise).
105
106 For example, consider the file below, named `ExampleTest.cpp`:
107
108 ```cpp
109 #include "WexTestClass.h" // this included be used for creating TAEF tests classes
110
111 #include "Common.h" // referring to /test/Common.h, where general utility functions for interacting with WSL in regards to testing reside
112
113 #define INLINE_TEST_METHOD_MARKUP // optional, but defined within the directory cmake build instructions. this is the practice that the preexisting tests use
114
115 namespace ExampleTest
116 {
117 class ExampleTest
118 {
119 TEST_CLASS(ExampleTest) // define this as a test class
120
121 // runs on both WSL1 and WSL2
122 TEST_METHOD(HelloWorldTest)
123 {
124 std::wstring outputExpected = L"Linux on Windows Rocks!\n";
125 auto [output, __] = LxsstuLaunchWslAndCaptureOutput(L"echo Linux on Windows Rocks!"); // from /test/Common.h
126 VERIFY_ARE_EQUAL(output, outputExpected); // TAEF test method that passes if both are equal, and fails otherwise.
127 }
128
129 // only runs on WSL2
130 WSL2_TEST_METHOD(Wsl2OnlyTest)
131 {
132 // ...
133 }
134 };
135 } //namespace ExampleTest
136 ```
137
138 For more in-depth examples of writing TAEF tests, check out `/tests/MountTests.cpp` and [Advanced Authoring Tests in C++](https://docs.microsoft.com/windows-hardware/drivers/taef/authoring-tests-in-c--#advanced-authoring-tests-in-c).
139
140 ## Building Tests
141
142 ### CMake
143
144 For examples on how to get your test/s building within the repo, please view `/test/CMakeLists.txt` for the structure of creating add to the `wsltest.dll`. For additional information on how to use CMake, try [CMake Documentation and Community](https://cmake.org/documentation/).
145
146 ### Building
147
148 Follow the same instructions listed at the root of this repository and build the application as you would regularly.
149
150 ### Executing
151
152 See the parts above for how to run your new test, but if nothing went awry, your shiny new test dll should be placed in the binary directory. Try running it with:
153
154 `TE.exe exampletest.dll`
155
156 ## Existing Tests
157
158 To run all existing tests: `TE.exe wsltests.dll`
159
160 ### SimpleTests
161
162 Very basic tests focusing on the connection to WSL. Tests examine commands like `wsl echo`, `wsl --user`, and `wsl --cd`.
163
164 Run these with: `TE.exe wsltests.dll /name:*SimpleTests*`
165
166 ### MountTests
167
168 Tests focusing on the `wsl --mount` functionality. These tests include things like: `--bare` mounting, mounting disk partitions, mounting FAT partitions, etc.
169
170 Run these with: `TE.exe wsltests.dll /name:*MountTests*`
171 ### NetworkTests
172
173 Tests focusing on the networking aspects of WSL. These are also used to test certain functionality like WSL configurations related to networking, mirrored networking, flow steering, etc.
174
175 Run these with `TE.exe wsltests.dll /name:*NetworkTests*`
176 ### Plan9Tests
177
178 Tests that focus on validating the functionality of the Plan 9 filesystem component of WSL, testing filesystem-related operations like the creation, deletion, and I/O of files and directories.
179
180 Run these with: `TE.exe wsltests.dll /name:*Plan9Tests*`
181
182 ### UnitTests
183
184 Tests that assess general Linux behavior from within the distribution and the features/changes WSL has made on the Linux side. This includes process creation, signals, sockets, etc.
185 The individual tests are located under `linux/unit_test/*.c` with the exception of `systemd` tests, which are defined in the `windows/UnitTests.cpp`.
186
187 Run all unit tests with: `TE.exe wsltests.dll /name:*UnitTests*`
188
189 To run only `systemd` tests, use: `TE.exe wsltests.dll /name:UnitTests::UnitTests::Systemd*`