ENABLE_VIRTUAL_TERMINAL_PROCESSING is set by default in pseudo console on Windows 10 20H2.

[tyan0]

Environment

Windows 10 20H2 (Version 10.0.19042.804)

Steps to reproduce

1. Compile the following test code.
2. Run the test program in pseudo console (e.g. in WSL).

#include <windows.h>
#include <stdio.h>

int main()
{
	DWORD mode;
	GetConsoleMode(GetStdHandle(STD_OUTPUT_HANDLE), &mode);
	printf("%d\n", mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING);
	return 0;
}

Expected behavior

At least on older Windows 10 such as 1809, the result is:

0

Actual behavior

On Windows 20H2, the result is:

4

Is this the intentional behavior?

[eryksun]

Is this the intentional behavior?

Yes. See src/host/srvinit.cpp.

[zadjii-msft]

Yep, this is by design.

/dup #1965

[ghost]

Hi! We've identified this issue as a duplicate of another one that already exists on this Issue Tracker. This specific instance is being closed in favor of tracking the concern over on the referenced thread. Thanks for your report!

[tyan0]

Thank you for your quick response. Just a question from my interest, is it possible to disable this flag by default in pseudo console? Setting HKCU\Console\VirtualTerminalLevel to 0 does not take effect.

[zadjii-msft]

A better question might be: why do you want to disable VT processing? It might help to better understand your scenario.

[tyan0]

I asked just because I was interested. I don't have the scenario in which this behavior causes any problem so far.

[vefatica]

I followed the link provided by @eryksun and I'm just curious ... is the conditional expression below (from srvinit.cpp) well-formed?

bool isEnabled = false;
if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled))

[zadjii-msft]

@vefatica Sure is. That statement is basically two parts:

1. CheckDefaultAppPolicy will get the state of the "default app policy", and stick it into the bool passed in. CheckDefaultAppPolicy will return true if it succeeded in looking that value up.
2. Was the "default app policy" enabled?

[vefatica]

In that case I'd expect it to look like this:

if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(&isEnabled)) && isEnabled)

[zadjii-msft]

Meh, stylistic difference. Taking a param by reference means the callee doesn't need to worry about if the out param is null or not. Most of our newer code with out params (as few as those are) uses params by reference, rather than pointers.

[eryksun]

bool isEnabled = false;
if (SUCCEEDED(Microsoft::Console::Internal::DefaultApp::CheckDefaultAppPolicy(isEnabled) && isEnabled))

The only argument of the SUCCEEDED macro should be the HRESULT that's returned by CheckDefaultAppPolicy, in order to get a boolean result of success or not. Negative HRESULT values indicate failure. Doing a logical && on an HRESULT and a bool is incorrect.

[vefatica]

OK, didn't know that was by reference. And the second isEnabled being inside SUCCEEDED(). It just looks really funny to me.

[zadjii-msft]

oh no

[miniksa]

WELP. I am fixing this as MSFT:32071839. We will replicate that back out to fix my parenthesis error in the next day or so. Good catch and thank you.