__COMPAT_LAYER=EnableNXShowUI

I encountered a strange problem today where my korn shell on XP wasn’t behaving properly. In particular, pipes weren’t working, effectively making ksh useless.

I eventually noticed that the problem would go away if I opened up Control Panel->System, Advanced tab, Environment Variables and then closed it. I opened a console before and after doing this and found that the environment was changing even though I wasn’t adding or removing anything.

Before going into the Environment Variables configuration window the __COMPAT_LAYER environment variable was set to “EnableNXShowUI” but afterwards it was not set at all. Further investigation found that the [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers] registry key had this set for explorer.exe. I’ve deleted this registry setting a few times but it keeps coming back.

I don’t really want to be having to unset this every time I start up, but until I find out what’s setting the registry setting I guess I’ll have to make environment variable configuration part of my morning boot routine.