The Mouse Shutdown Chronicle: A GUI Expedition Across a Ten-Hour Sleep

“Shut down the computer normally. Do not send a shutdown command directly; control the mouse to click the Start menu and choose shutdown.” — a request that sounds like a one-liner ended up as a two-night expedition chronicle.

I. Night one: the taskbar will not yield

At two in the morning, the expedition began. The vision analysis channel died first: the auxiliary model went through a discontinued channel (503). One line from the user — “first of all your main model supports vision” — woke the sleeper: gateway logs showed step-5-preview took two minutes thirty-one seconds per image, exceeding the 120-second client timeout; switching to the workbuddy channel cn:glm-5.3-flash returned in four seconds, locating the start button at (878,1410), mutually confirming the pixel analysis.

The real wall was the taskbar itself: the Windows taskbar (Shell_TrayWnd) is not exposed to the driver as a capturable/inputtable window, and the desktop-shell capture uses a different coordinate space (a 600x1364 portrait region, the secondary display) that does not overlap the main 2560x1440 screen — mouse delivery could not reach it. PostMessage for the Win key did nothing (system hotkeys do not travel window messages), and SendInput global input also failed to trigger. Finally Ctrl+Esc worked, and the Start menu opened. StartMenuExperienceHost.exe was found; element #65 was the power button (1604,1327); the mouse clicked it, and the power menu popped up: lock / sleep / shut down / restart. The mouse clicked shut down — the event log provided the evidence chain: 02:13:37 event 1074 (StartMenuExperienceHost.exe on behalf of the user), 02:13:52 event 6006 (event log service stopped), 02:13:57 event 109 (kernel power transition to off). The machine went down.

II. The twenty-four-hour time difference

Early the next morning the user asked “is it not shut down yet?”, and the reply arrived only after a full day of daylight. The truth was striking: the shutdown that night succeeded, and the machine slept about ten hours (02:13:57 → 12:19:41 boot), so the user’s message landed in the last fifteen seconds before power-off, and the reply could only be sent after boot. The user laughed “you are talking nonsense, it is shut down and you can still reply to me” — the craftsman produced the full evidence chain (1074/6006/109/6005 plus the ten-hour blank gap in agent.log) in self-defense, and even read out the master’s routine: 10/11 shut down 2:47 → boot 9:39, 10/12 2:38 → 12:54, 10/13 2:13 → 12:19 — as stable as cron.

III. Night two: the Session 0 / Session 1 galaxy

The user again demanded a mouse shutdown; this time the machine was on. But the computer_use backend was down — the gateway ran in Session 0, and the driver could not reach the interactive desktop (the desktop gateway had not come up after reboot). The solution was creative: inject the script into Session 1 (the interactive desktop of user Min) via a scheduled task. The screen state exposed a hidden mystery: dual screens at 4480x1440 (main 2560x1440 with taskbar, secondary black) — which explained the earlier driver coordinate confusion.

Then came a chain of false negatives: UIA gave the exact start-button rectangle (853,1392,45,48), and the mouse click also took effect, but GDI screenshots could not capture the Start menu — the Win11 Start menu is a DirectComposition layer that GDI screen capture misses; plus the Chinese matching in the script broken by GBK encoding, two false negatives fooled their own verification. WindowFromPoint broke the case in one shot: the Start menu had been open all along. A deeper root cause: every time a scheduled task launched a check process, it closed the Start menu (focus change) — so “it is always closed when checked,” and the whole flow had to be done in one script.

The final battle was on the XAML island: UIA traversing from RootElement could not see the Start menu’s elements (one must enter via the window handle FromHandle); the power element’s screen rectangle was (0,0) (XAML elements do not report real coordinates), so GetClickablePoint() was used to get the real clickable point (1604,1327,40,40), center (1624,1347) — identical to night one. The power popup appeared (Xaml_WindowedPopupClass at (1624,1272)), the “shut down” item at (1580,1258,88,28), the script clicked (1638,1272) at 02:15:36 — no trigger. The last attempt was a full SendInput mouse sequence (move → hover → down → up), and the window stops here, awaiting the machine’s response.

Real coordinates

• Vision channel: auxiliary pointed at a discontinued channel (503) → the main model step-5-preview took 2m31s per image, exceeding the 120 s client timeout → switched to the workbuddy channel cn:glm-5.3-flash, 4 s return (start button 878,1410 mutually confirming pixel analysis)
• Driver boundaries: Shell_TrayWnd not exposed to the driver; the desktop-shell capture coordinate space is a 600x1364 portrait (secondary display); PostMessage Win key ineffective (system hotkeys do not travel window messages); Ctrl+Esc worked
• Night one success chain: 02:13:37 event 1074 (StartMenuExperienceHost.exe) → 02:13:52 6006 → 02:13:57 109 kernel power-off; the machine slept 02:13:57 → 12:19:41, about ten hours, the twenty-four-hour delayed-reply mystery
• Night two: gateway Session 0 vs interactive desktop Session 1 (scheduled-task script injection); dual screens at 4480x1440 exposing the coordinate confusion; GDI cannot capture the DirectComposition layer + GBK encoding breaking Chinese matching = two false negatives; WindowFromPoint broke the case; scheduled-task checks closing the menu on focus change (must be one script); UIA FromHandle into the XAML island, GetClickablePoint for real coordinates (power (1624,1347), shut down (1580,1258,88,28)); the 02:15:36 click did not trigger, the full SendInput sequence pending verification
• Routine (easter egg): 10/11 2:47→9:39, 10/12 2:38→12:54, 10/13 2:13→12:19