Windows Task Manager can see AI work without understanding it

Diagnostic brief · Tech

Three rules keep AI telemetry honest

Task Manager has become a useful first instrument for local AI. Its new visibility is strongest when each counter is treated as a clue rather than a verdict.

The accelerator is now attributablePer-process NPU and GPU neural-engine activity can show which application is using AI-capable silicon and whether work lands on the intended engine.
Utilization is not model evaluationA busy NPU proves execution. It does not prove that the output is accurate, that the entire pipeline is efficient or that the chosen model is appropriate.
A reproducible symptom deserves a recorded traceMove from Task Manager to WPR, WPA and ONNX Runtime events when the question changes from “where?” to “why?”

Windows Task Manager now exposes per-process activity for the neural processing unit and the GPU neural engine on newer devices. That sounds like a small interface addition. In practice, it changes the first five minutes of diagnosing a local AI feature: a developer or IT operator can see whether computation is active, which process owns it and whether the workload reached the accelerator it was supposed to use.

The dangerous interpretation begins one step later. A visible spike does not say that every stage of the feature ran on the NPU, that the model loaded efficiently, that an inference completed quickly or that its answer was correct. Task Manager makes execution observable. It does not make the system self-explanatory.

The new columns answer a routing question

Microsoft’s August 2026 guidance describes NPU and GPU neural-engine activity appearing alongside familiar process and performance data. For IT professionals, the immediate value is performance triage. For developers, it is hardware-placement evidence: a model intended for the NPU should not silently consume CPU or ordinary GPU compute while the interface remains slow and the battery drains.

That distinction matters because Windows ML uses execution providers to connect ONNX Runtime with CPU, GPU or NPU backends. The silicon classes are not interchangeable. Microsoft characterizes the NPU as the battery-efficient choice for sustained on-device inference, the GPU as the high-throughput option for image, video and generative workloads, and the CPU as the universal fallback for small or unsupported workloads.

What the live view can establish

Evidence about execution

The process using NPU or GPU neural-engine capacity
Whether activity coincides with the reproduced feature
Whether the intended accelerator appears idle
Whether utilization persists after the feature should stop

What the counters cannot establish

Claims about the model

That the output is accurate, safe or useful
That pre-processing and post-processing are efficient
That a high percentage means high throughput
That the same route will hold on another device or driver

One inference can be slow outside the NPU

An AI feature is a pipeline, not one accelerator event. The CPU may decode an image, tokenize text, resize an input or prepare tensors before the model runs. After inference, CPU or GPU work may decode the output, rank candidates, apply safety logic, render a preview or move data through the interface. A short NPU burst inside a long interaction can therefore be completely healthy, while an apparently modest NPU graph can still hide the stage users actually feel.

This is also why raw utilization is not a throughput metric. A percentage does not disclose model size, batch size, queueing, memory traffic, power state or the amount of useful work completed. Comparing two applications by the height of their NPU graphs would be like comparing two downloads by CPU use: the signal is real, but the inference drawn from it is underspecified.

One inference · three observation points

The visible accelerator event sits inside a longer user interaction

Read a slow feature from left to right. The NPU is only the middle stage, and Task Manager is strongest at identifying that middle.

Prepare · mostly CPUDecode and shape the inputTokenization, resizing, tensor preparation and session setup can dominate before the accelerator receives work.

Execute · NPU or GPURun model operatorsTask Manager can reveal activity and process ownership; a trace can reveal timing, callstacks and execution-provider details.

Return · CPU or GPUTurn tensors into a featurePost-processing, safety rules, ranking, copying and rendering may explain the delay after inference has ended.

An evidence trace separating Task Manager accelerator visibility, Windows Performance Recorder and Analyzer diagnostics, and model evaluation — Task Manager, a recorded performance trace and model evaluation answer different questions. A strong diagnosis does not ask one layer to substitute for another.

Task Manager should decide the next instrument

The live view is sufficient when the question is binary and local: did the process use the NPU, did GPU neural activity appear, or did the workload fall back to CPU? When the symptom is repeatable but its cause is not visible, Microsoft’s own developer guidance moves to Windows Performance Recorder and Windows Performance Analyzer.

WPR includes a Neural Processing profile that records NPU activity. WPA can align that activity with CPU work, callstacks and ONNX Runtime events on one timeline. Those runtime events can expose model-load time, session creation, execution-provider configuration, individual inference timing and the operators that contributed to total latency. The important upgrade is not “more graphs.” It is a time-correlated record of the pipeline that produced the symptom.

Diagnostic tool · AI workload autopsy

Convert one user complaint into the smallest sufficient trace

Start with the visible symptom, then capture only the evidence needed to separate placement, latency and result quality.

The complaint

“Battery life collapses.”
Check whether sustained work is on CPU or standard GPU compute instead of the NPU.

“The first result is slow.”
Separate model and session load from subsequent inference time.

“The NPU is busy but the app waits.”
Look for CPU preparation, post-processing, copies and UI work around the accelerator event.

“The answer is bad.”
Stop profiling utilization and evaluate prompts, outputs and model behavior.

The next proof

Task Manager
Use for process attribution, live NPU/GPU neural activity and a fast check for unexpected fallback.

WPR + WPA
Use for a reproducible timeline joining accelerator activity, CPU callstacks and pre/post-processing.

ONNX Runtime events
Use for session load, execution-provider settings, per-inference timing and operator-level cost.

Model evaluation
Use for correctness, safety, relevance and failure cases. Hardware counters cannot supply this evidence.

Efficiency settings have to match the processor that actually runs

Microsoft’s current Windows ML efficiency guidance makes placement operational rather than cosmetic. Background CPU inference can use EcoQoS to favor efficient scheduling and power states. For an NPU session, the portable control is the ONNX Runtime dynamic option ep.dynamic.workload_type=Efficient, which lets the execution provider map efficiency intent to device-specific scheduling or power behavior.

The trade-off is explicit: efficient NPU mode may increase latency when higher-priority accelerator work is active. That makes it appropriate for background tasks without strict response requirements, not a universal “battery saver” switch. Microsoft recommends validating changes with representative inference latency, accelerator utilization and power consumption together. Task Manager supplies one of those observations, not the entire verdict.

The boundary protects both engineering and governance

Hardware placement and model quality belong to different evidence systems. An application can use the NPU exactly as designed and still hallucinate, rank badly or expose unsafe behavior. It can also produce excellent results while falling back to the CPU and damaging battery life. Combining those judgments into one “AI performance” score hides the action an engineer should take next.

This is different from the File Explorer context-menu performance budget, which measures the delay between a gesture and a usable command surface. It is also different from the Hotpatch servicing-state analysis, which follows a device across baseline and hotpatch months. AI workload visibility is an attribution problem nested inside a pipeline: first locate the computation, then explain its timing, then judge its output.

Use Task Manager to falsify the first assumption: “the model is running where we intended.” Once that assumption survives, stop staring at utilization and collect the evidence for the next question.

Questions about Task Manager AI workload metrics

Does high NPU utilization mean an AI feature is fast?

No. Utilization does not disclose completed inferences, latency, model size, batch size or the CPU and GPU work surrounding inference. Measure latency and power under a representative workload.

Can Task Manager show which application is using the NPU?

On supported newer devices with the update, Microsoft says per-process NPU and GPU neural-engine activity is available in Task Manager. That makes process attribution one of the feature’s strongest uses.

Why can an app feel slow while the NPU graph looks normal?

Input preparation, model loading, memory copies, post-processing and interface work can occur on the CPU or GPU before and after the NPU event. A recorded WPR/WPA trace can align those stages.

Does NPU activity prove that an AI answer is trustworthy?

No. NPU activity is evidence that computation occurred on an accelerator. Correctness, safety and usefulness require a separate model-evaluation process using prompts, outputs and defined acceptance criteria.

Official Microsoft AI workload sources

Microsoft materials checked 25 August 2026