TL;DR — Key Takeaways
- GPU Taskman provides deeper visibility into local GPU workloads. Created by Microsoft executive Scott Hanselman, the terminal-based tool connects GPU memory usage and activity with individual processes and detected AI runtimes.
- AI-specific insights help explain memory consumption. For supported workloads, it uses model metadata, server APIs and runtime logs to report or estimate memory allocated to model weights, KV caches and compute buffers.
- Support spans multiple operating systems. GPU Taskman runs on Windows, Linux, WSL and macOS on Apple Silicon, with available metrics varying by hardware, driver and operating system.
With AI fever engulfing almost everyone, the GPUs on developers’ computers are probably doing a lot more work these days, as devs test out new models and agents. So it would make sense that local would have its own dedicated monitoring tool for GPUs. Yet, there are surprisingly few dedicated options for the desktop.
Microsoft technical executive Scott Hanselman has created a tool called GPU Taskman, or “gputaskman” to the command line, to display what is occupying the resources of the GPU on your system.
“There’s a lot going on in my GPU. I’m making a GPU Taskman to get insights to where my GPU memory is going,” he wrote in an X post introducing the project.
He welcomes outside contributions to the project, which is written in Python and available as portable binaries or installable from PyPI.
What GPU Taskman Monitors
Out of the box, Windows’ built-in Task Manager already provides a basic set of information about the system’s GPU, including GPU utilization, which applications are using the GPU, the VRAM usage per application and other details.
There is still a lot that gets left out, however.
Hanselman likens GPU Taskman to Linux’s htop, offering a detailed view from the terminal of what is exactly gobbling up your GPU’s resources, as well as how it is operating overall. GPU Taskman can work with GPUs from both Nvidia and AMD.
For those running AI at home, such insight can be valuable. Users can see usage-by-process, revealing model weights, KV caches, Web browsers or some other sub-tasks chewing up memory.
It is particularly knowledgeable on AI and LLM workloads. The software can work with local servers such as Ollama or LM Studio to extract meta-information such as context window sizes and throughput metrics.
It identifies the loaded runtimes of widely-used apps and models. It can separate out the weights from the KV cache and compute buffers of a locally-running model. It also understands fine-tuning tools like Unsloth, as well as various vision and speech models.
One online user, replying to Hanselman’s X post, noted that GPU Taskman showed the great variance in how much GPU memory the Desktop Window Manager (dwm) uses from system to system. Another user appreciated that the software reported the temperature of the GPU, and in Celsius at that.
A Debugger Just for Local Environments
Best of all, GPU Taskman is not an app just for Windows, though it supports Windows, even the ARM64 version. It also runs on WSL (Windows Subsystem for Linux), and can run on regular Linux. It even recognizes containers on Linux, using a technique to resolve containerized PIDs, in order to give the user a glimpse of workloads within those containers. Apple is also supported.
Keep in mind, the manager is only for local environments; it won’t help with cloud services, or services that don’t use a GPU.
Coincidentally, Hanselman co-hosts a podcast with Mark Russinovich, who, in 1996, authored Sysinternals, a suite of diagnostic tools that revealed all sorts of operating details that Microsoft’s own Windows diagnostic tools didn’t cover then. Russinovich was working independently at the time, though Microsoft later hired him, and he now serves as a CTO for Azure. So this Hanselman fellow may be going places as well.
