Synopsis: Reserving a large GPU cluster does not mean an organization will keep it busy. Capacity estimates, failed runs and changing requirements can leave expensive resources underused. At the same time, specialized accelerators make it harder to decide where each part of an AI workflow belongs. Improving efficiency calls for more than acquiring hardware: it requires matching work to the right architecture and revisiting those decisions as demand changes.
Stephan Fabel, CEO of ThisWay Global, discusses that operational challenge with Mike Vizard. He describes a shift from treating AI primarily as training and inference toward a wider range of workloads embedded in applications, CI/CD pipelines and edge systems. Different parts of those workflows can favor different processors. Model updates also become another recurring responsibility within software delivery. Fabel expects smaller and specialized models to play a larger role, but does not assume that every useful model will be open source.
Fabel says the GPU utilization he encountered while investigating the problem was often below 20%. He connects that inefficiency with the practice of reserving a fixed number of GPUs for a fixed period based on an estimate. Training teams may need time to work up to the planned scale, and long commitments do not ensure consistent demand. His proposed alternative gives software more responsibility for workload estimation, placement and ongoing optimization rather than leaving those decisions tied to the original reservation.
Hardware suitability is only one constraint on that placement. Fabel describes orchestration that also accounts for sovereignty, compliance and organizational policies, particularly for regulated workloads. Kubernetes provides orchestration capabilities, but he argues that a more useful abstraction is needed for deciding how GPU capacity should be consumed. Over time, he expects this work to become accessible to mainstream IT administrators rather than remain the province of specialist teams. That makes operational control and efficient resource use part of the broader enterprise AI adoption plan.
