Main Content

GPU Power Consumption Prediction for DL Inference Workloads

A single inference workload rarely saturates Datacenter GPUs, so operators co-locate several models on one device using NVIDIA's Multi-Process Service (MPS) to partition streaming multiprocessors across processes. This increases utilization but makes power draw hard to anticipate. Since energy-aware scheduling requires power consumption data before a placement is committed, and exhaustive profiling grows combinatorially with the number of tenants, a predictive model is required.

The task is to build a lightweight model that predicts aggregate GPU power for co-located inference workloads, given the SM partition assigned to each and cheap per-workload characterizations. Workloads are profiled only in isolation, using always-on telemetry rather than costly kernel-level instrumentation; co-located runs supply training labels only.

Evaluation should answer questions about the linearity or non-linearity of power consumption with co-located workloads (ideally, reasoning about the disaggregated power), and the accuracy of the prediction model (compared to the measured power draw).

Link to literature:

• Power and Performance Aware Colocation for Efficient GPU Spatial Partitioning
• A GPU resource allocation algorithm with spatial sharing for near-deterministic inferring tasks

Suitability for student:

Bachelor or Master

Contact:

mehran.salmani [at] uni-marburg.de