Main Content

Fair Inference Serving System in Edge

In multi-tenant edge clusters it is important to retain the performance goals of the tenants in a fair manner.

The task is to design and implement an inference serving system that provisions resources of a fixed-size cluster to a set of tenants with dynamic resource requirements.

Experimental evaluation must compare the performance (e.g., goodput) and fairness with related work.

Link to literature:

• Bridging CPU and GPU autoscaling for cost-efficient inference serving
• A House United Within Itself: SLO-Awareness for On-Premises Containerized ML Inference Clusters via  Faro
• Vertical Scaling Can Save Time: Optimizing Container Scheduling  to Handle Sudden Bursts

Suitability for student:

Bachelor or Master

Contact:

mehran.salmani [at] uni-marburg.de