Tail-latency-aware scheduling and graceful degradation in edge-cloud continuum

(2026)

Files

thesis_elec.pdf
  • Open Access
  • Adobe PDF
  • 5.39 MB

Details

Authors
Supervisors
;
Abstract
(en) With the proliferation of latency-critical services and Edge AI, the edge-cloud continuum has emerged as a vital infrastructure for processing data closer to users. Managing resources in these environments is challenging, however, due to the heterogeneous network latencies between users and sites, the limited capacity of edge infrastructures, and the nonlinear queueing dynamics of containerized applications under load. Existing orchestrators rely on hardware utilization metrics or mean performance statistics and consequently fail to provide strict guarantees on tail latency, leading to Service Level Objective (SLO) violations during periods of contention or resource scarcity. This thesis proposes a unified framework for latency-aware scheduling and quality assurance, developed through four progressive contributions. We first establish an analytical foundation, introducing a queueing-theoretic mod\-el that predicts the end-to-end tail latency of containerized applications by faithfully capturing the per-instance request dispatching performed by Kubernetes. We then turn this prediction into a decision procedure, formulating the placement and scaling of a single application as a mixed-integer linear program whose tail-latency constraints are linearized without loss of accuracy. Extending the formulation to multi-tenant environments, we jointly schedule co-located applications with heterogeneous resource footprints under a shared capacity constraint, and introduce heuristics that keep the resulting problem tractable as the number of applications grows. To address the hard capacity limits of the edge, we then introduce a graceful degradation mechanism based on mixed-criticality principles, allowing the system to protect high-criticality workloads while preserving bounded service for low-priority ones through controlled and predictable trade-offs. Finally, we specialize this approach for Edge AI, jointly optimizing model variant selection, placement, and scaling so that resource shortfalls are absorbed by reducing inference quality rather than responsiveness. Validated on real distributed testbeds, the proposed methodology consistently satisfies SLO compliance, improves resource efficiency, and maximizes service quality under varying levels of resource pressure.
Affiliations

Citations

Cao, Y. (2026). Tail-latency-aware scheduling and graceful degradation in edge-cloud continuum. https://hdl.handle.net/2078.5/282367