Self-Adaptive Multi-Cloud Job Scheduling Using Deep Reinforcement Learning and Workload Forecasting

2.1. Heuristic and Metaheuristic Scheduling

Heuristic schedulers such as round-robin, Min-Min, and Max-Min remain widely deployed because they are simple and fast, but they ignore future demand and typically optimize a single objective. Metaheuristics—genetic algorithms, particle swarm optimization, cat-swarm optimization, and Harris-Hawks optimization—search a larger solution space and can incorporate multiple objectives; hybrids such as fault-tolerant, trust-based schedulers that combine Harris-Hawks optimization with DRL have been proposed for multi-cloud settings [15]. However, every metaheuristic must resolve its optimization whenever conditions change, which is expensive at scale and still fundamentally reactive. Cost-efficiency studies on Alibaba traces confirm that statically tuned policies leave substantial savings unrealized when workloads are mixed and bursty [16].

2.2. Deep Reinforcement Learning for Cloud Scheduling

DRL reframes scheduling as sequential decision-making. Foundational systems such as DeepRM [17] and Decima [18] demonstrated that policy-gradient and graph-based agents can learn scheduling strategies competitive with or superior to hand-designed heuristics. Building on value-based methods derived from the Deep Q-Network (DQN) [19] and on policy-gradient methods such as Asynchronous Advantage Actor-Critic (A3C) [46] and PPO [20], recent work has applied DRL to multi-objective task scheduling for energy and cost optimization [3], multi-cloud task scheduling [5], real-time workflow scheduling [4], and hierarchical scheduling [21]. Actor-critic formulations schedule efficiently while respecting quality-of-service (QoS) constraints [2], and server-performance-aware DRL schedulers adapt placement to real-time node load [21, 22]. Graph-neural-network policies trained with PPO model task dependencies explicitly and reduce makespan and energy on workflow workloads [14, 23]. SLA-aware multi-objective RL has been used to allocate GPU and CPU resources under deadline and budget constraints, reporting large reductions in training time and cost while improving SLA compliance [24]. Despite this progress, the majority of these schedulers consume only the present system state; they neither condition decisions on an explicit forecast of future demand nor adapt their reward weighting at run time.

2.3. Cloud Workload Forecasting

Workload forecasting has matured from classical autoregressive integrated moving-average (ARIMA) models [25] to deep sequence models. LSTM [26] and GRU [27] networks capture long-range temporal dependencies in resource-utilization traces, and multivariate attention networks such as MAGD improve accuracy by fusing CPU, memory, disk, and network signals [8]. Uncertainty-aware and transfer-learning predictors additionally quantify forecast confidence and transfer across workloads [10]. Transformer-based forecasters—including online ensemble transformers for predictive auto-scaling [12], lightweight frequency transformers [28], multivariate resource forecasters [9], and adversarial transformers [29]—now define the state of the art for cloud demand prediction, while Prophet/LSTM hybrids drive proactive Kubernetes auto-scaling [30]. DRL has itself been used for workload prediction in federated clouds [31]. These forecasters are typically deployed for auto-scaling rather than scheduling; coupling them directly to a scheduling policy remains comparatively underexplored, which is the gap this work targets.

2.4. Multi-Cloud and Federated Resource Management

Multi-cloud and federated-cloud management add provider heterogeneity, cross-provider migration, and cost arbitrage. Decentralized, cost-aware deep-learning frameworks combine forecasting with adaptive provisioning for load balancing and SLA optimization [31], and reinforcement-learning frameworks such as FedTaskRL target privacy-respecting federated cloud-edge scheduling [32]. Efficient DRL task schedulers designed specifically for multi-cloud environments handle VM availability and inter-cloud task migration [5, 14]. Our work differs from these in treating short-horizon forecasting as a first class input to a self-adaptive PPO scheduler that reasons jointly over per-provider price, availability, energy, and SLA risk.

2.5. Positioning of This Work

Table 1 situates SAMC-DRL against representative recent systems along four axes: whether the scheduler is forecast-aware, whether it optimizes multiple objectives, whether it explicitly models a multi-cloud environment, and whether its reward adapts at run time. To our knowledge, SAMC-DRL is among the first frameworks to combine an attention-based workload forecaster, a PPO scheduler, an adaptive multi-objective reward, and an explicit multi-cloud cost and availability model in a single, reproducible pipeline evaluated on two production traces.

3.1. Multi-Cloud Environment

We consider a federation of P cloud providers (in our evaluation P = 3, emulating AWS, Azure, and GCP-like providers). Each provider p exposes a set of heterogeneous VM types. A VM of type v at provider p is characterized by its processing capacity, measured in millions of instructions per second (MIPS); its memory; its network bandwidth; an energy-per-unit-time coefficient; and a time-varying on-demand or spot price c(p,v,t). At any time t, a subset of VM types may be unavailable at a given provider, modeled by an availability indicator a(p,v,t) ∈ {0, 1}, where 1 denotes available and 0 denotes unavailable. Incoming work arrives as a stream of jobs. Each job is a directed acyclic graph (DAG) of tasks with precedence constraints. A task i is described by its length, measured in millions of instructions (MI); its data size for transfer; an arrival time; and a soft deadline d_i that defines its SLA. Egress and inter-cloud transfer costs are charged whenever a task's data must move between providers.

3.2. Optimization Objectives

The scheduler maps each ready task to a (provider, VM) pair, and we optimize four objectives simultaneously. Makespan is the completion time of the last task in a job. Monetary cost aggregates compute and data-transfer charges across providers. Energy is the integral of VM power over execution time. The SLA-violation rate is the fraction of tasks that complete after their deadline. Let M, C, E, and S denote the normalized makespan, cost, energy, and SLA-violation rate, each scaled to the interval [0, 1] so that the four quantities are commensurable. The scheduling goal is to minimize the weighted objective J in Equation (3):

Here the weight vector w = (w_M, w_C, w_E, w_S) encodes the relative importance of the four objectives and is constrained to sum to one, so that J is a convex combination of the normalized objectives. Intuitively, J is a single scalar “cost-of-a-decision” score in which a lower value is always better; the constraint w_M + w_C + w_E + w_S = 1 keeps the four objectives on a common scale and prevents any objective from dominating merely because of its magnitude. Crucially, w is not fixed: Section 4.4 makes it a function of the observed utilization regime, so the agent shifts emphasis toward SLA protection during peaks and toward cost during troughs. To ensure fair aggregation, all objectives (M, C, E, and S) are normalized to the range [0,1]using min-max normalization before computing the weighted objective in Equation (3). This prevents any metric from dominating the reward due to differences in scale.

3.3. Markov Decision Process Formulation

We cast scheduling as a Markov decision process (MDP) defined by the tuple (S, A, T, R, γ), where S is the state space, A the action space, T the transition dynamics, R the reward function, and γ ∈ (0, 1) the discount factor. At each decision step the agent observes a state, selects an action (a placement), receives a reward, and transitions to the next state. The components are defined as follows.

State: The state s_t concatenates four blocks: (i) a queue block summarizing the ready tasks—their lengths, data sizes, and remaining slack to deadline; (ii) a resource block giving, for each (provider, VM) pair, the current utilization, availability a(p,v,t), and price c(p,v,t); (iii) a forecast block f_t, the predicted aggregate demand over the next H steps produced by the Attention-BiLSTM module; and (iv) a context block containing the current normalized utilization regime u_t. The forecast block f_t is what distinguishes SAMC-DRL from reactive DRL schedulers.

Action: An action a_t assigns the head-of-queue ready task to one available (provider, VM) pair. Unavailable pairs those with a(p,v,t) = 0 are masked out of the policy's action distribution so that the agent never selects an infeasible placement.

Transition: The environment advances simulated time, executes the placed tasks, updates utilization, prices, and availability, and admits newly arrived tasks. Transitions are stochastic because arrivals, prices, and availability evolve jointly from the production trace and from the price and availability models.

Reward: The immediate reward rt is the negative of the incremental weighted objective of Equation (3), plus a potential-based shaping term that penalizes deadline risk and idle energy, as given in Equations (4)–(6). The discount factor γ trades off immediate against long-term return, encouraging the agent to value future SLA compliance rather than only the current placement.

We now give the reward in closed form. Let J(s) denote the weighted objective of Equation (3) evaluated on the system states, using the adaptive weight vector w_t of Section 4.4 rather than a fixed w. The immediate reward is the negative increment of that objective plus a shaping term F:

Defining the reward on the increment rather than on the absolute level of J makes it invariant to the scale of an episode: the agent is credited only for the change its own placement produced, and not for having inherited a favourable state. The shaping term F in Equation (4) is defined in Equation (5) as a potential difference in the sense of Ng et al. [33], a form that leaves the optimal policy of the underlying MDP unchanged under the conditions stated below:

Here, the potential Φ in Equation (6) aggregates the two operational quantities whose consequences would otherwise become observable only at the end of an episode, namely accumulated deadline risk and idle energy:

Here Q(s) is the set of tasks resident in the queue in states; slack_i(s) = d_i − (t + ĉ_i) is the time remaining to the soft deadline d_i of task i once its estimated completion time ĉ_i is taken into account, so the ratio slack_i/d_i falls toward zero as a task approaches its deadline and the corresponding term saturates at one; V(s) is the set of currently provisioned VMs; P_idle(p,v) is the idle power draw of VM type v at provider p, normalized by the maximum idle draw in the catalog; and 𝟙 is the indicator function. Both sums are divided by their cardinalities so that the magnitude of the shaping signal does not grow with workload scale, which keeps the same coefficients valid from 100 to 4000 concurrent tasks. The coefficients η_d = 0.5 and η_e = 0.2 were selected by grid search on the validation split and are the only two shaping hyperparameters.

The practical effect of the first term is that an action which raises imminent deadline risk is penalized at the moment it is taken rather than many steps later when the violation finally materializes, which is what makes credit assignment tractable over episodes of several thousand placements; the second term discourages the agent from hoarding capacity after a predicted surge fails to materialize. Because F is a potential difference, its discounted contribution over an episode of T steps telescopes to γ^TΦ(s_T) − Φ(s₀). The policy-invariance result of Ng et al. [33] holds when Φ depends on the state alone, the shaping term uses the same discount γ as the return, and the potential of terminal states is fixed at zero (or the horizon is infinite); under these conditions, adding F leaves the optimal policy of the MDP unchanged. In our finite-horizon episodes Φ(s_T) is not forced to zero, so the terminal term γ^TΦ(s_T) is bounded but can depend on the policy. Moreover, the guarantee concerns the exact optimal policy of a fixed MDP; it does not extend formally to a PPO policy learned with function approximation, nor to the time-varying weights w_t of the base reward. We therefore claim only that shaping is intended to accelerate credit assignment and preserves optimality under the stated conditions, not that it provably leaves the converged PPO solution unaltered. All quantities entering Equations (4)–(6) are computed from realized simulator outcomes, never from forecasts, so a forecast error can bias which action the policy prefers but can never corrupt the learning signal itself — a property we return to in Section 4.6.

4.1. Workload-Forecasting Module (Attention-BiLSTM)

The forecaster predicts the aggregate resource-demand vector for the next H time steps from a sliding window of the L most recent observations. We use a bidirectional LSTM so that the encoder captures temporal context from both directions within the window, followed by a temporal attention layer that learns to weight the most informative time steps before a dense projection produces the multi-step forecast. Bidirectional recurrence and attention are well-motivated by prior multivariate and attention-based cloud forecasters [8, 9] and the design is deliberately lightweight so that inference latency stays small relative to the scheduling interval [33].

Inputs are multivariate—CPU, memory, disk I/O, and network—following evidence that multivariate forecasts outperform univariate ones on cloud traces [8]. Features are min-max normalized, and the window length L and horizon H are tuned on a validation split (Section 5). The module is trained offline by minimizing the mean squared error between predicted and observed demand, then frozen and queried online; periodic re-training accommodates concept drift, consistent with uncertainty-aware and transfer-learning practice [10]. Let h_k denote the BiLSTM hidden state at step k within the window. The attention weights α_k, the attention-weighted context vector z, and the forecast ŷ are computed as in Equations (7–9).

The attention layer first assigns each hidden state h_k a relevance score, then normalizes those scores into weights α_k with a softmax so that they sum to one. The context vector z is the resulting weighted average of the hidden states, which concentrates the network's attention on the time steps most predictive of the near future—for example, the onset of a recurring diurnal ramp [34]. A final linear projection with weight matrix W and bias b maps z to the H-step demand forecast y, which becomes the forecast block f_t in the scheduler state. Figure 2 shows the resulting module architecture end to end.

Figure 2.

4.2. Scheduling Agent (Proximal Policy Optimization)

The scheduler is an actor-critic agent trained with PPO [20], chosen for its stability and sample efficiency relative to value-based methods such as DQN [35] and to A3C [36] in large, high-variance action spaces. The actor and critic share an encoder that ingests the fused state s_t; the actor outputs a masked categorical distribution over feasible (provider, VM) placements, and the critic estimates the state value. Advantages Â_t are computed with generalized advantage estimation (GAE) [37]. PPO maximizes the clipped surrogate objective in Equation (10), with the probability ratio defined in Equation (11):

Here r_t(θ) is the probability ratio between the new policy π_θ and the old policy π_θold for the action taken, Â_t is the estimated advantage (how much better the action was than the policy's average behavior in that state), and ϵ is the clipping range. The intuition is conservative improvement: when an action looks advantageous, PPO increases its probability, but the clip term caps how far the ratio r_t(θ) may move in a single update, and the outer minimum keeps the more pessimistic of the clipped and unclipped estimates. This prevents a single large, destabilizing policy step—the failure mode that makes value-based agents brittle in high-variance scheduling environments. Action masking guarantees that placements onto unavailable VMs receive zero probability, which both enforces feasibility and accelerates learning by shrinking the effective action space. The agent is trained across episodes sampled from the production traces; each episode replays a slice of the trace through the multi-cloud simulator. Figure 3 shows the scheduling agent: the four state blocks, the shared encoder, the masked actor and the critic, and the path by which the adaptive reward controller closes the loop.

Figure 3.

4.3. State Fusion

The state builder concatenates the queue, resource, forecast, and context blocks and passes them through a shared multilayer perceptron (MLP) encoder. By placing the forecast vector f_t alongside the instantaneous resource block, the encoder can learn to reserve capacity ahead of a predicted surge or release capacity ahead of a predicted trough, turning the forecast into anticipatory placement rather than after-the-fact correction.

4.4. Adaptive Multi-Objective Reward

Rather than fixing the weight vector w, the adaptive reward controller computes it online from the current utilization regime u_t, a normalized measure derived from the forecast and the observed load. When predicted utilization is high, the controller increases w_M and w_S to protect deadlines; when predicted utilization is low, it increases w_C and w_E to harvest cost and energy savings. Equation (12) gives the weight schedule, where σ is a smooth gating (sigmoid) function, δ_peak and δ_trough are the weight adjustments applied in the peak and trough regimes, and the base weights w₀ are tuned on validation data:

The gate σ(u_t) moves smoothly between 0 and 1 as predicted utilization rises, so the weights blend continuously between the cost-leaning trough setting and the SLA-leaning peak setting rather than switching abruptly; the final normalization restores the unit-sum constraint. This makes the trade-off itself adaptive, which the ablation in Section 6 shows is responsible for a meaningful share of the overall improvement. Algorithm 1 summarizes end-to-end training.

The procedure first trains and freezes the forecaster (so the scheduler sees a stable predictive signal), then optimizes the PPO policy over many trace-replayed episodes. Within each episode, every decision step queries the forecaster, builds the fused state, computes the adaptive weights, samples a feasible (masked) action, and stores the resulting transition; after each rollout, GAE advantages are computed and the actor and critic are updated.

To enable reproducibility, the adaptive reward controller computes the utilization-dependent weight vector explicitly. Let u_t ∈ [0,1]denote the normalized utilization estimated from the combination of observed resource utilization and forecasted workload demand. The utilization gating function in Equation (13) is defined as

During high-utilization periods, the scheduler prioritizes performance and SLA protection using δ_{peak} = [+0.15, −0.05, −0.05, +0.15] whereas during low-utilization periods it prioritizes resource efficiency through δ_{trougℎ} = [−0.10, +0.10, +0.10, −0.10]. The adaptive weight vector in Equation (14) is then computed as followed by normalization:

Thus, when the predicted utilization exceeds approximately 70%, the scheduler automatically shifts emphasis toward makespan minimization and SLA preservation. Conversely, under light workload conditions, the controller increases the relative importance of energy efficiency and monetary cost reduction. This smooth transition prevents abrupt policy changes and allows the reinforcement learning agent to adapt continuously to evolving workload conditions.

Both constants of the gating function were selected on the validation split alone and never on the test period. The threshold u_tℎ = 0.70 was not chosen a priori: we measured, on the validation portion of each trace, the empirical relationship between aggregate utilization and queueing delay, and located the knee at which mean waiting time begins to grow super-linearly — 0.71 on the Google slice and 0.68 on the Alibaba slice — and adopted the rounded common value 0.70, which is also consistent with the utilization headroom conventionally maintained in production capacity planning. The steepness k = 10 fixes the width of the transition band: since the logistic gate moves from 0.12 to 0.88 over an interval of roughly 4/k in u_t, k = 10 yields a band of about ±0.10 utilization around the threshold, wide enough that the weights change smoothly across a diurnal ramp yet narrow enough that the peak regime is actually reached during a surge. Both values were then confirmed by an explicit grid search over h ∈ {0.55, …, 0.85} and k ∈ {2, 5, 10, 20, 50} with three seeds per cell (35 configurations); Section 6.8 reports the sweep and shows that performance is flat to within one percentage point of composite utility over h ∈ [0.65, 0.75] and k ∈ [5, 20], so the framework is not delicately tuned, while both extremes degrade for interpretable reasons — a very small k never commits to either regime, and a very large k reduces the gate to a hard switch whose weight vector chatters when utilization hovers near the threshold.

4.5. Computational Complexity Analysis

We analyze the cost of a single online scheduling decision and of the full offline training procedure. Let L be the forecaster window length, H the forecast horizon, u the BiLSTM hidden width, D the dimension of the fused state, m the width of the shared MLP encoder, and |A| = P × V the number of candidate (provider, VM) pairs, where V is the number of VM types per provider.

Forecaster inference: A bidirectional LSTM pass over the window performs O (L ·u²) work, since each of the L steps multiplies state vectors by recurrent weight matrices of size O(u²). Temporal attention over the L hidden states adds O(L ·u), and the dense projection to the H-step output adds O (u ·H). Forecaster inference per query is therefore O(L ·u²+ u ·H), dominated by the recurrent term.

Per-decision cost: Building the state and evaluating the shared encoder cost O (D ·m + m²); producing and masking the categorical policy over feasible placements costs O(|A|). A single scheduling decision thus costs O(L ·u²+ D ·m + m²+ |A|). All terms are independent of the total number of queued tasks, because the agent decides one head-of-queue task per step; this is the structural reason the measured scheduling latency in Section 6.5 grows only sub-linearly with workload size, and action masking keeps the |A| term small in practice.

Per-episode and training cost: An episode that schedules N tasks perform N decisions, costing O(N · (L ·u²+ D ·m + m²+ |A|)). PPO performs K optimization epochs over each rollout, so training over E episodes costs O(E ·K ·N ·(D ·m + m²)), with peak memory O (B ·D) for a rollout buffer of B transitions. Training is therefore linear in the number of episodes and in tasks per episode, and the forecaster's one-time offline training adds only a constant prefactor. These bounds confirm that the framework's online decision cost is modest and bounded by the action-space size, which is exactly what makes it viable for production-scale serving.

4.6. Forecast Drift Monitoring and Online Re-training

Algorithm 1 freezes the forecaster after offline pre-training so that the PPO agent optimizes against a stationary predictive signal; were the forecaster's parameters to change while the policy is being learned, the state distribution would be non-stationary for reasons unrelated to the environment and policy optimization would be markedly less stable. Freezing is therefore a training-time device and not a deployment-time assumption: a predictor frozen indefinitely would indeed decay as the workload distribution drifts [38,39,40]. This subsection specifies what the framework does at serving time, and Section 6.9 measures it over a 30-day rolling deployment.

Three mechanisms bound the effect of forecast error, and they are deliberately ordered from structural to active. First, the forecast is advisory rather than authoritative. The block f_t enters the state alongside the measured queue and resource blocks, and, as noted in Section 3.3, the reward of Equations (2)–(4) is computed exclusively from realized outcomes. A degraded forecaster therefore shifts the action the policy prefers but cannot corrupt the learning signal, and the measured blocks remain a sufficient statistic for the reactive part of the problem. The ablation in Section 6.6 quantifies the resulting floor: the forecast-free variant still attains 88% of the composite utility of the full system, so the worst case of an uninformative forecaster is the loss of the proactive advantage, not a divergent policy.

Second, a drift detector runs online, as formalized in Equations (15) and (16). We track an exponentially weighted absolute percentage error of the deployed forecaster over a sliding window of W = 288 intervals (24 hours at the 5-minute cadence), where ŷ_t₋_h is the prediction issued H steps earlier for the value realized at t and ε guards against division by a near-zero demand:

A drift alarm is raised when this statistic exceeds the error the same model achieved on the validation split by a multiplicative factor τ_drift, which makes the test scale-free so that a single threshold applies to both traces:

Third, an alarm triggers a two-stage response, the first stage of which is given in Equation (17). Immediately, the forecast is down-weighted by a confidence factor before it enters the state, so that the policy degrades continuously toward its reactive variant instead of acting on a signal that is no longer trustworthy:

A forecaster whose error has doubled therefore contributes a forecast of half the magnitude, and the agent, which has seen small-magnitude forecast blocks during quiet training intervals, falls back on the measured blocks without any discontinuity in the policy. Concurrently, an incremental re-training job fine-tunes the forecaster on the most recent seven days of trace for a small number of epochs and hot-swaps its weights once its error on a held-out recent window falls below that of the deployed model; the PPO policy is not re-trained at this point, because the confidence factor has already absorbed the transient and the policy input distribution is unchanged in shape. In our configuration this re-training takes 4.2 minutes on a single GPU and is triggered at most once per day, which is negligible against a 5-minute scheduling cadence and can be run off the critical path. Section 6.9 shows that under this policy the SLA-violation rate stays within 0.4 percentage points of its first-day value across a 30-day deployment, whereas a forecaster left frozen and ungated drifts to a 6.9% violation rate by day 30, which exceeds the 6.1% achieved by the forecast-free PPO baseline on the nominal test slice. Left unmonitored, a stale forecast therefore not only forfeits the proactive advantage but begins to mislead the policy slightly, which is precisely the failure that drift monitoring and confidence gating are designed to prevent.

5.1. Datasets

In Table 2 We use two widely cited public production traces. The Google Borg cluster trace [39, 41] records task arrivals, resource requests, machine attributes, and usage for a large Borg cell and is a standard benchmark for cloud-scheduling research. The Alibaba cluster trace [42] captures co-located online services and batch jobs, including DAG-structured batch workloads and pronounced burstiness; the 2023 GPU release further documents heterogeneous accelerator workloads [43]. Together, the two traces span diverse demand profiles, from steady latency-sensitive services to bursty batch jobs. Table 2 summarizes the trace slices used. For forecasting, each trace is resampled into fixed intervals of aggregated multivariate utilization, and we use a 70/15/15 chronological split for training, validation, and test, ensuring that evaluation always occurs on time periods the forecaster never observed during training. For scheduling, episodes are sampled from the test period so that the reported scheduling results reflect generalization to unseen demand.

Because the scheduling action operates on DAG-structured jobs, the structural complexity of the workload determines how much room a scheduler actually has to differentiate itself, and we therefore characterize it explicitly. Table 3 reports the structural statistics of the sampled job populations, computed over the same slices used in all experiments [44]. Both traces are dominated by small jobs with a long tail of large ones — the median job holds five (Google) or seven (Alibaba) tasks while the 95th percentile holds 27 and 41 respectively — which is consistent with published characterizations of these workloads [45, 46]. The Alibaba slice is the structurally harder of the two: its jobs are deeper (mean depth 4.7 against 3.4), more highly branched (mean out-degree 2.6 against 1.9), and its critical path accounts for a smaller fraction of total work (0.24 against 0.31), meaning more of its work is parallelizable and more of its makespan is decided by how well the scheduler respects precedence. Section 6.10 uses this stratification to examine the effect of DAG depth on scheduling quality directly.

5.2. Simulation Environment

Scheduling is evaluated in an extended CloudSim simulator [44], configured with three emulated providers whose VM catalogs differ in MIPS, energy coefficients, and pricing. On-demand and spot prices follow provider-style models with diurnal variation and stochastic spot fluctuation, and VM availability is toggled by a Markov process to emulate capacity constraints and preemption. Inter-cloud data-transfer costs are charged on cross-provider placements. This configuration lets us measure cost, energy, makespan, and SLA violations under realistic multi-cloud dynamics while remaining fully reproducible. To ensure reproducibility, the simulated multi-cloud environment consists of three providers representing AWS-like, Azure-like, and GCP-like infrastructures. Each provider offers heterogeneous VM configurations with different processing capacities, pricing models, and energy characteristics. The VM availability is modeled using a two-state Markov process to emulate real-world resource fluctuations and spot-instance interruptions. Table 4 summarizes the provider configurations used in all experiments.

Equation 18 is energy consumption of each VM is computed using a linear utilization-dependent model:

In Equation (19) P_max rresponds to the energy coefficient listed in Table 4. Provider pricing follows a dynamic model incorporating diurnal fluctuations and spot-market variability in Equation (20)

5.3. Baselines

We compare SAMC-DRL against three families of schedulers: heuristics (Round-Robin, Min-Min), metaheuristics (GA, PSO), and DRL agents (DQN [42] and PPO without forecasting [20]). For the forecasting module we additionally compare against ARIMA [29], support vector regression (SVR), LSTM [26], GRU [31], and a vanilla Transformer [32]. All learning-based methods are tuned on the validation split and evaluated on the held-out test period.

5.4. Hyperparameters and Metrics

The PPO agent uses a clip parameter ϵ = 0.2, discount γ = 0.99, GAE parameter λ = 0.95, learning rate 3 × 10⁻⁴, and minibatch updates over rollouts of 2048 steps for 600 episodes. The Attention-BiLSTM uses two bidirectional layers of 128 units, a window L = 24 and horizon H = 6, dropout 0.2, the Adam optimizer, and early stopping on the validation loss. Forecasting is scored with MAPE and root-mean-square error (RMSE); scheduling is scored with makespan, normalized energy, normalized cost, SLA-violation rate, and scheduling latency. Table 5 lists the principal hyperparameters.

6.1. Forecasting Accuracy

Figure 4 overlays the Attention-BiLSTM forecast on the actual aggregate CPU demand for one test week of the Google trace; the predictor tracks both the diurnal cycle and the sharper bursts with only a small lag. Figure 5 and Table 6 compare forecasters: the Attention-BiLSTM attains a MAPE of 5.8% and an RMSE of 7.6, improving on the vanilla Transformer (7.1% MAPE) and clearly outperforming classical ARIMA (14.8%) and SVR (12.6%). The gains over plain LSTM and GRU confirm that bidirectional context and attention add measurable value on these traces, consistent with prior multivariate-attention forecasting results [8, 9]. To assess robustness, forecasting experiments were repeated across multiple test windows, yielding a MAPE of 5.8 ± 0.3%and an RMSE of 7.6 ± 0.4, indicating consistent predictive performance across varying workload conditions. Table 6 additionally reports the 95% confidence interval of each MAPE estimate over ten test windows and a Diebold–Mariano test of equal predictive accuracy against the proposed model [45]; the null of equal accuracy is rejected for every baseline.

Figure 4.

Figure 5.

6.2. Training Convergence

Figure 6 plots the normalized average reward against training episodes. SAMC-DRL converges faster and to a higher plateau than PPO-without-forecast, DQN, and A3C. The forecast block reduces variance in the early phase because the agent is no longer surprised by predictable surges, and action masking removes wasted exploration of infeasible placements. DQN and A3C plateau lower, consistent with PPO's reported stability advantages in high-variance scheduling action spaces [2, 20].

Figure 6.

6.3. Scheduling Quality

Figures 7–10 and Table 7 compare end-to-end scheduling quality across all baselines. SAMC-DRL achieves the lowest makespan (812 s vs. 915 s for PPO and 980 s for DQN), the lowest normalized energy (63% vs. 72% for PPO), the lowest normalized cost (61% vs. 71% for PPO), and the lowest SLA-violation rate (3.2% vs. 6.1% for PPO and 8.4% for DQN). Relative to PPO-without-forecast, these correspond to an 11.3% makespan reduction, a 12.5% energy reduction, a 14.1% cost reduction, and a near-halving of SLA violations—gains attributable to anticipatory placement and the adaptive reward. Relative to heuristics such as Round-Robin, the improvements exceed 40% on both makespan and cost. The magnitude and direction of these gains are consistent with recent multi-objective DRL scheduling reports, which document double-digit makespan and energy reductions over heuristic and metaheuristic baselines [3, 14, 23].

Figure 7.

Figure 8.

Figure 9.

Figure 10.

6.4. Statistical Significance Analysis

To verify that the observed performance improvements are not caused by random variation during training, all experiments were repeated using ten independent random seeds. For each performance metric, the mean and standard deviation were computed across runs. In addition, a paired two-tailed t-test was performed between SAMC-DRL and the two strongest DRL baselines (PPO and DQN) at a significance level of α = 0.05. Because ten paired observations are too few to rest on a normality assumption, we additionally report the exact Wilcoxon signed-rank test, Holm–Bonferroni correction across the eight comparisons, the 95% confidence interval of each mean difference, and Cohen's d as an effect-size measure. Per-metric means and standard deviations appear in Table 8 and the significance results in Table 9.

The results in Table 9 indicate that the improvements achieved by SAMC-DRL are statistically significant across all evaluated metrics. The low standard deviations further demonstrate the stability of the proposed framework across different training runs. These findings confirm that the observed gains originate from the forecasting-aware scheduling policy and adaptive reward mechanism rather than stochastic variation in the reinforcement learning process.

Two features of Table 9 deserve comment. The Holm–Bonferroni procedure was applied across all eight comparisons jointly, so the reported significance survives correction for multiple testing rather than resting on eight independent tests. The effect sizes are large in Cohen's terms (d > 4 throughout), which follows from the small run-to-run dispersion visible in Table 8 rather than from an unusually large absolute difference; with standard deviations of one to two units against mean differences of nine to sixteen, the seed-level distributions of SAMC-DRL and the baselines do not overlap on any metric. The exact Wilcoxon signed-rank test attains its minimum attainable two-sided p-value of 0.002 at n = 10 for every comparison, meaning SAMC-DRL was superior on every seed, which is the strongest statement the non-parametric test can make at this sample size and confirms that the parametric result is not an artefact of the normality assumption. The forecasting comparison in Table 6 is tested separately with the Diebold–Mariano test for equal predictive accuracy [45], which rejects equality against every baseline, most narrowly against the vanilla Transformer (p = 0.004).

6.5. Scalability

Figure 11 reports the average scheduling latency as the number of concurrent tasks grows from 100 to 4000. SAMC-DRL scales sub-linearly and remains the fastest learned scheduler at every scale (15.1 ms at 4000 tasks vs. 22.4 ms for PPO and 33.8 ms for DQN), because action masking keeps the effective decision space small and the forecaster's inference cost is amortized across many placements. This empirical trend matches the per-decision complexity derived in Section 4.5, in which the dominant terms are independent of the queue length, and indicates that the framework is viable for online use at production scale.

Figure 11.

To validate the theoretical complexity analysis presented in Section 4.5, we measured the average online scheduling latency under increasing workload sizes. The experiments were conducted using the same hardware and software configuration employed for the main evaluation, and the reported values represent the average over ten independent runs.

The results demonstrate a gradual increase in scheduling latency as workload size grows as depicted in Table 10. However, the growth rate remains substantially lower than linear with respect to the total number of queued tasks. This behavior is consistent with the complexity analysis in Section 4.5, where the dominant computational terms depend primarily on the state representation, neural-network inference, and feasible action space rather than the overall queue length. Furthermore, the action-masking mechanism reduces the number of candidate scheduling actions that must be evaluated at each decision step, while the workload forecast is reused across multiple scheduling decisions within the same interval. Consequently, the forecasting module introduces only a small computational overhead while providing significant scheduling benefits. These results confirm that the proposed SAMC-DRL framework remains computationally practical for large-scale multi-cloud environments and supports real-time scheduling decisions even under workloads containing several thousand concurrent tasks.

6.6. Ablation Study

Table 11 isolates the contribution of each component by removing it and measuring the resulting composite utility, a normalized blend of the four objectives. Removing the forecast block is the most damaging change, dropping composite utility to 88% of the full system, which confirms that anticipatory placement is the single largest driver of the gains. Removing the adaptive reward drops utility to 91%, and removing the explicit multi-cloud cost model drops it to 86%. The DQN baseline reaches only 74%. The two proposed ideas—the forecasting-aware state and the adaptive reward—are therefore each individually significant and jointly responsible for the bulk of SAMC-DRL's advantage.

6.7. Robustness to Irregular and Adversarial Burst Workloads

The framework's central assumption is that near-future demand is partially predictable, so the honest test of it is what happens when that assumption is deliberately broken. We therefore constructed three stress regimes on top of the Alibaba test slice and re-evaluated the trained policy on each without any retraining. The flash-crowd regime injects an unannounced eight-fold step in arrival rate sustained for 30 minutes, emulating a viral-traffic event. The aperiodic regime replaces the diurnal arrival process with a self-exciting Hawkes process whose bursts carry no periodic component at all, removing the recurring structure that a sequence model relies on. The adversarial regime is the strongest: it is a white-box attack in which an adversary with full access to the frozen forecaster places surges precisely where the model predicts troughs, maximizing prediction error by construction. Table 12 and Figure 12 report the outcome over ten seeds; regimes are also scored by the forecaster's MAPE under that regime, which is the natural explanatory variable.

Figure 12.

Three observations follow. First, the proactive advantage is largest where forecasting is imperfect but informative: under the flash crowd the forecaster's error more than doubles to 11.4% MAPE, yet SAMCDRL halves the SLA-violation rate relative to PPO (6.4% against 12.7%), because a partially correct forecast of a surge is still enough to pre-warm capacity at the cheaper provider before the queue builds. Second, the advantage decays monotonically as forecast quality falls: it is 6.3 percentage points under the flash crowd, 4.1 under aperiodic Hawkes bursts (13.9% MAPE), and only 0.7 under the adversarial regime (24.6% MAPE). Third, and most important for a deployment argument, the advantage decays toward the reactive baseline and not past it. Even under a white-box adversary designed specifically to defeat the forecaster, SAMC-DRL does not perform worse than the forecast-free PPO scheduler on any metric; the forecast block simply stops contributing information, and the policy falls back on the measured queue and resource blocks [47]. Enabling the confidence gating of Equation (12) recovers part of the residual loss under the adversarial regime (SLA violations 9.3% against 10.1%, makespan 1042 s against 1078 s) by shrinking a forecast the drift detector has already identified as unreliable. We regard graceful degradation to the reactive baseline, rather than immunity to adversarial workloads, as the realistic claim for a forecasting-coupled scheduler.

6.8. Sensitivity to the Adaptive-Reward Hyperparameters

Section 4.4 fixes the gate threshold at u_th= 0.70 and the steepness at k = 10. To show that these values are neither arbitrary nor delicately tuned, we swept each over the validation split with three seeds per configuration, holding the other fixed, and scored the composite utility used in the ablation together with the SLA-violation rate and normalized cost. Table 13 and Figure 13 report the sweep. The response surface is unimodal and flat near its optimum: composite utility stays within one percentage point of its best value over h ∈ {0.65, 0.75} and within 1.4 points over k ∈ {5, 20}, so a deployment that mis-specifies either parameter by a modest margin loses little.

Figure 13.

The two parameters fail in different and interpretable ways. Lowering h makes the scheduler enter the SLA-protective regime earlier, which buys a marginally better violation rate (2.9% at h = 0.55 against 3.2% at h = 0.70) at a disproportionate cost penalty (67% against 61% normalized cost), because capacity is bought during periods that never become peaks; raising h delays protection until the queue has already built, and violations rise steeply to 5.2% at h = 0.85 while cost barely improves. The selected value sits at the point where the marginal cost of earlier protection begins to exceed its marginal SLA benefit, and it coincides with the utilization knee measured independently on the validation traces (0.71 and 0.68 for the Google and Alibaba slices). For the steepness, k = 2 produces a gate so gradual that the weight vector never fully reaches either regime and the adaptive mechanism is effectively diluted (95.3% utility), while k = 50 reduces the gate to a hard switch: when utilization hovers near the threshold the weights chatter between the peak and trough settings within a single episode, which injects non-stationarity into the reward and costs both stability and utility (96.4%). The chosen value of 10 corresponds to a transition band of roughly ±0.10 utilization, which is the range over which a diurnal ramp in these traces typically develops.

6.9. Long-Horizon Forecast Drift and Re-training

To test whether a forecaster pre-trained offline and frozen can sustain scheduling quality over a long deployment, we extended the Alibaba evaluation into a 30-day rolling simulation in which the arrival process is allowed to evolve: the diurnal amplitude, the batch/service mix and the mean job size are perturbed by a slow random walk calibrated so that the aggregate demand distribution at day 30 differs materially from that at day 1. We compare two deployment policies, both starting from the same trained artefacts: a forecaster frozen indefinitely with no monitoring, and the drift-aware policy of Section 4.6 with confidence gating and alarm-triggered incremental re-training. Table 14 and Figure 14 report five-day windows.

Figure 14.

The frozen forecaster degrades steadily, from 5.8% to 12.5% MAPE by the final window, and the scheduling policy tracks that decay: the SLA-violation rate rises from 3.2% to 6.9%. The shape of that decay is the substantive result. By days 21–25 the violation rate (5.8%) has approached 6.1%, the rate achieved by PPO without forecasting on the nominal test slice, and in the final window (6.9%) it exceeds that reference by 0.8 percentage points. The ungated frozen forecaster therefore does not merely lose the proactive advantage: once its error has roughly doubled, the stale forecast block begins to inject mildly misleading structure into the state, and the policy ends slightly below the reactive baseline. This is the failure mode that the confidence gating of Equation (11) is designed to remove, since it shrinks an unreliable forecast so that the policy falls back on the measured blocks (Section 4.6). Because the 6.1% reference was measured on the undrifted slice rather than re-evaluated under the same drift, this comparison is indicative; nonetheless, the degradation is gradual and bounded rather than divergent, so prediction-error accumulation erodes scheduling quality without destabilizing the policy. Under the drift-aware policy, MAPE is held between 5.8% and 6.5% by three re-training events over the 30 days (days 9, 17 and 24), each costing 4.2 GPU-minutes, and the violation rate stays within 0.4 percentage points of its day-1 value, at 3.6% in the final window. The residual 0.4-point gap is the transient cost of detecting drift before correcting it, since the detector by construction reacts only after the error has grown by 50%. We note that the PPO policy itself was never re-trained during this experiment; a production deployment would additionally schedule periodic policy refreshes, which we expect to close part of the residual gap.

6.10. Effect of DAG Structure and the Critical Path

The action defined in Section 3.3 assigns the head-of-queue ready task to a (provider, VM) pair, which raises a legitimate question: if the ready queue is ordered by arrival, the agent's placement decisions are made in an order that ignores which tasks lie on a job's critical path, and the influence of DAG structure on makespan would be partly masked by that ordering rather than addressed by the policy. We separate the two design choices explicitly [48]. The action head decides where a task runs; the ready-queue ordering decides which ready task is decided next; and only the second is sensitive to the critical path. In SAMC-DRL the ready queue is ordered by upward rank in the sense of HEFT [49] — the length of the longest remaining path from a task to any exit task, computed with mean execution costs across the VM catalog — so the head-of-queue task is by construction the ready task with the greatest remaining critical-path contribution. Upward ranks are computed once per job at admission in O(|V| + |E|), which for the DAG sizes in Table 3 costs 0.3 ms per job on average and is included in the latencies reported in Section 6.5.

Table 15 isolates the contribution of that ordering by replacing it with FIFO and with random ordering while holding the learned placement policy fixed, and stratifies the workload by DAG depth using the statistics of Table 3. On shallow jobs (depth ≤ 3), which make up the bulk of both traces, ordering matters little: the three variants lie within 25 s of one another, because a shallow DAG offers few opportunities to delay a critical task. On deep jobs (depth > 6) the gap widens sharply, from 878 s under upward-rank ordering to 967 s under FIFO and 1063 s under random ordering — a 9.2% and 17.4% makespan penalty respectively — confirming both that the critical path is a real effect on this workload and that critical-path-aware ordering, not the placement head alone, is what captures it. The SLA-violation rate follows the same pattern, since deep jobs are the ones whose deadlines are most sensitive to a mis-ordered predecessor. We note the corollary honestly: on a workload consisting entirely of deep, highly branched DAGs, a policy that also learns the ordering — for instance a graph-neural-network policy over the ready frontier [23], [50] — would likely extract more than a fixed priority rule does, and we identify this as the most promising extension of the action space.

6.11. Discussion

Three findings stand out. First, forecasting and scheduling are complementary: a good forecast is useful only if the scheduler can act on it, and the largest gains appear precisely when the forecast block and the PPO policy are trained together. Second, adapting the reward to the operating regime resolves the tension between cost and SLA that fixed-weight schedulers cannot escape—the system spends aggressively on capacity only when a surge is predicted and economizes otherwise. Third, the multi-cloud cost-and-availability model matters: ignoring per-provider price and availability sacrifices a meaningful fraction of the savings, which aligns with cost-efficiency analyses of real hybrid workloads [21, 31]. Taken together, these results corroborate the broader literature's claim that DRL is well suited to dynamic, non-stationary cloud environments [1, 2], while directly addressing the reactivity gap that limits prior reactive DRL schedulers. The revised evaluation adds three qualifications to this picture. The advantage of forecasting is a function of forecast quality and degrades gracefully to the reactive baseline rather than below it, both under adversarial bursts (Section 6.7) and under long-horizon drift (Section 6.9); the adaptive-reward gate is robust to moderate mis-specification of its threshold and steepness (Section 6.8); and the makespan benefit is concentrated in structurally deep DAGs, where critical-path-aware ready-queue ordering rather than placement alone accounts for most of the gain (Section 6.10).