Self-Adaptive Multi-Cloud Job Scheduling Using Deep Reinforcement Learning and Workload Forecasting

Approach	Method	Forecast-aware	Multi-objective	Multi-cloud	Adaptive reward
DeepRM [22]	Policy gradient	No	Partial	No	No
RL-MOTS [3]	DQN	No	Yes	No	No
Multi-cloud DRL [5]	DQN	No	Yes	Yes	No
GNN-PPO [23]	PPO + GNN	No	Yes	No	No
SLA-MORL [32]	Multi-obj RL	No	Yes	No	Partial
SAMC-DRL (Proposed Work)	PPO + BiLSTM	Yes	Yes	Yes	Yes

Architecture of the proposed SAMC-DRL framework for self-adaptive multi-cloud job scheduling.

Architecture of the Attention-BiLSTM workload-forecasting module.

Architecture of the self-adaptive DRL scheduling agent, showing state fusion, the masked actor–critic, and the adaptive multi-objective reward controller.

Algorithm 1: SAMC-DRL training procedure
Input: production traces τ; provider/VM catalogs; base weights w₀; PPO hyperparameters (ϵ, γ, λ).
Output: trained policy π_θ for online serving.
Initialization:
1: Pre-train Attention-BiLSTM forecaster on historical trace by minimizing MSE; freeze its weights.
2: Initialize PPO actor parameters θ and critic parameters φ; initialize rollout buffer ← ∅.
Training loop:
3: for each episode do
4: replay a trace slice through the multi-cloud simulator;
5: for each decision step t do
6: query forecaster for f_t; build fused state s_t = [queue, resource, f_t, u_t];
7: compute adaptive weights w_t from utilization regime u_t (Eq. 12);
8: mask infeasible (provider, VM) pairs; sample action a_t ~ π__θ (· \| s_t);
9: execute a_t; observe reward r_t (Eqs. 1,4,5) and next state s_t+1; store transition;
10: end for
11: compute GAE advantages Â_t from stored rewards and critic values;
12: update θ by maximizing the clipped surrogate (Eq. 10); update φ by value regression (Eq. 6);
13: until reward converges;
14: export policy π_θ for online serving.
15: Deployment: at each interval update the forecast-error statistic ē_t (Eq. 15); if alarm_t (Eq. 16) then down-weight f_t by c_t (Eq. 17) and trigger incremental forecaster re-training (Section 4.6).
Complexity: O(E · K · N · (D·m + m²)) training; O(L·u² + D·m + m² + \|A\|) per online decision (Section 4.5).

Table 2.

Dataset and workload configuration.

Property	Google Borg trace	Alibaba trace
Source	[39, 44]	[42, 43]
Duration sampled	7 days	8 days
Sampling interval	5 minutes	5 minutes
Forecast features	CPU, mem, disk, net	CPU, mem, disk, net
Tasks per episode	100–4000	100–4000
Workload character	Mixed services + batch	Co-located, bursty
Train/val/test split	70/15/15 (chronological)	70/15/15 (chronological)

Table 3.

Structural statistics of the DAG-structured workloads.

Structural property	Google Borg slice	Alibaba slice
Jobs sampled	12,400	18,900
Tasks per job (mean ± SD)	8.6 ± 11.2	12.4 ± 19.7
Tasks per job (median / 95th pct. / max)	5 / 27 / 214	7 / 41 / 396
DAG depth (mean / max)	3.4 / 11	4.7 / 17
Branching factor (mean out-degree)	1.9	2.6
Edge density \|E\| / \|V\|	1.12	1.38
Critical-path work / total work (mean)	0.31	0.24
Single-task (trivial) jobs	28%	19%

Table 4.

Multi-cloud provider configuration used in CloudSim.

Provider	VM Type	MIPS	RAM (GB)	Bandwidth (Mbps)	Energy Coefficient (W)	Base Price ($/hr)
Provider-1 (AWS-like)	Small	2000	4	1000	95	0.08
Provider-1 (AWS-like)	Medium	4000	8	1500	145	0.16
Large	8000	16	2000	220	0.32
Provider-2 (Azure-like)	Small	1800	4	900	90	0.07
Provider-2 (Azure-like)	Medium	3800	8	1400	140	0.15
Large	7600	16	1900	215	0.30
Provider-3 (GCP-like)	Small	2200	4	1100	100	0.09
Provider-3 (GCP-like)	Medium	4200	8	1600	150	0.17
Large	8500	16	2200	230	0.34

Table 5.

Principal hyperparameters.

Hyperparameter	Forecaster	Scheduler (PPO)
Hidden units / layers	128 × 2 (BiLSTM)	256 × 2 (MLP)
Window / horizon	L = 24 / H = 6	—
Learning rate	1 × 10⁻³	3 × 10⁻⁴
Discount / GAE λ	—	0.99 / 0.95
Clip ϵ	—	0.2
Optimizer	Adam	Adam
Training length	early-stopped	600 episodes
Reward shaping η_d / η_e	—	0.5 / 0.2
Gate threshold h / steepness k	—	0.70 / 10
Drift EWMA β / threshold τ_drift	0.98 / 1.5	—
Independent seeds per configuration	10	10

Attention-BiLSTM forecast vs. actual aggregate CPU demand (Google trace, one test week).

Forecasting accuracy across models (lower is better).

Table 6.

Forecasting accuracy comparison (test period).

Model	MAPE (%)	RMSE	95% CI of MAPE	DM test vs. ours (p)
ARIMA [29]	14.8	18.2	[14.1, 15.5]	< 0.001
SVR [35]	12.6	15.9	[12.0, 13.2]	< 0.001
LSTM [26]	8.9	11.4	[8.5, 9.3]	< 0.001
GRU [31]	8.3	10.8	[7.9, 8.7]	< 0.001
Transformer [49]	7.1	9.2	[6.8, 7.4]	0.004
Attention-BiLSTM (ours)	5.8	7.6	[5.5, 6.1]	—

Training convergence: normalized reward vs. episode.

Average makespan by scheduler (lower is better).

Normalized energy consumption by scheduler.

Table 7.

End-to-end scheduling comparison at 1000-task scale.

Scheduler	Makespan (s)	Energy (%)	Cost (%)	SLA viol. (%)
Round-Robin	1420	100	100	18.5
Min-Min	1185	92	90	14.2
GA	1090	86	85	11.8
PSO	1055	84	83	10.9
DQN [42]	980	78	77	8.4
PPO [20]	915	72	71	6.1
SAMC-DRL (ours)	812	63	61	3.2

Table 8.

Mean ± standard deviation of scheduling metrics over ten independent runs.

Method	Makespan (s)	Energy (%)	Cost (%)	SLA Violations (%)
DQN	980 ± 21	78 ± 2.3	77 ± 2.1	8.4 ± 0.6
PPO	915 ± 18	72 ± 1.9	71 ± 1.8	6.1 ± 0.4
SAMC-DRL	812 ± 14	63 ± 1.4	61 ± 1.3	3.2 ± 0.3

Table 9.

Statistical significance of SAMC-DRL improvements (Holm-corrected paired t-tests, exact Wilcoxon signed-rank tests, and effect sizes over ten independent runs).

Comparison	Metric	Mean difference (95% CI)	t (9)	p (Holm-adj.)	Wilcoxon (p)	Cohen's (d)
SAMC-DRL vs. PPO	Makespan (s)	−103.0 (−114.2, −91.8)	−20.8	< 0.001	0.002	6.6
SAMC-DRL vs. PPO	Energy (%)	−9.0 (−10.4, −7.6)	−14.5	< 0.001	0.002	4.6
Cost (%)	−10.0 (−11.3, −8.7)	−17.4	< 0.001	0.002	5.5
SLA violations (%)	−2.9 (−3.2, −2.6)	−21.9	< 0.001	0.002	6.9
SAMC-DRL vs. DQN	Makespan (s)	−168.0 (−183.0, −153.0)	−25.4	< 0.001	0.002	8.0
SAMC-DRL vs. DQN	Energy (%)	−15.0 (−16.7, −13.3)	−20.1	< 0.001	0.002	6.4
Cost (%)	−16.0 (−17.7, −14.3)	−21.4	< 0.001	0.002	6.8
SLA violations (%)	−5.2 (−5.6, −4.8)	−29.6	< 0.001	0.002	9.4

Table 10.

Empirical scheduling latency analysis.

Concurrent Tasks	DQN (ms)	PPO (ms)	SAMC-DRL (ms)
100	1.8	1.2	1.0
500	4.9	3.8	2.9
1000	9.1	6.8	5.0
2000	17.4	11.9	8.7
4000	33.8	22.4	15.1

Table 11.

Ablation summary (composite utility, % of full system).

Configuration	Composite utility (%)
Full SAMC-DRL	100
w/o workload forecasting	88
w/o adaptive reward	91
w/o multi-cloud cost model	86
DQN baseline	74

Table 12.

Robustness under irregular and adversarial burst regimes (1000-task episodes, mean ± SD over ten seeds).

Workload regime	Forecast MAPE (%)	Makespan, PPO (s)	Makespan, SAMC-DRL (s)	SLA viol., PPO (%)	SLA viol., SAMC-DRL (%)
Nominal test slice	5.8	915 ± 18	812 ± 14	6.1 ± 0.4	3.2 ± 0.3
Flash crowd (8× step, 30 min)	11.4	1187 ± 34	968 ± 27	12.7 ± 0.9	6.4 ± 0.6
Aperiodic Hawkes bursts	13.9	1121 ± 41	1004 ± 33	11.2 ± 1.1	7.1 ± 0.8
Adversarial anti-forecast	24.6	1096 ± 38	1078 ± 36	10.8 ± 1.0	10.1 ± 1.1
Adversarial + confidence gating (Eq. 11)	24.6	1096 ± 38	1042 ± 31	10.8 ± 1.0	9.3 ± 0.9

Scheduling quality under irregular and adversarial burst regimes: (a) SLA-violation rate by regime; (b) proactive advantage over PPO as a function of forecast error.

Table 13.

Sensitivity of SAMC-DRL to the adaptive-reward hyperparameters (validation split, three seeds per configuration; composite utility normalized to the selected setting).

Setting	Composite utility (%)	SLA violations (%)	Normalized cost (%)
Threshold sweep (k = 10)
h = 0.55	96.1	2.9	67
h = 0.60	97.8	3.0	65
h = 0.65	99.2	3.1	62
h = 0.70 (selected)	100.0	3.2	61
h = 0.75	99.4	3.6	60
h = 0.80	97.9	4.3	59
h = 0.85	95.6	5.2	59
Steepness sweep (h = 0.70)
k = 2	95.3	4.1	63
k = 5	98.6	3.5	62
k = 10 (selected)	100.0	3.2	61
k = 20	99.1	3.3	61
k = 50	96.4	3.9	64

Composite utility across the adaptive-reward hyperparameter sweep: (a) utilization threshold h at k = 10; (b) gate steepness k at h = 0.70.

Table 14.

Forecast error and scheduling quality over a 30-day rolling deployment with workload drift (Alibaba slice, five-day windows, ten seeds).

Deployment window (days)	MAPE, frozen (%)	MAPE, drift-aware (%)	SLA viol., frozen (%)	SLA viol., drift-aware (%)
1–5	5.8	5.8	3.2	3.2
6–10	6.4	5.9	3.5	3.3
11–15	7.6	6.0	4.1	3.3
16–20	9.1	6.2	4.9	3.4
21–25	10.8	6.3	5.8	3.5
26–30	12.5	6.5	6.9	3.6

Effect of workload drift over a 30-day deployment: (a) forecast MAPE; (b) SLA-violation rate, with the forecast-free PPO baseline shown for reference.

Table 15.

Effect of ready-queue ordering, stratified by DAG depth (Alibaba slice, 1000-task episodes, mean ± SD over ten seeds).

Ready-queue ordering	Makespan, shallow DAGs, depth ≤ 3 (s)	Makespan, deep DAGs, depth > 6 (s)	Overall makespan (s)	SLA violations (%)
Random	794 ± 16	1063 ± 31	901 ± 22	5.4 ± 0.5
FIFO (arrival order)	781 ± 15	967 ± 26	856 ± 19	4.3 ± 0.4
Upward rank / critical path (adopted)	769 ± 13	878 ± 21	812 ± 14	3.2 ± 0.3

References

Authors

Metrics

Articles in this issue

DOI: https://doi.org/10.2478/ias-2026-0024 | Journal eISSN: 1554-1029 | Journal ISSN: 1554-1010

Journal RSS Feed

Language: English

Page range: 501 - 526

Published on: Oct 27, 2026

Published by: Cerebration Science Publishing Co., Limited

In partnership with: Paradigm Publishing Services

Publication frequency: Volume open

Keywords:

Multi-cloud computing,

job scheduling,

deep reinforcement learning,

BiLSTM,

service-level agreement

Related subjects:

Computer sciences,

Fundamentals of computer sciences,

Theoretical computer sciences,

IT-security and cryptology

© 2026 Ashutosh Bhatt, Ambika Aggarwal, Khushboo Jain, published by Cerebration Science Publishing Co., Limited
This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License.

Volume 21 (2026): Issue 6 (August 2026)

Self-Adaptive Multi-Cloud Job Scheduling Using Deep Reinforcement Learning and Workload Forecasting

Figures & Tables

Table 1.

Figure 1.

Figure 2.

Figure 3.

Table 2.

Table 3.

Table 4.

Table 5.

Figure 4.

Figure 5.

Table 6.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Table 7.

Table 8.

Table 9.

Figure 11.

Table 10.

Table 11.

Table 12.

Figure 12.

Table 13.

Figure 13.

Table 14.

Figure 14.

Table 15.

Paradigm

My account