Cloud · Machine Learning
Predicting Workload Instability in Cloud VMs
Rather than react to CPU thresholds after they're crossed, this project models VM telemetry to predict spikes before they happen, forecasting whether a cloud VM will exceed 85% CPU within 15 minutes. A Random Forest reached 99.66% accuracy, and SHAP analysis identified short-term workload momentum as the dominant signal.
- 99.66% Accuracy
- 0.9999 ROC AUC
- 0.964 F1
Stack Python / scikit-learn / XGBoost / SHAP
Problem
Cloud autoscaling is largely reactive: a VM has to cross a utilization threshold for some period before the platform migrates or resizes it, leaving a window where performance suffers and SLAs are at risk. Over-provisioning avoids that but wastes power and money. The question is whether the spike can be predicted instead.
Approach
Using the GWA-T-12 Bitbrains fastStorage trace (1,250 financial-workload VMs, 5-minute telemetry over a month), the work frames a point-in-time classification task: will this VM exceed 85% CPU within the next 15 minutes?
Rolling features are engineered over the raw signals, and Logistic Regression, Random Forest, and XGBoost are trained using chronological train/test splits to avoid temporal leakage. Models are compared on accuracy, precision, recall, F1, ROC-AUC, and PR-AUC, with PR-AUC weighted heavily given how rare the spike events are.
Results
- Random Forest: 99.66% accuracy, F1 0.964, ROC-AUC 0.9999, PR-AUC 0.999
- Tree-based models clearly outperformed logistic regression, which over-predicted spikes at lower precision
- SHAP analysis identified max CPU usage over the trailing 15- and 30-minute windows as the key predictors, indicating that workload momentum drives the need for additional resources
The findings support proactive scaling: acting on a forecast minutes ahead rather than waiting for a threshold breach.