Application of Deep Learning on Integrating Prediction, Provenance, and Optimization
Abstract
In this research, we investigated two approaches to detect job anomalies and/or contention for large scale computing efforts: 1. Preemptive job scheduling using binomial classification long short-term memory networks 2. Forecasting intra-node computing loads from the active jobs and additional job(s) For approach 1, we achieved a 14% improvement in computational resources utilization and an overall classification accuracy of 85% on real tasks executed in a High Energy Physics computing workflow. For this paper, we present the preliminary results used in second approach.
- Publication:
-
European Physical Journal Web of Conferences
- Pub Date:
- July 2019
- DOI:
- 10.1051/epjconf/201921406007
- Bibcode:
- 2019EPJWC.21406007S