Flow-Bench: A Dataset for Computational Workflow Anomaly Detection
A computational workflow, also known as workflow, consists of tasks that must
be executed in a specific order to attain a specific goal. Often, in fields
such as biology, chemistry, physics, and data science, among others, these
workflows are complex and are executed in large-scale, distributed, and
heterogeneous computing environments that are prone to failures and performance
degradations. Therefore, anomaly detection for workflows is an important
paradigm that aims to identify unexpected behavior or errors in workflow
execution. This crucial task to improve the reliability of workflow executions
must be assisted by machine learning-based techniques. However, such
application is limited, in large part, due to the lack of open datasets and
benchmarking. To address this gap, we make the following contributions in this
paper: (1) we systematically inject anomalies and collect raw execution logs
from workflows executing on distributed infrastructures; (2) we summarize the
statistics of new datasets, as well as a set of open datasets, and provide
insightful analyses; (3) we benchmark unsupervised anomaly detection techniques
by converting workflows into both tabular and graph-structured data. Our
findings allow us to examine the effectiveness and efficiencies of the
benchmark methods and identify potential research opportunities for improvement
and generalization. The dataset and benchmark code are available online with
MIT License for public usage.