Running DAG-based workflows using multiple processes, in Ruby
The Jongleur gem is a process scheduler and manager. It allows users to declare a number of executable tasks as Ruby classes, define precedence between those tasks and run each task as a separate process.
Jongleur is particularly useful for implementing workflows modeled as a DAG (Directed Acyclic Graph), such as ETL or Transactions, but can be also used to run multiple tasks in parallel or even sequential workflows where each task needs to run as a separate OS process.
Comments
Great tool! How is it compared to Gush ? Looks like Gush doesn’t support workflows as DAGs. What about other features? Like parallel execution of jobs. It would be great to have all features of Gush and DAGs.
Post a comment