Large scientific computing data-centers require a distributed dependability subsystem that can provide fault isolation and recovery and is capable of learning and predicting failures to improve the reliability of scientific workflows. This paper extends our previous work on the scientific workflow management systems by presenting a hierarchical dynamic workflow management system that tracks the state of job execution using timed state machines.