Resource Allocation for Distributed Processing

Choosing and configuring cluster resources for distributed data processing jobs can be a challenging task. Even expert users often do not fully understand system and workload dynamics, also just because there is usually no full information for all the factors that influence the performance of processing jobs. At the same time, configuring cluster resources so that jobs execute without significant bottlenecks and taking into account constraints for the execution time and utilized resources is important.
We, therefore, work on resource allocation methods and tools that take such requirements into account and utilize profiling, monitoring, and performance modeling to select adequate sets of resources.

Ongoing Research

We currently work on multiple topics in this area: