Loading [a11y]/accessibility-menu.js
Optimization and scheduling algorithm for data intensive workflows in distributed data mining architecture | IEEE Conference Publication | IEEE Xplore

Optimization and scheduling algorithm for data intensive workflows in distributed data mining architecture


Abstract:

New Grid and cloud solutions for distributed data mining and data processing are needed for execution of data intensive workflows. In contrast of the standard workflows, ...Show More

Abstract:

New Grid and cloud solutions for distributed data mining and data processing are needed for execution of data intensive workflows. In contrast of the standard workflows, in which data between the jobs are exchanged in the form of files and the jobs are finished when they process the input data, data intensive workflows receive data organized in blocks which are streamed on inputs, analyze the data and produce stream output. Each job is active for a long period of time and can receive new data. In our previous research works we proposed the Open Grid Service Architecture for Data Mining (OGSA-DM), which is capable of executing data intensive workflows. According to our analysis, the current algorithms for scheduling workflows can't be applied on data intensive workflows because they produce unsatisfactory results and can't guarantee optimal solution. In this paper we propose new optimization and scheduling algorithm which is developed on the advantages of data intensive workflows. In several experiments we've shown that our proposed algorithm works and gives satisfactory results.
Date of Conference: 06-08 July 2017
Date Added to IEEE Xplore: 17 August 2017
ISBN Information:
Conference Location: Ohrid, Macedonia

Contact IEEE to Subscribe

References

References is not available for this document.