高级检索

    孙怀英, 虞慧群, 范贵生, 陈丽琼. 大数据流计算环境下的低延时高可靠性的资源调度方法[J]. 华东理工大学学报(自然科学版), 2017, (6): 855-862. DOI: 10.14135/j.cnki.1006-3080.2017.06.016
    引用本文: 孙怀英, 虞慧群, 范贵生, 陈丽琼. 大数据流计算环境下的低延时高可靠性的资源调度方法[J]. 华东理工大学学报(自然科学版), 2017, (6): 855-862. DOI: 10.14135/j.cnki.1006-3080.2017.06.016
    SUN Huai-ying, YU Hui-qun, FAN Gui-sheng, CHEN Li-qiong. Low Latency and High-Reliability Resource Scheduling Method in Big Data Streaming Computing Environment[J]. Journal of East China University of Science and Technology, 2017, (6): 855-862. DOI: 10.14135/j.cnki.1006-3080.2017.06.016
    Citation: SUN Huai-ying, YU Hui-qun, FAN Gui-sheng, CHEN Li-qiong. Low Latency and High-Reliability Resource Scheduling Method in Big Data Streaming Computing Environment[J]. Journal of East China University of Science and Technology, 2017, (6): 855-862. DOI: 10.14135/j.cnki.1006-3080.2017.06.016

    大数据流计算环境下的低延时高可靠性的资源调度方法

    Low Latency and High-Reliability Resource Scheduling Method in Big Data Streaming Computing Environment

    • 摘要: 在大数据处理过程中,如何保证流数据处理的可靠性及实时性变得日益重要。本文使用数据流图(DSG)对大数据流应用过程进行描述,并将DSG表示为扩展的Petri网以便对数据流过程进行建模。提出了基于CPU利用率平均变化率的资源熵算法计算资源组可靠性,并根据资源熵算法提出了基于时间和可靠性的资源调度算法(TRS-SCHE)以获得高可靠性、低延时的资源调度方案。通过仿真实验,模拟实现soda交通大数据分析应用并进行资源的调度,验证了TRS-SCHE相比于Storm隔离调度算法在响应时间、请求失败率和算法时间复杂度方面的优势。

       

      Abstract: It is quite important for the application of big data to ensure the high-reliability and low-latency of processing the stream data.In this paper,the data stream graph (DSG) is used to describe and model the application process of big data streaming by taking DSG as an extended Petri net.And then,this paper proposes a computing algorithm of resource group reliability via the resource entropy that is based on the average changing rate of CPU utilization.Furthermore,a resource scheduling algorithm,termed as TRS-SCHE,is introduced to attain the high-reliability and low-latency.Finally,through simulation experiments of soda traffic big data analysis,it is shown that compared with the Storm isolated scheduling,the proposed TRS-SCHE scheduling algorithm has more advantages in response time,failure rate and the time complexity.

       

    /

    返回文章
    返回