大数据环境下并行数据传输完整度控制方法

Parallel data transfer integrity control method in large data environment

摘要:

针对当前并行数据传输过程中节点拥塞严重、数据传输速率低、数据完整度低的问题,提出大数据环境下并行数据传输完整度控制方法。通过并行数据传输完整度控制原理,对传输中并行数据获取和传输拥塞度测量两大干扰因素加以分析。引用单服务窗混合制排队模型和栅格分析法分别对单节点和并行数据传输速率进行预控制,利用数据流优先级完成并行数据传输完整度控制方法的实现。实验结果表明,所提方法传输控制效果好,有效缓解了节点拥塞现象,提高了数据完整度。

Abstract:

Aiming at the problems of serious congestion, low data transmission rate and low data integrity in the process of parallel data transmission, a method of integrity control for parallel data transmission in large data environment is proposed. Based on the principle of integrity control of parallel data transmission, two interference factors of parallel data acquisition and transmission congestion measurement in transmission are analyzed. Reference single service window mixed queuing model and method were pre control of single node and parallel data transmission rate, using data flow priority to realize parallel data transmission method of complete control. The experimental results show that the proposed method has good transmission control effect, effectively alleviates the congestion of nodes and improves the data integrity.