Abstract:
Data transfer is a primary mechanism that can directly affect the overall performance of Big Data analytic systems. This is because most data are generated from several locations. It has been challenging to collect and transfer data among multiple storage regions. Using the traditional single-path transfer approaches is not efficient to serve several requirements of Big Data applications. In this dissertation, we propose an SDN-coordinated multipath transmission steering framework for Big Data transfer application. Multipath TCP protocol (MPTCP) and SDN architecture are mainly considered to design and develop our multipath transmission framework. To provide a practical routing solution, we propose a novel OpenFlow-Stats routing algorithm. In our algorithm, a new topology-pruning technique is applied, and the transmission paths are selected based on switch-port statistics. Our proposed framework is evaluated using the Mininet emulator and ONOS controller. The results show that our routing scheme can reduce the completion time of Big Data transfer up to 90% compared with the traditional routing scheme with disjoint paths and up to 35% compared with the previous work. Moreover, our proposed routing is more scalable than other previous works in that it can provide lower complexity and system overhead. The results show that our routing scheme produces 57% less overhead compared with the previous work.
