Asian Journal of Information Technology

Year: 2016
Volume: 15
Issue: 15
Page No. 2645 - 2655

Efficient Map Reduce Task Scheduling and Micro-Partitioning Mechanism for Optimizing Large Data Analysis

Authors : A. Kamaleswari and P. Thangaraj

Abstract: Virtualization is a key technology to enable cloud computing. A vast mass of popular content is transferred frequently across network links in the cloud. In network-level Redundancy Elimination (RE) techniques, it minimizes traffic flow on bandwidth-constrained network paths by eliminating the transmission of repetitive byte sequences. In previous research, the protocol is independent of redundancy elimination which cannot eliminate duplicate packets from within arbitrary network flows. In emerged cloud we require a potent technique to improve the performance of network links in the face of frequent data. Our proposed research present the packet reduction technique is used for finding and removes the duplicate packets in a network environment. In addition, the data itself can be moreover large to store on a single machine. In order to reduce the time it takes to process the data and to have the storage space to store the data, we introduce a new approach called map reduce method. In this approach, it has to separate the workload among computers in a network. As an outcome, the performance of map reduce robustly determined on how equal it distributes this workload among the computer. In map reduce, workload allocation depends on the algorithm that separating the data. To avoid the issues of uneven distribution of data we use data sampling. By using the partitioning mechanism, the partitioning is done on the data which depends on how huge and representative the sample is and on how well the samples are examined. In addition to that we use partitioning methods to divide the workload into small tasks that are dynamically scheduled at runtime based on deadline. To improve the accuracy in scheduling, we propose a novel method called deadline constraints based task scheduling algorithm in map reduce. This method allows the user to specify a job’s deadline and attempts to formulate the job to be completed before the deadline. This method is simple and efficient systems with high-throughput, low-latency task schedulers and proficient data materialization.

How to cite this article:

A. Kamaleswari and P. Thangaraj, 2016. Efficient Map Reduce Task Scheduling and Micro-Partitioning Mechanism for Optimizing Large Data Analysis. Asian Journal of Information Technology, 15: 2645-2655.

Design and power by Medwell Web Development Team. © Medwell Publishing 2024 All Rights Reserved