文章基本信息

标题：The Dynamic Replication Mechanism of HDFS Hot File based on Cloud Storage
本地全文：下载
作者：Mingyong Li ; Yan Ma ; Meilian Chen 等
期刊名称：International Journal of Security and Its Applications
印刷版ISSN：1738-9976
出版年度：2015
卷号：9
期号：8
页码：439-448
DOI：10.14257/ijsia.2015.9.8.39
出版社：SERSC
摘要：As an open source cloud storage scheme, HDFS is used by more and more large enterprises and researchers, and is actually applied to many cloud computing systems to deal with huge amounts of data. HDFS has many advantages, but there are some problems such as NameNode single point of failure, small file problem, hot issues, etc. For HDFS hot issues, this paper proposes a dynamic Replication mechanism of HDFS hot file based on cloud storage （ HDFS-DRM ） . The mechanism includes a Replication of the dynamic adjustment mechanism and adding, deleting duplicate node selection mechanism in two parts, by increasing the NameNode, BlockMap parameters, it records the number of reading requests of each file in a certain period of time to decide whether to increase or decrease the number of copies. The mechanism presents a replica placement method based on stage historical information and node load and selects the appropriate node to add or delete copies of documents to improve the utilization efficiency of the data node storage space effectively. Experimental results show that, HDFS - DRM in hot files case, compared to native HDFS file system access latency is significantly reduced, HDFS-DRM can solve the hot issues successfully.
关键词：Cloud storage; HDFS; hot files; dynamic Replication