ODCP: Optimizing Data Caching and Placement in Distributed File System Using Erasure Coding
ODCP: Optimizing Data Caching and Placement in Distributed File System Using Erasure Coding
Many current distributed file systems use erasure-coding based data redundancy techniques to improve the reliability of data storage. Such techniques can significantly improve the effective storage utilization. However, there are several drawbacks to the above techniques. Firstly, they introduce non-negligible computation overhead for decoding. Secondly, traditional data caching and placement strategies become less effective in such cases. To solve the above drawbacks, this paper proposes a new data cache allocation mechanism based on simulated annealing and a new data placement strategy based on convex optimization, which effectively reduces data block transmission delay and decoding delay. We have implemented the proposed data placement strategy in the real-world distributed file system Alluxio, and evaluated the performance of our strategy. Experiment results show that our strategy can significantly reduce the file read delay compared to traditional data placement strategies.
- Beijing University of Technology China (People's Republic of)
- Beihua University China (People's Republic of)
- Beihang University China (People's Republic of)
Decoding latency, Cache allocation strategy, Distributed file system, Data placement strategy, Erasure coding, [INFO] Computer Science [cs]
Decoding latency, Cache allocation strategy, Distributed file system, Data placement strategy, Erasure coding, [INFO] Computer Science [cs]
3 Research products, page 1 of 1
- 2007IsAmongTopNSimilarDocuments
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).0 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Average influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Average
