“异地多活”分布式存储系统设计和实现

The design and implementation of a "Multi-active across data centers " Distributed Storage System

摘要:

随着互联网进入了大数据和云计算时代，分布式存储技术近年来受到了工业界和学术界的广泛关注。为了解决服务器压力过大、异地快速容灾和用户就近访问等问题，本文从异地多活、分布式CAP理论着手，分析了分布式存储的特性、应用场景和技术挑战，进而利用开源的Redis NoSql数据库，RabbitMQ消息队列等技术，搭建了一个满足最终一致性，可用性和分区容忍性的“异地多活”分布式存储系统。通过对系统的压力测试结果表明，基本可以满足工业级应用的吞吐量和一致性的要求。

Abstract:

As the Internet has entered the era of big data and cloud computing, distributed storage technology has received widespread attention in industry and academia in recent years. In order to solve the problems of excessive server pressure, rapid disaster recovery in remote locations and users' proximity access, this paper analyzes the characteristics, application scenarios and technical challenges of distributed storage based on the theory of CAP and “Multi-active across data centers”, and then uses the open source Redis cache、 RabbitMQ Message Queuing to setup a "Multi-active across data centers" distributed storage system that satisfies the ultimate consistency, availability and partition tolerance. Through the stress test results of the system, it can basically meet the throughput and consistency requirements of industrial applications.