欢迎关注微信公众号:石杉的架构笔记(id:shishan100)

我的新课**《C2C 电商系统微服务架构120天实战训练营》在公众号儒猿技术窝**上线了,感兴趣的同学,可以点击下方链接了解详情:
《C2C 电商系统微服务架构120天实战训练营》
在《亿级流量系统架构》系列第一阶段中,我们从零开始,讲述了一个大型数据平台的几个方面的构建,包括:
- 如何承载百亿级数据的存储挑战
- 如何承载设计高容错的分布式架构
- 如何设计高性能架构,使之能承载百亿级流量
- 如何设计高并发架构,能够支撑住每秒数十万的并发查询
- 如何设计全链路99.99%的高可用架构
好!架构演进到这个时候,系统是否无懈可击了呢?
当然不是!
自古以来,能够瓦解一个军队战斗力的,不仅有外力冲击,还有内部因素。
同样,对于咱们的亿级流量系统,外部的冲击我们抗住了,现在的考验,来自于系统自身。而首当其冲的,就是系统的可扩展性带来的严重挑战。。。
因此在第二阶段,咱们用了大量的篇幅,分为上中下三篇,详细的讨论了该架构在可扩展性方面的痛点和改进。
跨过了2018年,你是否还记得这些痛点以及针对的技术方案呢?
如果忘了,没关系,跟着本文,温故知新。笔者希望各位在重拾记忆的同时,能有新的收获,并且能把文中的某些技术方案在自己公司中实际落地实践。
同样,对于可扩展性方案的复习,也是为后面系统在其他方面的改进打下基础,这样大伙儿读后面的文章时,不至于因为中间知识的断层而一脸懵逼。。。
一
对亿级流量架构可扩展性的讨论,咱们分成了上中下三篇。其中上篇,开门见山,发现问题:实时计算平台与数据查询平台之间耦合严重,并造成了诸多痛点:
-
数据查询团队被动承担了本不该他们承担的高并发写入压力
-
数据库运维操作导致的线上系统性能剧烈抖动
-
实时计算平台团队因为自身写入机制的bug导致数据丢失,结果让数据查询团队来进行排查,典型的甩锅!
-
实时计算平台团队,竟然需要自己来实现双写一致性的保障机制,直接导致代码里混合了大量不属于自己团队业务逻辑的代码
-
数据查询平台做了分库分表的操作,需要实时计算平台team一起修改配置,一起测试部署上线
总之,这些痛点,导致的结果是两个团队的同学天天腻在一起,而且是被迫的。。。
对于上面这些系统痛点的成因,你还有印象吗?如果忘了,猛戳下面链接,先赶紧去复习一波吧,知道了病症,才好对症下药!
猛戳下方链接:
亿级流量系统架构之如何在上万并发场景下设计可扩展架构(上)?
二
好,通过了上篇文章,我们已经知道了系统耦合造成的各种痛点,真的很痛!
那么现在,就该针对这些痛点,对症下药。看看下面的内容,你还能记起吗?
-
类似于中医的“望闻问切”,解决问题的第一步,就是找到病因。而到咱们这里,解决耦合的第一步,则是清晰的划分出系统边界。
-
划分出边界之后,第二件事,当然就是解耦。如何解耦:利用消息中间件
-
好!现在我们引入了消息中间件解耦,你是否还记得上篇文章中的一个痛点:实时计算平台高并发写入时,数据查询平台要无辜承受高并发的写入压力
-
那我们引入了中间件之后,通过消息中间件进行削峰填谷,就能解决这个问题了,关于什么是削峰填谷,以及如何实行,还记得吗?
-
解耦过后,我们通过手动流量开关来配合数据库运维,直接自己团队的同学在某个低锋时段关闭流量开关,迅速完成数据库运维操作。这不又解决了一大痛点吗!
-
好处还不止这些,比如,我们引入中间件解耦之后,其他系统不也可以按需去MQ里,订阅实时计算平台计算好的数据吗!再不用看其他平台的脸色了
总体来讲,解耦之后,各个团队各司其职,不用天天被迫腻在一起。而没有了人为的各种干预,系统也运转的更加流畅高效。
关于这些针对性的解决方案,笔者建议大家再仔细看看,这都是真实线上生产总结出的经验,也许里面的某些方案能够帮到你!
猛戳下方链接:
亿级流量系统架构之如何在上万并发场景下设计可扩展架构(中)?
三
讲完了实际的落地方案,我们来到了亿级流量架构可扩展性的下篇。
在可扩展性中篇的讨论中,我们提到了解耦的好处之一,是可以实现消息的“Pub/Sub”模型,即不同平台都可以根据自身需要去订阅同一份数据。
那么下篇,我们讨论的主题就是基于消息中间件的“Pub/Sub”模型,并以RabbitMQ为例,详细阐述了其在代码层面的落地实践。
什么是exchange?默认的exchange是啥?如何绑定自己的队列到exchange上去消费?这些还记得吗?如果忘了,猛戳下面的链接,赶紧的回顾一下!
总体来讲,解耦之后,各个团队各司其职,不用天天被迫腻在一起。而没有了人为的各种干预,系统也运转的更加流畅高效。
关于这些针对性的解决方案,笔者建议大家再仔细看看,这都是真实线上生产总结出的经验,也许里面的某些方案能够帮到你!
猛戳下方链接:
亿级流量系统架构之如何在上万并发场景下设计可扩展架构(下)?
以上就是关于亿级流量可扩展性做的一个阶段性小结,重构之路漫无止境,且环环相扣。笔者希望通过这个总结,在咱们继续上路之前,打牢基础,加深理解,磨刀不误砍柴工。
如有收获,请帮忙转发,您的鼓励是作者最大的动力,谢谢!
一大波微服务、分布式、高并发、高可用的原创系列文章正在路上
欢迎扫描下方二维码,持续关注:
石杉的架构笔记(id:shishan100)
十余年BAT架构经验倾囊相授
**> **推荐阅读:**
>
> 1、[拜托!面试请不要再问我Spring Cloud底层原理](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5be13b83f265da6116393fc7)
>
> 2、[【双11狂欢的背后】微服务注册中心如何承载大型系统的千万级访问?](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5be3f8dcf265da613a5382ca)
>
> 3、[【性能优化之道】每秒上万并发下的Spring Cloud参数优化实战](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5be83e166fb9a049a7115580)
>
> 4、[微服务架构如何保障双11狂欢下的99.99%高可用](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5be99a68e51d4511a8090440)
>
> 5、[兄弟,用大白话告诉你小白都能听懂的Hadoop架构原理](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5beaf02ce51d457e90196069)
>
> 6、[大规模集群下Hadoop NameNode如何承载每秒上千次的高并发访问](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5bec278c5188253e64332c76)
>
> 7、【[性能优化的秘密】Hadoop如何将TB级大文件的上传性能优化上百倍](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Flink.juejin.im%2525252525252F%2525252525253Ftarget%2525252525253Dhttps%252525252525253A%252525252525252F%252525252525252Flink.juejin.im%252525252525252F%252525252525253Ftarget%252525252525253Dhttps%25252525252525253A%25252525252525252F%25252525252525252Fjuejin.im%25252525252525252Fpost%25252525252525252F5bed82a9e51d450f9461cfc7)
>
> [8、](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Fjuejin.im%2525252525252Fpost%2525252525252F5bf2c6b6e51d456693549af4)[拜托,面试请不要再问我TCC分布式事务的实现原理坑爹呀!](https://juejin.cn/post/6844903716089233416) [](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Fjuejin.im%2525252525252Fpost%2525252525252F5bf2c6b6e51d456693549af4)
>
> 9、[【坑爹呀!】最终一致性分布式事务如何保障实际生产中99.99%高可用?](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Fjuejin.im%2525252525252Fpost%2525252525252F5bf2c6b6e51d456693549af4)
>
> 10、[拜托,面试请不要再问我Redis分布式锁的实现原理!](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Fjuejin.im%2525252525252Fpost%2525252525252F5bf3f15851882526a643e207)
>
> **11、****[【眼前一亮!】看Hadoop底层算法如何优雅的将大规模集群性能提升10倍以上?](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Flink.juejin.im%25252525253Ftarget%25252525253Dhttps%2525252525253A%2525252525252F%2525252525252Fjuejin.im%2525252525252Fpost%2525252525252F5bf5396f51882509a768067e)**
>
> **12、****[亿级流量系统架构之如何支撑百亿级数据的存储与计算](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Flink.juejin.im%252525252F%252525253Ftarget%252525253Dhttps%25252525253A%25252525252F%25252525252Fjuejin.im%25252525252Fpost%25252525252F5bfab59fe51d4551584c7bcf)**
>
> 13、[亿级流量系统架构之如何设计高容错分布式计算系统](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Flink.juejin.im%2525253Ftarget%2525253Dhttps%252525253A%252525252F%252525252Fjuejin.im%252525252Fpost%252525252F5bfbeeb9f265da61407e9679)
>
> 14、[亿级流量系统架构之如何设计承载百亿流量的高性能架构](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Flink.juejin.im%25253Ftarget%25253Dhttps%2525253A%2525252F%2525252Fjuejin.im%2525252Fpost%2525252F5bfd2df1e51d4574b133dd3a)
>
> 15、[亿级流量系统架构之如何设计每秒十万查询的高并发架构](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Flink.juejin.im%252F%253Ftarget%253Dhttps%25253A%25252F%25252Fjuejin.im%25252Fpost%25252F5bfe771251882509a7681b3a)
>
> 16、[亿级流量系统架构之如何设计全链路99.99%高可用架构](https://link.juejin.im/?target=https%3A%2F%2Flink.juejin.im%3Ftarget%3Dhttps%253A%252F%252Fjuejin.im%252Fpost%252F5bffab686fb9a04a102f0022)
>
> 17、[七张图彻底讲清楚ZooKeeper分布式锁的实现原理](https://link.juejin.im/?target=https%3A%2F%2Fjuejin.im%2Fpost%2F5c01532ef265da61362232ed)
>
> 18、[大白话聊聊Java并发面试问题之volatile到底是什么?](https://juejin.cn/post/6844903730303746061)
>
> 19、[大白话聊聊Java并发面试问题之Java 8如何优化CAS性能?](https://juejin.cn/post/6844903731234865160)
>
> 20、[大白话聊聊Java并发面试问题之谈谈你对AQS的理解?](https://juejin.cn/post/6844903732061159437)
>
> 21、[大白话聊聊Java并发面试问题之公平锁与非公平锁是啥?](https://juejin.cn/post/6844903732883226637)
>
> 22、[大白话聊聊Java并发面试问题之微服务注册中心的读写锁优化](https://juejin.cn/post/6844903734267510798)
>
> 23、[互联网公司的面试官是如何360°无死角考察候选人的?(上篇)](https://juejin.cn/post/6844903734930046989)
>
> 24、[互联网公司面试官是如何360°无死角考察候选人的?(下篇)](https://juejin.cn/post/6844903735655661581)
>
> 25、[Java进阶面试系列之一:哥们,你们的系统架构中为什么要引入消息中间件?](https://juejin.cn/post/6844903736444207117)
>
> 26、[【Java进阶面试系列之二】:哥们,那你说说系统架构引入消息中间件有什么缺点?](https://juejin.cn/post/6844903737123667975)
>
> 27、[【行走的Offer收割机】记一位朋友斩获BAT技术专家Offer的面试经历](https://juejin.cn/post/6844903741213130765)
>
> 28、[【Java进阶面试系列之三】哥们,消息中间件在你们项目里是如何落地的?](https://juejin.cn/post/6844903742114906125)
>
> 29、[【Java进阶面试系列之四】扎心!线上服务宕机时,如何保证数据100%不丢失?](https://juejin.cn/post/6844903742928601095)
>
> 30、[一次JVM FullGC的背后,竟隐藏着惊心动魄的线上生产事故!](https://juejin.cn/post/6844903743712935944)
>
> 31、[【高并发优化实践】10倍请求压力来袭,你的系统会被击垮吗?](https://juejin.cn/post/6844903744514031624)
>
> 32、[【Java进阶面试系列之五】消息中间件集群崩溃,如何保证百万生产数据不丢失?](https://juejin.cn/post/6844903746636349453)
>
> 33、[亿级流量系统架构之如何在上万并发场景下设计可扩展架构(上)?](https://juejin.cn/post/6844903747345219598)
>
> 34、[亿级流量系统架构之如何在上万并发场景下设计可扩展架构(中)?](https://juejin.cn/post/6844903747903029261)
>
> 35、[亿级流量系统架构之如何在上万并发场景下设计可扩展架构(下)?](https://juejin.cn/post/6844903748565745678)**