NoSQL 与 流式计算

虽然业务量的增加，转筒的 ALTP 主键无法支撑这么大的业务量与实时性。

NoSQL

Redis

Mongo

InfluxData-分布式时序数据库

Opentsdb-分布式时序数据库

图数据库-Neo4j

感觉这些 DBA 更加擅长，我更加关心背后的思想和对于业务的帮助。

相关框架

MQ

Apache Kafka

apache-pulsar

流式计算

Apache Spark

  • 新生代

Storm

Jstrom

Trident-strom 基础之上开发

Twitter Heron-第二代流式计算

Apache Flink

Ali Blink-Flink 的改造

其他框架

  1. S4 Distributed Stream Computing Platform.?http://incubator.apache.org/s4/

  2. Spark Streaming. https://spark.apache.org/streaming/?

  3. Apache Samza. http://samza.incubator.apache.org

  4. Tyler Akidau, Alex Balikov, Kaya Bekiroglu, Slava Chernyak, Josh?Haberman, Reuven Lax, Sam McVeety, Daniel Mills, Paul?Nordstrom, Sam Whittle: MillWheel: Fault-Tolerant Stream?Processing at Internet Scale.?PVLDB 6(11): 1033-1044 (2013) 5.?Mohamed H. Ali, Badrish Chandramouli, Jonathan Goldstein,Roman Schindlauer: The Extensibility Framework in Microsoft?StreamInsight.?ICDE?2011: 1242-1253

  5. Rajagopal Ananthanarayanan, Venkatesh Basker, Sumit Das, Ashish?Gupta, Haifeng Jiang, Tianhao Qiu, Alexey Reznichenko, Deomid?Ryabkov, Manpreet Singh, Shivakumar Venkataraman: Photon:?Fault-tolerant and Scalable Joining of Continuous Data Streams.?SIGMOD?2013: 577-588

  6. DataTorrent.?https://www.datatorrent.com

  7. Simon Loesing, Martin Hentschel, Tim Kraska, Donald Kossmann:?Stormy: An Elastic and Highly Available Streaming Service in the?Cloud. EDBT/ICDT Workshops 2012: 55-60

参考资料

深度解析 Twitter Heron 大数据实时分析系统