[Hazelcast 介绍]

What is Hazelcast

Hazelcast is a distributed computation and storage platform for consistently low-latency querying, aggregation and stateful computation against event streams and traditional data sources. It allows you to quickly build resource-efficient, real-time applications. You can deploy it at any scale from small edge devices to a large cluster of cloud instances.

A cluster of Hazelcast nodes share both the data storage and computational load which can dynamically scale up and down. When you add new nodes to the cluster, the data is automatically rebalanced across the cluster and currently running computational tasks (known as jobs) snapshot their state and scale with processing guarantees.

When to use Hazelcast

Hazelcast provides a platform that can handle multiple types of workloads for building responsive applications.

• Stateful data processing over streaming data or data at rest
• Querying streaming and batch data sources directly using SQL
• Ingesting data through a library of connectors and serving it using low-latency SQL queries
• Pushing updates to applications on events
• Low-latency queue-based or pub-sub messaging
• Fast access to contextual and transactional data via caching patterns such as read/write-through and write-behind)
• Distributed coordination for microservices
• Replicating data from one region to another or between data centers in the same region

Key Features

• Stateful and fault-tolerant data processing and querying over data streams and data at rest using SQL or dataflow API
• A comprehensive library of connectors such as Kafka, Hadoop, S3, RDBMS, JMS and many more
• Distributed messaging using pub-sub and queues
• Distributed, partitioned, queryable key-value store with event listeners, which can also be used to store contextual data for enriching event streams with low latency
• A production-ready Raft-implementation which allows lineralizable (CP) concurrency primitives such as distributed locks.
• Tight integration for deploying machine learning models with Python to a data processing pipeline
• Cloud-native, run everywhere architecture
• Zero-downtime operations with rolling upgrades
• At-least-once and exactly-once processing guarantees for stream processing pipelines
• Data replication between data centers and geographic regions using WAN
• Microsecond performance for key-value point lookups and pub-sub
• Unique data processing architecture results in 99.99% latency of under 10ms for streaming queries with millions of events per second (link to blog post).
• Client libraries in Java, Python, Node.js, .NET, C++ and Go

Operational Data Store

Hazelcast provides distributed in-memory data structures which are partitioned, replicated and queryable. One of the main use cases for Hazelcast is for storing a working set of data for fast querying and access.

The main data structure underlying Hazelcast, called IMap is a key-value store which has a rich set of features, including:

• Integration with data sources for one time or continuous ingestion
• Read-through and write-through caching patterns
• Indexing and querying through SQL
• Processing entries in place for atomic updates
• Expiring items automatically based on certain criteria like TTL or last access time
• Near cache for caching entries on the client
• Listeners for pushing changes to clients
• Data Replication between datacenters (Enterprise version only)
• Persistence of data on disk (Enterprise version only)

Hazelcast stores data in partitions, which are distributed to all the nodes. You can increase the storage capacity by adding additional nodes, and if one of the nodes go down, the data is restored automatically from the backup replicas.

什么是 Hazelcast

Hazelcast 是一个分布式计算和存储平台，用于针对事件流和传统数据源进行一致的低延迟查询、聚合和有状态计算。它允许您快速构建资源高效的实时应用程序。您可以以任何规模部署它，从小型边缘设备到大型云实例集群。

 

Hazelcast 节点集群共享数据存储和计算负载，可以动态扩展和缩减。当您向集群添加新节点时，数据会自动在集群中重新平衡，当前正在运行的计算任务（称为作业）快照它们的状态并通过处理保证进行扩展。

 

何时使用 Hazelcast

Hazelcast 提供了一个平台，可以处理多种类型的工作负载以构建响应式应用程序。

 

对流数据或静态数据进行有状态数据处理

直接使用 SQL 查询流式和批处理数据源

通过连接器库摄取数据并使用低延迟 SQL 查询为其提供服务

将更新推送到事件上的应用程序

基于低延迟队列或发布订阅消息

通过缓存模式（如读/写和后写）快速访问上下文和事务数据）

微服务的分布式协调

将数据从一个区域复制到另一个区域或在同一区域的数据中心之间复制

主要特征

使用 SQL 或数据流 API 对数据流和静态数据进行有状态和容错的数据处理和查询

一个综合性的连接器库，例如 Kafka、Hadoop、S3、RDBMS、JMS 等等

使用 pub-sub 和队列的分布式消息传递

具有事件侦听器的分布式、分区、可查询的键值存储，也可用于存储上下文数据，以低延迟丰富事件流

一种生产就绪的 Raft 实现，它允许线性化 (CP) 并发原语，例如分布式锁。

使用 Python 将机器学习模型部署到数据处理管道的紧密集成

云原生，无处不在的架构

通过滚动升级实现零停机操作

流处理管道的至少一次和恰好一次处理保证

使用 WAN 在数据中心和地理区域之间进行数据复制

键值点查找和发布订阅的微秒性能

独特的数据处理架构导致 99.99% 的延迟低于 10 毫秒，用于每秒数百万个事件的流式查询（链接到博客文章）。

Java、 Python、Node.js、.NET、C++和Go 中的客户端库

操作数据存储

Hazelcast 提供了分布式内存数据结构，这些数据结构是分区、复制和可查询的。Hazelcast 的一个主要用例是存储一组工作数据以进行快速查询和访问。

 

Hazelcast 底层的主要数据结构称为IMap键值存储，它具有丰富的功能集，包括：

 

与数据源集成 一次或连续摄取

读取和写入 缓存模式

通过SQL索引和查询

处理原子更新的条目

根据某些标准（如 TTL 或上次访问时间）自动过期项目

Near cache 用于缓存客户端上的条目

用于向客户端推送更改的侦听器

数据 中心之间的数据复制（仅限企业版）

磁盘上的数据持久性（仅限企业版）

Hazelcast 将数据存储在 分区中，分区分布到所有节点。您可以通过添加额外节点来增加存储容量，如果其中一个节点出现故障，数据会自动从备份副本中恢复。