Gearman provides a generic application framework to farm out work to other machines or processes that are better suited to do the work.
Gearman提供了能把任务分发(承包)给其他更适合做该任务的机器或程序来处理的一个通用应用程序框架。
It allows you to do work in parallel, to load balance processing, and to call functions between languages.
它允许你并行操作,负载均衡处理以及在不同的语言调用函数。
It can be used in a variety of applications, from high-availability web sites to the transport of database replication events.
它可以应用到诸如高性能网站,数据库复制事件传输等多类应用程序中。
In other words, it is the nervous system for how distributed processing communicates.
换句话说,它就是分布式处理赖以交互的神经系统。
A few strong points about Gearman:
以下是几个关于Geaman的要点:
- Open Source It’s free! (in both meanings of the word) Gearman has an active open source community that is easy to get involved with if you need help or want to contribute. Worried about licensing? Gearman is BSD.
- 开源 It's free! (包括free的两个意思,自由和免费) Gearman有一个活跃的开源社区,不论你用寻求帮助或共享资源,你都可以很容易的加入其中。担心版权问题吗?Gearman是BSD开源模式的。
- Multi-language - There are interfaces for a number of languages, and this list is growing. You also have the option to write heterogeneous applications with clients submitting work in one language and workers performing that work in another.
- 多语言支持 它有支持很多语言的应用接口,并且还在不断强化中。你可以使用一种语言编写不同的客户端应用程序,而处理端的处理平台用另一种语言来编写。
- Flexible - You are not tied to any specific design pattern. You can quickly put together distributed applications using any model you choose, one of those options being Map/Reduce.
- 灵活的 - 不必局限于某种设计模式。你可以用你选择的模型很快的把分散的程序合并起来,众多方法中有一种就是Map/Reduce。
- Fast - Gearman has a simple protocol and interface with an optimized, and threaded, server written in C/C++ to minimize your application overhead.
- 快速的 - Gearman有一个简单的协议和用C/C++编写的线程优化的服务来减少你应用程序的开销。
- Embeddable - Since Gearman is fast and lightweight, it is great for applications of all sizes. It is also easy to introduce into existing applications with minimal overhead.
- 可嵌入式的 - 因其高效和轻量级的,Gearman对各种体积的程序都是极适用的。它也可以积小的开销嵌入到已存在的程序中。
- No single point of failure - Gearman can not only help scale systems, but can do it in a fault tolerant way.
- No limits on message size - Gearman supports single messages up to 4gig in size. Need to do something bigger? No problem Gearman can chunk messages.
- 数据信息大小不限 - Gearman可以支持达4G的信息。需要更大吗?没问题,Gearman可以分块化。
- Worried about scaling? - Don’t worry about it with Gearman. Craig’s List, Tumblr, Yelp, Etsy,… discover what others have known for years.
How Does Gearman Work?
Gearman如何工作?
A Gearman powered application consists of three parts: a client, a worker, and a job server. The client is responsible for creating a job to be run and sending it to a job server. The job server will find a suitable worker that can run the job and forwards the job on. The worker performs the work requested by the client and sends a response to the client through the job server. Gearman provides client and worker APIs that your applications call to talk with the Gearman job server (also known as gearmand) so you don’t need to deal with networking or mapping of jobs. Internally, the gearman client and worker APIs communicate with the job server using TCP sockets. To explain how Gearman works in more detail, lets look at a simple application that will reverse the order of characters in a string. The example is given in PHP, although other APIs will look quite similar.
基于Gearman的程序包括三部分:一个client(发起者),一个worker(处理者),一个job server(分派者)。client端负责运行一个job并把它传送到server端。job server 端。job server端找到可以处理传送过来的job的worker,并且让worker处理该job。 worker处理client的请求并通过job server把处理结果返回给client。Gearman提供了可以和job server(也叫gearmand)交互的woker和client的API, 因而你不必处理网络或映射问题。在内部,client和worker的api通过TCP socket和job server进行交互。我们通过一个反转字串顺序的程序来加以解释Gearman的详细工作过程。该例子用PHP语言,其它的语言API也是大概如此。
We start off by writing a client application that is responsible for sending off the job and waiting for the result so it can print it out. It does this by using the Gearman client API to send some data associated with a function name, in this case the function reverse
. The code for this is (with error handling omitted for brevity):
我们从写一个小client程序开始,这个程序负责发送一job(任务)并等待结果然后把结果打印出来。通过使用Gearman的client API来发送一些关联函数名的数据, 在这个小例子中函数名为"reverse"。代码如下:
<?php // Reverse Client Code $client = new GearmanClient(); $client->addServer(); print $client->do("reverse", "Hello World!");
This code initializes a client class, configures it to use a job server with add_server
(no arguments means use 127.0.0.1
with the default port), and then tells the client API to run the reverse
function with the workload “Hello world!”. The function name and arguments are completely arbitrary as far as Gearman is concerned, so you could send any data structure that is appropriate for your application (text or binary). At this point the Gearman client API will package up the job into a Gearman protocol packet and send it to the job server to find an appropriate worker that can run the reverse
function. Let’s now look at the worker code:
这段代码初始化了一个client,用add_server设置它使用一个job server(没有参数表示用127.0.0.1默认端口),然后告诉client的API调用reverse函数来处理"Hello world!"数据。对Gearman来说,函数名和参数都都是任意的,所以你可以发送任何适合你应用程序的的数据结构(可以是文本或二进制数据)。在这里,Gearman client API 将把这个任务打包到一个Gearman协议包的job然后把它发送到job server来找个合适的可以运行reverse函数的worker。下面我们看看worker的代码:
<?php // Reverse Worker Code $worker = new GearmanWorker(); $worker->addServer(); $worker->addFunction("reverse", "my_reverse_function");
function my_reverse_function($job) { return strrev($job->workload()); } while ($worker->work());
This code defines a function my_reverse_function
that takes a string and returns the reverse of that string. It is used by a worker object to register a function named reverse
after it is setup to connect to the same local job server as the client. When the job server receives the job to be run, it looks at the list of workers who have registered the function name reverse
and forwards the job on to one of the free workers. The Gearman worker API then takes this request, runs the function my_reverse_function
, and sends the result of that function back through the job server to the client.
代码定义了一个my_reverse_function函数来接收字串并返回该字串的的倒序。当以client方式连接到本地的job server后,worker就把它注册到本身且命名为reverse。当job server接收到请求的job包, 它就搜索那些已经注册reverser函数的worker,然后选择空闲的worker来处理请求的job包。Gearman worker API获取job请求,根据job请求的参数运行my_reverse_function,然后通过job server把函数的处理结果发送给client端。
As you can see, the client and worker APIs (along with the job server) deal with the job management and network communication so you can focus on the application parts. There a few different ways you can run jobs in Gearman, including background for asynchronous processing and prioritized jobs. See the documentation available for the various APIs for details.
如你所见,client和worker的APIs(通过job server)已经处理工作(job)管理和网络交互, 这样你可以把重点放到程序开发上来. 有几种方式可以运行工作(job), 包括异步处理和分期作业. 详细请看相关文档.
How Is Gearman Useful?
用途
The reverse example above seems like a lot of work to run a function, but there are a number of ways this can be useful. The simplest answer is that you can use Gearman as an interface between a client and a worker written in different languages. If you want your PHP web application to call a function written in C, you could use the PHP client API with the C worker API, and stick a job server in the middle. Of course, there are more efficient ways of doing this (like writing a PHP extension in C), but you may want a PHP client and a Python worker, or perhaps a MySQL client and a Perl worker. You can mix and match any of the supported language interfaces easily, you just need all applications to be able to understand the workload being sent. Is your favorite language not supported yet? Get involved with the project, it’s probably fairly easy for either you or one of the existing Gearman developers to put a language wrapper on top of the C library.
上面倒序字串的例子好像很多工作是运行一个函数,但还有许多其他可以用法。一个最简单的例子就是可以把Gearman当作不同语言之间的接口。如果你想让你的PHP web程序调用C语言写的函数,你可以用PHP的client API写client端, 用C写worker端,定一个job server在两者之间。当然,还有更多不同的方式来实现类似的功能,如可以用一个PHP client端和一个Python的worker端,或者可能是MySQL client端和一个Perl worker端。。。。
相关推荐
Gearman中文手册,gearman手册chm,Gearman中文详解,分享gearman技术心得,主要是为了分享技术,所以不要大家的资源分。
Gearman是一种分布式任务队列系统,它允许应用程序在不同的服务器之间分发工作负载,从而实现负载均衡和异步处理。本篇文章将详细讲解如何在Linux环境中搭建Gearman,包括依赖库的安装和Gearman服务的配置。 首先,...
### Gearman文档与集群计算框架的关键知识点 #### 一、Gearman概述 **Gearman**是一种分布式任务调度框架,主要用于跨机器或者同一机器上的不同进程之间分发任务。它允许开发者将任务分解并分配给一组工作节点...
Gearman 是一个分布式任务队列系统,它允许应用程序在多台服务器之间分发工作负载,以提高处理效率和可扩展性。在这个“Gearman C# API和示例”项目中,我们将探讨如何使用C#语言与Gearman进行交互,以及如何在...
Gearman是一款开源的分布式任务队列系统,它允许应用程序在多台机器上分发工作负载,从而实现异步处理和负载均衡。这个系统的重点在于解耦任务的发起者和执行者,使得系统能够灵活地扩展并提高处理能力。在本讨论中...
在 Laravel 中集成 Gearman,你可以创建自定义的 Job 类来封装需要异步处理的任务,然后使用 Gearman 客户端将这些任务发送到 Gearman 服务器。下面是一些关键步骤: #### 安装扩展 首先,你需要通过 Composer ...
Gearman 是一个分布式任务队列系统,它允许应用程序在需要时分发工作负载,并在有可用资源时异步处理这些工作。在Java中,我们可以利用 Gearman 的客户端库来与 Gearman 服务器进行交互,执行分布式任务。本篇文章将...
Gearman中文手册,gearman手册chm,Gearman中文详解,分享gearman技术心得,主要是为了分享技术,所以不要大家的资源分。
GearMAN是一个高效的分布式任务框架,最初设计用于解决Web2.0环境中图片处理等耗时任务的快速分配和执行问题。它的名字来源于“Generic Worker Manager”,意在成为一个通用的任务分发工具。GearMAN通过将任务分配给...
**Laravel 开发与 Gearman RPC 整合详解** 在现代Web开发中,异步任务处理和分布式计算已经成为提升系统性能的关键技术。Laravel作为PHP的流行框架,提供了丰富的工具来支持这类需求。本篇文章将深入探讨如何在...
Gearman是一个分布式任务队列系统,它允许应用程序将工作分发到多个服务器或进程,以实现负载均衡和异步处理。在本实例中,我们关注的是如何在C++环境中,利用Visual Studio 2008(VS2008)在Windows平台上创建一个...
1. Gearman的介绍和应用场景: Gearman是一个工作负载分发服务器和库,允许将工作分散到多个机器或者机器上的多个核心上进行异步处理。它在Web应用程序中处理耗时的后台任务时尤其有用,如文件上传、邮件发送、图片...
java-gearman-service-0.6.6.zip 包,gearman分为3部分,client - server - worker,创建 java 版本的client和worker部分。 其实在gearman中,client和worker的编写不复杂,但是不同厂商提供的API是不大相同的,本...
Gearman 是一个分布式任务队列系统,它允许应用程序在需要时分发工作并异步处理结果。这个名为 "gearman-mysql-udf-0.6.tar.gz" 的压缩包包含了一个 Gearman 与 MySQL 结合使用的用户定义函数(UDF)插件,版本为 ...
java实现gearman的job实现的jar包,包括gearman server,client和work客户端API
gearman的java库有两个,一个是gearman service ,一个是gearman java,相比来说service版本更好用一些,并且网上的教程一般是用的这个版本。因此我打好了gearman service的包提供给需要的开发者使用。
Gearman是一个分布式任务队列系统,它允许应用程序在任何地方运行任务,而无需关心它们在哪里运行或如何运行。Gearman-Java是Java语言对Gearman的客户端库,它提供了与Gearman服务器通信的能力,用于分发工作负载...
该资源包括安装Gearman时可能涉及到的软件包 具体包括gearmand、gearman、php、gperf、libevent
安装gearman 如果没有mysql客户端,需要安装mysql客户端 yum install -y libevent-devel 上传gearman.zip,解压unzip gearman.zip rpm -ivh uuid-1.5.1-3.el5.x86_64.rpm rpm -ivh libgearman-1.1.8-2.el5.x86_64...
Gearman是一个分发任务的程序框架,可以用在各种场合,与Hadoop相比,Gearman更偏向于任务分发功能。它的 任务分布非常 简单,简单得可以只需要用脚本即可完成。Gearman最初用于LiveJournal的图片resize功能,由于...