`
shonelau
  • 浏览: 17025 次
  • 性别: Icon_minigender_1
  • 来自: 深圳
社区版块
存档分类
最新评论

SCGI 源码解析

阅读更多
最近在看unix高级环境编程,对服务器环境下,fork处理请求和select处理请求,一直没有直观的认识,于是重读了一下scgi的源代码,以理解其实际应用.scgi对请求的处理是基于进程的,但是,它采用了一个进程池的方式去处理客户端的链接:

第一步:

 

在你的程序端,初始化:SCGIServer类,开启一个SCGI服务,等待web 服务器(假设是apache)将相应的请求转发过来

 

SCGIServer类的__init__方法如下:

 

    def __init__(self, handler_class=SCGIHandler, host="", port=DEFAULT_PORT,
                 max_children=5):
        self.handler_class = handler_class
        self.host = host
        self.port = port
        self.max_children = max_children
        self.children = []
        self.spawn_child()
        self.restart = 0

self.handler_class : 来自apache的请求 ,最终都会分配给子进程,由这个类的实例负责处理,并返回给apache

self.host 绑定到scgi server 的socket IP   (现在:Apache 是发出请求的client的端,scgi是对请求处理的服务器端)

self.port 绑定的端口号

self.max_children : 进程池里的最大子进程个数

self.children: 保存在进程池里的子进程的详细信息。里面放的是: Child实例。这个类很简单,保存了子进程的pid(进程id),和与子进程进行握手用的fd(文件描述符)

self.spawn_child(): 在实例化SCGIServer时,会调用该方法在进程池里先生成一个子进程。

self.restart:  若系统产生了SIGHUP则状态改变. (注1: 不知道这个SIGHUP什么时候发生?)

 

self.spawn_child()方法:

def spawn_child(self, conn=None):
        parent_fd, child_fd = passfd.socketpair(socket.AF_UNIX,
                                                socket.SOCK_STREAM)
        # make child fd non-blocking
        flags = fcntl.fcntl(child_fd, fcntl.F_GETFL, 0)
        fcntl.fcntl(child_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
        pid = os.fork()
        if pid == 0:
            if conn:
                conn.close() # in the midst of handling a request, close
                             # the connection in the child
            os.close(child_fd)
            self.handler_class(parent_fd).serve()
            sys.exit(0)
        else:
            os.close(parent_fd)
            self.children.append(Child(pid, child_fd))

1. socketpair在当前进程中生成一对采用socket文件进行通信的文件描述符对,

2. 把留在父进程中和子进程进行通信的child_fd设置为非阻塞

3. fork()调用

4_子:  1.子进程收到conn,但是现在没用,就关掉先。

4_子:  2.因为子进程通过parent_fd和父进程同步,因此关闭不用的child_fd.

4_子:  3.生成一个真正处理apache请求的hander对象,并永远的serve下去

4_子:  4. 若serve方法出错,则退出子进程,进行exit()调用的关闭各fd处理

4_父: 1. 因为父进程通过child_fd和子进程通信,因此关闭不用的parent_fd

4_父: 2.把新产生的子进程的pid及和子进程进行通信用的fd包装称一个Child对象,并放在列表self.children中备用

 

第二步:

现在SCGIServer对象已经生成,调用该对象的serve方法开始死循环等待apache的请求

    def get_listening_socket(self):
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        s.bind((self.host, self.port))
        return s

    def serve_on_socket(self, s):
        self.socket = s
        self.socket.listen(40)
        signal.signal(signal.SIGHUP, self.hup_signal)
        while 1:
            try:
                conn, addr = self.socket.accept()
                self.delegate_request(conn)
                conn.close()
            except socket.error, e:
                if e[0] != errno.EINTR:
                    raise  # something weird
            if self.restart:
                self.do_restart()

    def serve(self):
        self.serve_on_socket(self.get_listening_socket())
 

 

这段代码就是socket通信的常规代码,没什么可说的:

生成一个监听socket,对这个socket命名,设置SIGHUP的信号处理函数,在监听socket上等待apache的请求到来,请求建立后,生成一个和apache直连的新的socket连接,把这个新的socket传递给delegate_request方法,委托delegate_request去处理这个真正的请求.关闭这个新的socket(在这个socket真正传递给子进程处理前,conn.close()并不会真正关闭它),继续监听apache传递新的请求

 

第三步:

委托子进程处理apache的请求:

    def delegate_request(self, conn):
        """Pass a request fd to a child process to handle.  This method
        blocks if all the children are busy and we have reached the
        max_children limit."""

        # There lots of subtleties here.  First, we can't use the write
        # status of the pipes to the child since select will return true
        # if the buffer is not filled.  Instead, each child writes one
        # byte of data when it is ready for a request.  The normal case
        # is that a child is ready for a request.  We want that case to
        # be fast.  Also, we want to pass requests to the same child if
        # possible.  Finally, we need to gracefully handle children
        # dying at any time.

        # If no children are ready and we haven't reached max_children
        # then we want another child to be started without delay.
        timeout = 0

        while 1:
            fds = [child.fd for child in self.children if not child.closed]
            try:
                r, w, e = select.select(fds, [], [], timeout)
            except select.error, e:
                if e[0] == errno.EINTR:  # got a signal, try again
                    continue
                raise
            if r:
                # One or more children look like they are ready.  Sort
                # the file descriptions so that we keep preferring the
                # same child.
                child = None
                for child in self.children:
                    if not child.closed and child.fd in r:
                        break
                if child is None:
                    continue # no child found, should not get here

                # Try to read the single byte written by the child.
                # This can fail if the child died or the pipe really
                # wasn't ready (select returns a hint only).  The fd has
                # been made non-blocking by spawn_child.  If this fails
                # we fall through to the "reap_children" logic and will
                # retry the select call.
                try:
                    ready_byte = os.read(child.fd, 1)
                    if not ready_byte:
                        raise IOError # child died?
                    assert ready_byte == "1", repr(ready_byte)
                except socket.error, exc:
                    if exc[0]  == errno.EWOULDBLOCK:
                        pass # select was wrong
                    else:
                        raise
                except (OSError, IOError):
                    pass # child died?
                else:
                    # The byte was read okay, now we need to pass the fd
                    # of the request to the child.  This can also fail
                    # if the child died.  Again, if this fails we fall
                    # through to the "reap_children" logic and will
                    # retry the select call.
                    try:
                        passfd.sendfd(child.fd, conn.fileno())
                    except IOError, exc:
                        if exc.errno == errno.EPIPE:
                            pass # broken pipe, child died?
                        else:
                            raise
                    else:
                        # fd was apparently passed okay to the child.
                        # The child could die before completing the
                        # request but that's not our problem anymore.
                        return

            # didn't find any child, check if any died
            self.reap_children()

            # start more children if we haven't met max_children limit
            if len(self.children) < self.max_children:
                self.spawn_child(conn)

            # Start blocking inside select.  We might have reached
            # max_children limit and they are all busy.
            timeout = 2
 

再转回第一步的:spawn_child方法,这个方法在子进程中开起了一个处理请求的服务循环,服务循环代码如下:

 

 

服务循环开始,就是通过parent_fd向父进程发一个"1"的字符,告诉父进程,我准备好了,然后在 passfd.recvfd(self.parent_fd) 阻塞, 等待父进程把apache请求的socket fd传递过来。

 

    def serve(self):
        while 1:
            try:
                os.write(self.parent_fd, "1") # indicates that child is ready
                fd = passfd.recvfd(self.parent_fd)
            except (IOError, OSError):
                # parent probably exited  (EPIPE comes thru as OSError)
                raise SystemExit
            conn = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
            # Make sure the socket is blocking.  Apparently, on FreeBSD the
            # socket is non-blocking.  I think that's an OS bug but I don't
            # have the resources to track it down.
            conn.setblocking(1)
            os.close(fd)
            self.handle_connection(conn)
 

再回到delegate_request:

 

通过select监控self.children中所有的未关闭的child的状态,若收到某个子进程的可读的状态,则通过保存child_fd把父进程中处理请求的套接字fd传递给该子进程。

 

然后就不管了,返回。

 

若中间出现错误着看看是否有子进程挂掉了,清理挂掉的子进程,看看进程池是不是满了,若不满就再spawn_child一下

然后再循环一次,看能不能处理.

 

不过这也验证了,我的另外一篇文章中,关于quixote的session的问题: SessionManager和每个子进程相关。不能保证每个请求都转向同一个子进程,因此在quixote中要实现session的持久化,以在多个子进程中共享。(或多个服务器中共享)

 

 

0
9
分享到:
评论

相关推荐

    Lighttpd源码分析

    ### Lighttpd源码分析 #### 一、Lighttpd简介 Lighttpd是一款轻量级的Web服务器软件,以其高效、稳定、安全的特点而受到广泛欢迎。它特别适合于高并发访问场景,并且占用系统资源相对较少。Lighttpd支持FastCGI、...

    lighttpd-1.4.20源代码

    源码中的`src`目录包含了所有核心组件,如`config`用于解析配置文件,`http`包含HTTP协议的实现,`mod_access`、`mod_auth`等模块则实现了各种功能扩展。 2. **配置解析** lighttpd使用灵活的配置文件,`lighttpd-...

    nginx安装及其配置详细教程.docx

    1. 把 nginx 源码包上传到 linux 系统上 2. 解压到 /usr/local 下面:# tar -xvf nginx-1.14.0.tar.gz -C /usr/local 3. 使用 configure 命令创建一个 makeFile 文件,执行下面的命令的时候,一定要进入到 nginx-...

    nginx安装手册教程

    --http-scgi-temp-path=/var/temp/nginx/scgi ``` 注意,配置中的临时文件路径需要在`/var`下创建`temp`和`nginx`子目录。 3. 执行`make`进行编译,然后使用`make install`进行安装。 4. 安装完成后,你可以...

    跟我学Nginx

    --http-scgi-temp-path=/var/temp/nginx/scgi ``` 注意:上述命令将临时文件路径设为`/var/temp/nginx`,需预先创建这些目录: ```bash mkdir -p /var/temp/nginx ``` **编译与安装** 编译Nginx: ```bash make...

    sph-lib:超过80个gpl3+许可的guile方案库

    为了获取具体的知识点,我们需要查看解压后的具体内容,包括源码文件、README文件、示例程序等。 总结: `sph-lib`是一个全面的Guile Scheme库集合,涵盖了许多功能,如网络服务(服务器)、日期时间处理、加密...

    centos7系统nginx部署前后端分离1

    2. **PCRE (Perl Compatible Regular Expressions)**:Nginx 的 HTTP 模块使用 PCRE 解析正则表达式。安装 PCRE 库和 pcre-devel,命令是 `yum install -y pcre pcre-devel`。 3. **ZLIB 安装**:ZLIB 库用于对 ...

    nginx配置教程

    - **GCC**(GNU Compiler Collection):用于编译Nginx源码。 - **Wget**:用于下载文件。 - **Automake/Autoconf**:自动化构建过程。 - **Libtool**:帮助跨平台编译。 - **Libxml2-devel/Libxslt-devel**:XML...

    CentOS 7下安装Nginx服务器

    --http-scgi-temp-path=/var/temp/nginx/scgi ``` 2. 编译并安装: ```bash make make install ``` 五、启动与管理Nginx 安装完成后,你可以找到Nginx的可执行文件: ```bash whereis nginx ``` 然后在Nginx...

    Linux上安装搭建Nginx服务器的详细步骤

    - 进入Nginx源码目录后,运行`./configure`脚本进行配置。根据实际需求设置各项参数,例如日志路径、工作目录等。示例命令如下: ```bash ./configure \ --prefix=/usr/local/nginx \ --pid-path=/var/run/...

Global site tag (gtag.js) - Google Analytics