构建自动化mochiweb-nitrogen

1 在生产环境下，用inets实现的nitrgoen框架遇到并发量大的情况下会出现瓶颈，mochiweb在并发处理上优势要远大于inets，因此决定实现mochiweb-nitrogen框架。

2 准备工作，下载各应用

下载nitrogen源码http://github.com/nitrogen/nitrogen/tarball/v2.0.4

下载mochiweb源码https://github.com/mochi/mochiweb/tarball/master

下载rebar源码 https://github.com/basho/rebar/tarball/master

解压缩：

tar -zxvf mochi-mochiweb-v2.3.1-5-gd541e9a.tar.gz 
tar -zxvf nitrogen-nitrogen-v2.0.4-0-gc1cbc00.tar.gz 

tar -zxvf basho-rebar-2.0.0-15-gb1d06a4.tar.gz

重命名：

 

 

mv mochi-mochiweb-d541e9a/ mochiweb 
mv nitrogen-nitrogen-c1cbc00/ nitrogen

构建rebar

 

 

cd basho-rebar-b1d06a4/
./bootstrap

以上命令生产了一个rebar文件

 

 

3 编译mochiweb_nitrogen

1 拷贝mochiweb到apps目录下

2修改makefile文件，使nitrogen能够顺利编译通过

在nitrogen/Makefile 中，rel_inner:位置下添加
@(cd apps/mochiweb; make)

3 make rel_mochiweb

4 在rel/目录下生成了一个名为nitrogen的文件夹

5 执行

<span>	</span>./rel/nitrogen/bin/nitrogen console

Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:5] [kernel-poll:true]
Eshell V5.8  (abort with ^G)
(nitrogen@127.0.0.1)1> Starting Mochiweb Server (nitrogen) on 0.0.0.0:8000, root: './site/static'

证明nitrogen已经编译通过并启动成功。

4 以上编译已经通过，可以进行正常开发，但以前经常使用rebar构建项目，感觉这样开发很别扭，所以想把结构重新调整一下，如下：

5 新建自己的应用程序

mkdir myapp
cd myapp

把刚才生成的rebar文件考到此目录下

$ ./rebar create-app appid=myapp
==> myapp (create-app)
Writing src/myapp.app.src
Writing src/myapp_app.erl
Writing src/myapp_sup.erl
$ ls
<span style="background-color: rgb(51, 204, 0);" mce_style="background-color: #33cc00;">rebar</span>  src

mkdir deps rel site

把mochiweb,nitrgen/apps目录下的nitrogen, simple_bridge,nprocreg考到deps目录下

$ ls
mochiweb  nitrogen  nprocreg  simple_bridge

 

在myapp目录下新建文件Makefile，rebar.config,start.sh

Makefile文件中内容：

 

$ cat Makefile 
MYAPP_VERSION=1.0.1
all: get-deps compile
help:
<span>	</span>@echo
<span>	</span>@echo "Usage: "
<span>	</span>@echo "    ./make {compile|clean}"    
<span>	</span>@echo
<span>	</span>@echo "    ./make {rel|package}" 
<span>	</span>@echo
<span>	</span>@echo
get-deps:
<span>	</span>./rebar get-deps
compile: get-deps
<span>	</span>./rebar compile
clean:
<span>	</span>./rebar clean
# MOCHIWEB
rel: compile
<span>	</span>@rm -rf rel/myapp
<span>	</span>@rm -rf rel/reltool.config
<span>	</span>@ln rel/myapp.config rel/reltool.config
<span>	</span>@(make rel_inner)
<span>	</span>@mv rel/myapp rel/myapp-${MYA<pre name="code" class="plain">$ cat rebar.config
%% -*- mode: erlang;erlang-indent-level: 4;indent-tabs-mode: nil -*-
%% ex: ts=4 sw=4 ft=erlang et
%% Additional library directories to add to the code path
%%{lib_dirs, ["deps","apps"]}.
{sub_dirs, ["rel","site","deps"]}.
{require_otp_vsn, "R13B04|R14"}.
{cover_enabled, true}.
{erl_opts, [debug_info, fail_on_warning]}.
{deps, [
   %% mochiweb for JSON and header parsing
   mochiweb,
   %% webmachine for multipart content parsing
    nitrogen,
    nprocreg,
    simple_bridge
 ]}.</pre>

PP_VERSION}
@echo Generated a self-contained myapp project
@echo "Usage: rel/myapp-${MYAPP_VERSION}/bin/myapp {start|stop|restart|reboot|ping|console|attach}"

package:
mkdir -p ./builds
tar -C rel -c myapp-${MYAPP_VERSION} | gzip > ./builds/myapp-${MYAPP_VERSION}.tar.gz

# SHARED
rel_inner:
@(cd rel; ./rebar generate)
@(cd rel/myapp; make)
@printf "myapp Version:\n${MYAPP_VERSION}\n\n" > rel/myapp/BuildInfo.txt
@echo "Built On (uname -v):" >> rel/myapp/BuildInfo.txt
@uname -v >> rel/myapp/BuildInfo.txt
@rm -rf rel/reltool.config

$ cat rebar.config
%% -*- mode: erlang;erlang-indent-level: 4;indent-tabs-mode: nil -*-
%% ex: ts=4 sw=4 ft=erlang et
%% Additional library directories to add to the code path
%%{lib_dirs, ["deps","apps"]}.
{sub_dirs, ["rel","site","deps"]}.
{require_otp_vsn, "R13B04|R14"}.
{cover_enabled, true}.
{erl_opts, [debug_info, fail_on_warning]}.
{deps, [
   %% mochiweb for JSON and header parsing
   mochiweb,
   %% webmachine for multipart content parsing
    nitrogen,
    nprocreg,
    simple_bridge
 ]}.

$ cat start.sh
#!/bin/sh
cd `dirname $0`
exec erl -pa ./deps/*/ebin -pa ./site/ebin -boot start_sasl -config "etc/app.config" -config "etc/mochiweb.config" -args_file "etc/vm.args" -eval "application:start(myapp)"  -eval "application:start(inets)"

拷贝nitrogen/rel/nitrogen/site目录下的文件到site目录中

 

cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/ebin ./ -r
cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/include ./ -r
cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/src ./ -r
cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/static ./ -r
cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/templates ./ -r
cp /home/andy/baidu/personal/mochiweb_test/nitrogen/rel/nitrogen/site/Emakefile ./

复制myapp/src 下的三个文件myapp.app.src,myapp_app.erl,myapp_sup.erl 到目录myapp/site/src下， 修改myapp_sup.erl如下：

 

$ cat myapp_sup.erl
-module(myapp_sup).
-behaviour(supervisor).
%% API
-export([start_link/0]).
%% Supervisor callbacks
-export([init/1]).
%% Helper macro for declaring children of supervisor
-define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}).
-include_lib("nitrogen/include/wf.hrl").
%% ===================================================================
%% API functions
%% ===================================================================
start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).
%% ===================================================================
%% Supervisor callbacks
%% ===================================================================
init([]) ->
     %% Start the Process Registry...
    application:start(nprocreg),
    %% Start Mochiweb...
    application:load(mochiweb),
    {ok, BindAddress} = application:get_env(mochiweb, bind_address),
    {ok, Port} = application:get_env(mochiweb, port),
    {ok, ServerName} = application:get_env(mochiweb, server_name),
    {ok, DocRoot} = application:get_env(mochiweb, document_root),
    io:format("Starting Mochiweb Server (~s) on ~s:~p, root: '~s'~n", [ServerName, BindAddress, Port, DocRoot]),
    % Start Mochiweb...
    Options = [
        {name, ServerName},
        {ip, BindAddress}, 
        {port, Port},
        {loop, fun loop/1}
    ],
    mochiweb_http:start(Options),
    
%    Web = {erlang_mcpack,
%       {erlang_mcpack, start_link, []},
%       permanent, 5000, worker, dynamic},
%    Processes=[Web],
    {ok, { {one_for_one, 5, 10}, []} }.
loop(Req) ->
    {ok, DocRoot} = application:get_env(mochiweb, document_root),
    RequestBridge = simple_bridge:make_request(mochiweb_request_bridge, {Req, DocRoot}),
    ResponseBridge = simple_bridge:make_response(mochiweb_response_bridge, {Req, DocRoot}),
    nitrogen:init_request(RequestBridge, ResponseBridge),
    nitrogen:run().

删除nitrogen_init.erl,nitrogen_mochiweb.erl

 

修改 myapp/etc下配置文件

 

andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp/etc$ cat app.config 
%% -*- mode: erlang -*-
[
    {myapp, [
        % If a signkey is not specified, then Nitrogen uses the Erlang cookie.
        % {signkey, "SECRET"}, 
        % Default session timeout is 20 minutes.
        % {session_timeout, 20},
        % Specify a session cookie name. Uses "wf" by default.
        % {cookie_name, "wf"}
    ]},
    %% SASL config
    {sasl, [
        {sasl_error_logger, {file, "log/sasl-error.log"}},
        {errlog_type, error},
        {error_logger_mf_dir, "log/sasl"},      % Log directory
        {error_logger_mf_maxbytes, 10485760},   % 10 MB max file size
        {error_logger_mf_maxfiles, 5}           % 5 files max
    ]}
].
andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp/etc$ cat mochiweb.config 
%% -*- mode: erlang -*-
[{mochiweb, [
    {bind_address, "0.0.0.0"},
    {port, 8000},<pre name="code" class="plain">andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp$ ./start.sh 
Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:5] [kernel-poll:true]
Eshell V5.8  (abort with ^G)
(myapp@127.0.0.1)1> Starting Mochiweb Server (nitrogen) on 0.0.0.0:8000, root: './site/static'
</pre>

{server_name, nitrogen}, {document_root, "./site/static"}]}].andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp/etc$ cat vm.args ## Name of the riak node-name myapp@127.0.0.1## Cookie for distributed erlang-setcookie myapp## Heartbeat management; auto-restarts VM if it dies or becomes unresponsive## (Disabled by default..use with caution!)##-heart## Enable kernel poll and a few async threads+K true+A 5## Increase number of concurrent ports/sockets-env ERL_MAX_PORTS 4096## Tweak GC to run more often -env ERL_FULLSWEEP_AFTER 10## Include .beam files for site.-pa ./site/ebin## Run code at startup.-eval "application:start(myapp)"

删除掉myapp下的src目录
修改start.sh文件属性

 

chmod 777 start.sh

测试一下：

 

andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp$ ./start.sh 
Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:5] [kernel-poll:true]
Eshell V5.8  (abort with ^G)
(myapp@127.0.0.1)1> Starting Mochiweb Server (nitrogen) on 0.0.0.0:8000, root: './site/static'

这里是开发环境，已经OK

 

下一步进行打包版本的修改

 

把nitrgen/rel下的overlay文件夹,overlay_mochiweb文件夹,mochiweb.config文件,rebar文件拷贝到myapp/rel目录下

修改mochiweb.config文件名为myapp.config,修改内容

 

{sys, [
    {lib_dirs, ["../deps","../site"]},
    {rel, "myapp", "2.0.4",
        [
            kernel,
            stdlib,
            sasl,
            inets,
            crypto,
            runtime_tools,
            mochiweb,
            simple_bridge,
            nprocreg
    ]},
    {rel, "start_clean", "",
        [
            kernel,
            stdlib
    ]},
    {boot_rel, "myapp"},
    {profile, embedded},
    {excl_sys_filters, [
        "^bin/.*",
        "^erts.*/bin/(dialyzer|typer)"
    ]},
    {app, mochiweb, [{incl_cond, include}]},
    {app, myapp, [{incl_cond, include}]},
    {app, nitrogen, [{incl_cond, include}]},
    {app, simple_bridge, [{incl_cond, include}]},
    {app, nprocreg, [{incl_cond, include}]},
    {app, sasl, [{incl_cond, include}]}
]}.
{rebar, [
    {empty_dirs, [
        "log/sasl"
    ]},
    {overlay, "overlay"},
    {overlay, "overlay_mochiweb"}
]}.

 

修改myapp/rel/overlay/bin目录下的nitrogen文件，重命名为myapp

把myapp/etc下的app.config文件与vm.args文件拷贝到myapp/rel/overlay/etc下替换掉之前的文件

把myapp/etc下的mochiweb.config文件拷贝到myapp/rel/overlay_mochiweb/etc下替换掉之前的文件

把myapp/site/ebin/myapp.app文件文件拷贝到myapp/rel/overlay_mochiweb/site/ebin目录下删除掉之前的文件

把myapp/site/src/下的myapp.app.src,myapp_app.erl,myapp_sup.erl文件拷贝到myapp/rel/overlay_mochiweb/site/src目录下删除掉之前的文件

一切ok

测试一下

 

andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp$ make rel
./rebar get-deps
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/mochiweb'
==> mochiweb (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/mochiweb'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nitrogen'
==> nitrogen (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nitrogen'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nprocreg'
==> nprocreg (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nprocreg'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/simple_bridge'
==> simple_bridge (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/simple_bridge'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/rel'
==> rel (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/rel'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/site'
==> site (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/site'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps'
==> deps (get-deps)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps'
==> myapp (get-deps)
./rebar compile
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/mochiweb'
==> mochiweb (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/mochiweb'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nitrogen'
==> nitrogen (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nitrogen'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nprocreg'
==> nprocreg (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/nprocreg'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/simple_bridge'
==> simple_bridge (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps/simple_bridge'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/rel'
==> rel (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/rel'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/site'
==> site (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/site'
==> Entering directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps'
==> deps (compile)
==> Leaving directory `/home/andy/baidu/personal/mochiweb_test/myapp/deps'
==> myapp (compile)
make[1]: 正在进入目录 `/home/andy/baidu/personal/mochiweb_test/myapp'
==> rel (generate)
make[2]: 正在进入目录 `/home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp'
Recompile: ./src/myapp_sup
Recompile: ./src/myapp_app
Recompile: ./src/index
make[2]:正在离开目录 `/home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp'
make[1]:正在离开目录 `/home/andy/baidu/personal/mochiweb_test/myapp'
Generated a self-contained myapp project
Usage: rel/myapp-1.0.1/bin/myapp {start|stop|restart|reboot|ping|console|attach}
andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp$ cd rel/myapp-1.0.1/bin/
andy@yuhaitao:~/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/bin$ ./myapp console
Exec: /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/erts-5.8/bin/erlexec -boot /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/releases/2.0.4/myapp -embedded  -config /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/etc/app.config -config /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/etc/mochiweb.config -args_file /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1/etc/vm.args -- console
Root: /home/andy/baidu/personal/mochiweb_test/myapp/rel/myapp-1.0.1
Erlang R14A (erts-5.8) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:5] [kernel-poll:true]
Eshell V5.8  (abort with ^G)
(myapp@127.0.0.1)1> Starting Mochiweb Server (nitrogen) on 0.0.0.0:8000, root: './site/static'

 

启动成功，ok

 

评论

2 楼 李政道 2017-03-03  

我还是个新手，我百度来半天没有解决到这个问题，请问你知道是什么原因么，麻烦帮我看哈这个问题怎么解决，非常感谢你的这篇博客。

1 楼 李政道 2017-03-03  

你好，我把你的工程下载下来：

  rebar compile


  然后 执行 start.sh


  没跑起来，报错  Conflicting -start_erl and -config options

  我试了另一种方式：

  执行  make rel 报错

  {"init terminating in do_boot","Module mochiglobal potentially included by two d
ifferent applications: mochiweb and nitrogen."}