基于openstack上的MICRO BOSH 安装cf

bosh部署3要素

1. manifest

2. release包

3. stemcell

如何获得？

创建manifest

用spiff生成manifest

spiff下载地址：

https://github.com/cloudfoundry-incubator/spiff/releases

在cf-release根目录

修改spec/fixtures/openstack/cf-stub.yml中的配置

运行./generate_deployment_manifest openstack cf-stub.yml > cf.yml

资源

cf-release下载

https://github.com/cloudfoundry-community/community-shared-boshreleases

stemcell下载

http://bosh-artifacts.cfapps.io/file_collections?type=stemcells

cf-release版本和stemcell，bosh版本对应关系查询

https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/v187

部署

1. 上传release

$ bosh upload release cf-release

验证：

$ bosh releases

+---------------------+------------+-------------+

| Name | Versions | Commit Hash |

+---------------------+------------+-------------+

| cf | 169 | ae04d8ba+ |

+---------------------+------------+-------------+

 

 2. 上传stemcell

bosh upload stemcell 

验证：

$ bosh stemcells

+------------------------------------+---------+--------------------------------------+

| Name | Version | CID |

+------------------------------------+---------+--------------------------------------+

| bosh-openstack-kvm-centos-go_agent | 2748 | 0a4da153-4878-4cd4-8451-6ddefe7f6e34 |

3.部署

$ bosh deployment cf.yml

$ bosh deploy

部署完成后

1. 所有DEA节点的cc api域名需解析到内部网络

例如：/etc/hosts 加入：10.60.31.112 api.ae2.demo

不然，dea上传droplet会报错。

问题排查

上传stemcell不能调用CPI

Director task 2
Error 100: Unable to connect to the OpenStack Compute API. Check task debug log for details.

原因：microbosh不能访问openstack cc

解决：

在microbosh的机器里，/etc/hosts 加上：

172.16.40.14    cc14.ae2

超过cpu核配置限制

E, [2014-10-10T03:34:37.945276 #6683] [create_vm(decb786a-0cdc-4eca-8827-17e5bb1d1fcb, ...)] ERROR -- : OpenStack API Request Entity Too Large error. Check task debug log f
or details.
E, [2014-10-10T03:34:37.945342 #6683] [create_vm(decb786a-0cdc-4eca-8827-17e5bb1d1fcb, ...)] ERROR -- : Expected([200, 202]) <=> Actual(413 Request Entity Too Large)
response => #<Excon::Response:0x007f7e942384e8 @data={:body=>"{\"overLimit\": {\"message\": \"Quota exceeded for cores: Requested 2, but already used 49 of 50 cores\", \"
code\": 413, \"retryAfter\": \"0\"}}", :headers=>{"Retry-After"=>"0", "Content-Length"=>"132", "Content-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"r
eq-fd2272b6-d749-4b9f-bd8d-e2b6792ed988", "Date"=>"Fri, 10 Oct 2014 03:34:39 GMT"}, :status=>413, :remote_ip=>"172.16.40.14"}, @body="{\"overLimit\": {\"message\": \"Quota 
exceeded for cores: Requested 2, but already used 49 of 50 cores\", \"code\": 413, \"retryAfter\": \"0\"}}", @headers={"Retry-After"=>"0", "Content-Length"=>"132", "Content
-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"req-fd2272b6-d749-4b9f-bd8d-e2b6792ed988", "Date"=>"Fri, 10 Oct 2014 03:34:39 GMT"}, @status=413, @remot
e_ip="172.16.40.14"> (Excon::Errors::RequestEntityTooLarge)

修改/etc/nova.conf

quota_cores=-1 # 无限制

get_state超时

Director task 29
Started preparing deployment
Started preparing deployment > Binding deployment. Done (00:00:00)
Started preparing deployment > Binding releases. Done (00:00:00)
Started preparing deployment > Binding existing deployment. Failed: Timed out sending `get_state' to 65941315-33fb-446c-af36-cef368e7e62d after 45 seconds (00:02:15)

Error 450002: Timed out sending `get_state' to 65941315-33fb-446c-af36-cef368e7e62d after 45 seconds

用bosh cck命令删除vm reference，每次都选3

3. Delete VM reference (DANGEROUS!)

创建虚机失败

| fault | {"message": "No valid host was found. ", "code": 500, "details": " File \"/usr/lib/python2.6/site-packages/nova/scheduler/filter_scheduler.py\", line 107, in schedule_run_instance |
| | raise exception.NoValidHost(reason=\"\") |
| | ", "created": "2014-10-11T06:16:43Z"}

nova日志/var/log/nova/scheduler.log：

raise exception.PortLimitExceeded()

网卡超过限制

neutron port-list

修改/etc/neutron/neutron.conf

quota_port = -1

编译失败

错误：

Compile failed

解决办法：

a. 编译

用stemcell创建一个虚机，上传release包到该虚机，解压，运行编译失败模块包里的packging脚本，查看具体错误，解决编译错误，一般是第三方依赖不满足。

b.重新打包

重新打包依赖包和模块包，计算该包的sha1sum，并用base64编码，替换release.MF中该模块的sha1sum，重新打包整个release包：cf-xxx.tgz

c. 重新上传release包

bosh upload release cf-xxx.tgz

 

上传stemcell时报SchemaValidationError错

Director task 1
Error 100: Invalid OpenStack cloud properties: #<Membrane::SchemaValidationError: { openstack => { api_key => Expected instance of String, given an instance of Fixnum } }>

解决办法：

进入micro bosh虚拟机

a. 修改director配置

vi /var/vcap/jobs/director/config/director.yml.erb

api_key: 123456  =>  api_key: '123456'

注意是修改director.yml.erb文件，

b. 重启director服务

bosh/bin/monit restart director

你会发现/var/vcap/jobs/director/config/director.yml已经被修改。

c. 重新上传stemcell。