開源PaaS平台Cloud Foundry在阿裏雲上部署實戰

Cloud Foundry介紹

Cloud Foundry是業界第一個開源的PaaS，號稱工業界唯一的雲應用平台。
本文重點講述Cloud Foundry在阿裏雲上的部署方案，對於Cloud Foundry介紹的部分簡單略過。

Cloud Foundry解決了什麼問題？

關鍵詞：PaaS、開源
Cloud Foundry是一種平台即服務（PaaS），兼容多種基礎設施雲，提供多種開發框架和應用服務。目前開源支持部署Cloud Foundry的基礎設施雲包括AWS、Azure、OpenStack等等，CF在開發框架上支持Java、.NET、Ruby等等，有很強的靈活性。
IaaS幫助開發者和客戶解決了基礎設施的問題，Cloud Foundry作為PaaS，在雲上讓開發者的視角更高，開發者隻需要關注應用和數據。

Cloud Foundry組件（v2版本）

Cloud Foundry是由相對獨立的多個模塊構成的分布式係統，每個模塊單獨存在和運行，各模塊之間通過消息機製進行通信。

Cloud Foundry目前最新的架構為v3版本，各個組件的功能，官網有詳細的介紹，這邊不再贅述。
參考：https://docs.cloudfoundry.org/concepts/architecture/

阿裏雲上部署Cloud Foundry實戰

環境說明：

（本文使用軟件版本和環境的說明，涉及到的概念下文會詳細解釋）
阿裏雲region：華東1
ruby 版本：2.3.0
bosh-init 版本：0.0.96
Bosh CLI 版本：1.3262.4.0
CF CLI 版本：6.21.0

部署流程

在阿裏雲上部署Cloud Foundry，部署流程分為三大部分，分別為：
1. 開通阿裏雲環境資源：包括準備阿裏雲賬號，開通相關雲產品資源等等
2. 部署Bosh：安裝bosh-init，使用bosh-init部署Bosh
3. 部署Cloud Foundry：使用Bosh部署Cloud Foundry

本文後續內容會根據三步部署流程詳細講述。

一、開通阿裏雲環境資源

Cloud Foundry需要部署在VPC環境下，需要開通VPC資源，並創建虛擬交換機，規劃好部署Cloud Foundry的內網網段。
同時，也需要創建一對Access Key ID和Access Key Secret，也可以使用現有的Access Key ID和Access Key Secret對。

創建專有網絡VPC

網段沒有限製，筆者選擇了10.0.0./8網段

創建虛擬交換機

可用區和網段沒有限製，方便起見，筆者選擇了華東1可用區E，網段為10.0.0.0/25

創建Access Key ID和Access Key Secret

二、部署Bosh

Bosh介紹

Bosh是一個統一了平台即服務軟件（如Cloud Foundry）的發布、部署和生命周期管理的自動化配置部署工具。簡單的說，Bosh的作用之一就是部署Cloud Foundry，部署Cloud Foundry之前，首先要部署Bosh。
在講怎麼在阿裏雲上部署Bosh之前，首先需要了解Bosh中的幾個基本概念。
Bosh官網參考：https://bosh.io/docs

CPI

CPI全稱Cloud Platform Interface，是Bosh對外開放的一組標準化接口，在IaaS上部署Bosh，需要實現這組接口，Bosh用CPI對IaaS的資源進行管理，包括創建虛擬機，釋放虛擬機，等等......
在阿裏雲上部署Bosh，需要用阿裏雲OpenAPI實現CPI。
CPI參考：https://bosh.io/docs/cpi-api-v1.html
CPI API v1完整接口列表：

bosh-init

bosh-init是Bosh官網提供的一個開源工具，它的作用就是用來部署Bosh。
bosh-init參考：https://bosh.io/docs/using-bosh-init.html

stemcell

stemcell是虛擬機鏡像，預裝了部署過程中需要的組件（其中最重要的組件是Bosh Agent），官方的描述是：A stemcell is a versioned Operating System image wrapped with IaaS specific packaging. Bosh用CPI創建的虛擬機，用stemcell鏡像啟動。
stemcell參考：https://bosh.io/docs/stemcell.html

release

release是Bosh中一個安裝部署包的概念，包含了所有安裝分布式係統需要的源代碼、配置文件、腳本文件等等，官網描述為：A release is a versioned collection of configuration properties, configuration templates, start up scripts, source code, binary artifacts, and anything else required to build and deploy software in a reproducible way.
例如，安裝Bosh，我需要一個Bosh-release；用Bosh安裝Cloud Foundry，我需要一個cf-release。
所有release都有版本迭代，都可以在Bosh官網找到。
release參考：https://bosh.io/docs/release.html

Deployment

一個Deployment是一組VM的集合，由指定的stemcell鏡像啟動，用於部署對應release的係統。官方描述為：A deployment is a collection of VMs, built from a stemcell, that has been populated with specific releases and disks that keep persistent data. These resources are created based on a manifest file in the IaaS and managed by the BOSH Director, a centralized management server.
在Bosh的概念裏，一個Deployment對應一個release，Deployment的具體配置，寫在Deployment manifest裏，在部署過程中提供。
Deployment參考：https://bosh.io/docs/deployment.html

阿裏雲上部署Bosh

1. 創建ECS

通過阿裏雲控製台創建一個ECS（包年包月、按量均可），用於安裝bosh-init。方便起見，下文以bosh-init指代這台ECS。

推薦配置：
規格：2核4G及以上
鏡像：Ubuntu 14.04 64位
係統盤：40GB以上，高效雲盤
網絡類型：VPC實例，選擇在上一步創建好的VPC和虛擬交換機，公網IP可以選擇不分配。

2. 給bosh-init配置公網IP

因為bosh-init是VPC實例，需要給bosh-init綁定彈性公網IP，讓bosh-init可以通過公網訪問。

按需購買彈性公網IP，並給bosh-init綁定彈性公網IP

3. 安裝bosh-init

參考文檔：https://bosh.io/docs/install-bosh-init.html

SSH登陸到bosh-init這台ECS
下載bosh-init，下載地址見：https://bosh.io/docs/install-bosh-init.html
執行權限
chmod +x ~/Downloads/bosh-init-*
移動到/usr/local/bin
sudo mv ~/Downloads/bosh-init-* /usr/local/bin/bosh-init
驗證安裝成功
bosh-init -v
安裝對應環境，筆者使用的是Ubuntu的機器
sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3
確保已經安裝Ruby 2+環境

4. 下載bosh-release

bosh-release是開源的，可以從Bosh官網下載：
https://bosh.io/releases/github.com/cloudfoundry/bosh?all=1

筆者使用的是255.3版本的bosh-release

5. 下載cpi-release

CPI近期會開源，請關注：
https://github.com/alibaba/opstools

6. 配置manifest

根據以下模板，新建一個manifest文件bosh.yml，在模板中填充阿裏雲資源相關的內容。

---
name: bosh

releases:
- name: bosh
  url: file:///root/downloads/bosh-255.3.tgz
- name: bosh-aliyun-cpi
  url: file:///root/downloads/bosh-aliyun-cpi.tgz

resource_pools:
- name: vms
  network: private
  cloud_properties:
    instance_type: ecs.n4.large   # <--- 實例規格

networks:
- name: private
  type: manual
  subnets:
  - range: 10.0.0.0/8
    gateway: 10.0.0.1
    cloud_properties: {
      SecurityGroupId: SECURITY_GROUP_ID,    # <--- 安全組ID
      VSwitchId: VSWITCH-ID    # <--- 虛擬交換機ID
    }
- name: public
  type: vip

jobs:
- name: bosh
  instances: 1

  templates:
  - {name: nats, release: bosh}
  - {name: redis, release: bosh}
  - {name: postgres, release: bosh}
  - {name: blobstore, release: bosh}
  - {name: director, release: bosh}
  - {name: health_monitor, release: bosh}
  - {name: registry, release: bosh}
  - {name: aliyun_cpi, release: bosh-aliyun-cpi}

  resource_pool: vms

  networks:
  - name: private
    static_ips: [10.0.0.2]    # <--- ECS內網IP
    default: [dns, gateway]
  - name: public
    static_ips: [STATIC_IP]   # <--- 彈性公網IP

  properties:
    nats: &nats
      address: 127.0.0.1
      user: nats
      password: nats-password

    redis:
      listen_address: 127.0.0.1
      address: 127.0.0.1
      password: redis-password

    postgres: &db
      listen_address: 127.0.0.1
      host: 127.0.0.1
      user: postgres
      password: postgres-password
      database: bosh
      adapter: postgres

    registry: &registry
      address: 127.0.0.1
      host: 127.0.0.1
      db: *db
      http: {user: admin, password: admin, port: 25777}
      username: admin
      password: admin
      port: 25777

    blobstore: &blobstore
      address: 127.0.0.1
      port: 25250
      provider: dav
      director: {user: director, password: director-password}
      agent: {user: agent, password: agent-password}


    director:
      address: 127.0.0.1
      name: my-bosh
      db: *db
      cpi_job: aliyun_cpi
      max_threads: 10
      user_management:
        provider: local
        local:
          users:
          - {name: admin, password: admin}
          - {name: hm, password: hm-password}

    hm:
      director_account: {user: hm, password: hm-password}
      resurrector_enabled: true

    aliyun: &aliyun
      access_key_id: ACCESS_KEY_ID # <--- 阿裏雲 Access Key ID
      access_key: ACCESS_KEY # <--- 阿裏雲 Access Key Secret
      default_key_name: bosh
      default_security_groups: [bosh]
      region_id: cn-hangzhou  # <--- 阿裏雲 Region

    ntp: &ntp [0.pool.ntp.org, 1.pool.ntp.org]

cloud_provider:
  template: {name: aliyun_cpi, release: bosh-aliyun-cpi}

  mbus: "https://mbus:mbus-password@10.0.0.2:6868" # <--- ECS內網IP

  properties:
    aliyun: *aliyun
    agent:
      mbus: "nats://nats:nats-password@10.0.0.2:4222"  # <--- ECS內網IP
      blobstore:
        provider: "dav"
        options:
          endpoint: "https://10.0.0.2:25250"  # <--- ECS內網IP
          user: "agent"
          password: "agent-password"
    blobstore: {provider: local, path: /var/vcap/micro_bosh/data/cache}
    ntp: *ntp
    nats: *nats
    registry: *registry
    blobstore: *blobstore

7. 部署Bosh

執行部署命令：
bosh-init deploy bosh.yml

一個部署過程的示例：

8. 驗證Bosh

我們用Bosh CLI驗證Bosh是否部署成功。
Bosh CLI是Bosh官方提供的，用於和Bosh交互的命令行工具。在部署完成Bosh之後，用Bosh CLI和Bosh交互，執行相關命令，進行下一步Cloud Foundry的部署。

安裝Bosh CLI

Bosh CLI可以安裝在任意一台ECS上，或者本地主機上。
安裝Bosh CLI：https://bosh.io/docs/bosh-cli.html

驗證Bosh

執行Bosh CLI命令：
bosh target 10.0.0.2
筆者的Bosh CLI安裝在同一個安全組的ECS上，因此可以和部署Bosh的ECS進行內網通訊，直接通過內網IP，target到Bosh。如果需要通過公網通訊，需要給部署Bosh的ECS綁定彈性公網IP，或者使用NAT網關產品，保證網絡能通。

如圖顯示，成功連接到目標Bosh，驗證Bosh成功。
bosh releases、bosh stemcells這兩條命令，使用Bosh部署Cloud Foundry的時候會用到，我們下節細講。

三、部署Cloud Foundry

基本概念

首先我們也需要了解用Bosh部署Cloud Foundry過程中的幾個基本概念。

Bosh CLI

上一節講到，我們需要使用已經部署成功的Bosh來部署Cloud Foundry，通過Bosh CLI和Bosh進行交互，執行相關部署命令。因此，使用Bosh部署Cloud Foundry之前，首先需要了解Bosh CLI命令的使用。

Bosh CLI幾個基本命令：

1.連接到指定Bosh
bosh target [DIRECTOR_URL]

2.列出當前release倉庫中所有的release
bosh releases

3.上傳release到Bosh的release倉庫，隻有倉庫裏的release可以用於部署
bosh upload release [RELEASE_FILE]

4.列出當前stemcell倉庫裏的所有stemcell
bosh stemcells

5.上傳stemcell到Bosh的stemcell倉庫，隻有倉庫裏的stemcell可以用於部署
bosh upload stemcell STEMCELL_PATH
bosh upload stemcell STEMCELL_URL

6.列出當前所有的Deployment
bosh deployments

7.列出當前Deployment的信息
bosh deployment

8.切換到指定manifest對應的Deployment
bosh deployment [MANIFEST_PATH]

9.執行部署當前Deployment
bosh deploy

Bosh CLI官網參考：https://bosh.io/docs/sysadmin-commands.html

CPI

在上一步部署Bosh中，已經部署好的Bosh中已經包含了CPI組件，因此在部署Cloud Foundry中不再需要CPI-release

stemcell

和上一步部署Bosh中類似。

cf-release

Cloud Foundry官網提供的壓縮包，包含Cloud Foundry所有組件的源碼。

Deployment manifest

Deployment的配置文件，主要描述了用哪個stemcell，用哪個cf-release，需要部署哪些CF組件，需要多少VM，VM規格信息，VM的IP信息，具體哪個VM部署哪個CF組件，等等......

部署Cloud Foundry流程

1. 下載cf-release

cf-release是開源的，可以從Bosh官網下載：
https://bosh.io/releases/github.com/cloudfoundry/cf-release?all=1

筆者部署用的是215版本的cf-release

2. 登陸到安裝有Bosh CLI的機器

我們需要用Bosh CLI和Bosh進行交互，執行部署Cloud Foundry命令

3. 用Bosh CLI連接到部署好的Bosh

bosh target 10.0.0.2

4. 上傳cf-release

上傳下載完成的cf-release，執行以下Bosh CLI命令：
bosh upload release <release_file>

5. 配置manifest

根據以下模板，新建一個manifest文件cf.yml，在模板中填充阿裏雲資源相關的內容。

---
name: ali-cf
director_uuid: BOSH_DIRECTOR_UUID  # <--- Bosh Director UUID

releases:
- {name: cf, version: 215}

networks:
- name: private
  type: manual
  subnets:
  - range: 10.0.0.0/8
    gateway: 10.0.0.1
    #dns: [10.0.0.2]
    reserved: ["10.0.0.2"]
    static: ["10.0.0.3 - 10.0.0.100"]
    cloud_properties:
      SecurityGroupId: SECURITY_GROUP_ID  # <--- 安全組ID
      VSwitchId: VSWITCH_ID  # <--- 虛擬交換機ID

resource_pools:
- name: small_ecs
  network: private
  cloud_properties:
    instance_type: ecs.n1.tiny  # <--- 實例規格

compilation:
  workers: 1
  network: private
  reuse_compilation_vms: true
  cloud_properties:
    instance_type: ecs.n1.medium  # <--- 實例規格

update:
  canaries: 1
  max_in_flight: 1
  serial: false
  canary_watch_time: 30000-600000
  update_watch_time: 5000-600000

jobs:
- name: nats
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: nats, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.20]  # <--- ECS內網IP

- name: nfs
  instances: 1
  persistent_disk: 50
  resource_pool: small_ecs
  templates:
  - {name: debian_nfs_server, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.21]  # <--- ECS內網IP

- name: postgres
  instances: 1
  persistent_disk: 50
  resource_pool: small_ecs
  templates:
  - {name: postgres, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.22]  # <--- ECS內網IP
  update:
    serial: true

- name: cloud_controller
  instances: 2
  resource_pool: small_ecs
  templates:
  - {name: cloud_controller_ng, release: cf}
  - {name: cloud_controller_worker, release: cf}
  - {name: cloud_controller_clock, release: cf}
  - {name: nfs_mounter, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.23, 10.0.0.33]  # <--- ECS內網IP
  propertis:
    nfs_server:
      address: 10.0.0.21
      allow_from_entries: [10.0.0.0/24]

- name: hm9000
  instances: 1
  resource_pool: small_ecs
  templates:
  #  - {name: consul_agent, release: cf}
  - {name: hm9000, release: cf}
  #  - {name: metron_agent, release: cf}
  - {name: route_registrar, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.24]  # <--- ECS內網IP

- name: doppler_z1
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: doppler, release: cf}
  networks:
  - name: private
  properties:
    doppler: {zone: z1}
    doppler_endpoint:
      shared_secret: PASSWORD

- name: loggregator_trafficcontroller_z1
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: loggregator_trafficcontroller, release: cf}
  - {name: metron_agent, release: cf}
  - {name: route_registrar, release: cf}
  networks:
  - name: private
  properties:
    traffic_controller: {zone: z1}
    route_registrar:
      routes:
      - name: doppler
        registration_interval: 20s
        port: 8081
        uris:
        - "doppler.REPLACE_WITH_SYSTEM_DOMAIN"
      - name: loggregator
        registration_interval: 20s
        port: 8080
        uris:
          - "loggregator.REPLACE_WITH_SYSTEM_DOMAIN"

- name: uaa
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: uaa, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.25]  # <--- ECS內網IP
  properties:
    login:
      catalina_opts: -Xmx768m -XX:MaxPermSize=256m
    uaa:
      admin:
        client_secret: PASSWORD
      batch:
        password: PASSWORD
        username: batch_user
      cc:
        client_secret: PASSWORD
      scim:
        userids_enabled: false
        users:
        - cps@aliyun.com|Cps123456|scim.write,scim.read,openid,cloud_controller.admin
    uaadb:
      address: 10.0.0.22  # <--- postgres組件內網IP
      databases:
      - {name: uaadb, tag: uaa}
      db_scheme: postgresql
      port: 5524
      roles:
      - {name: uaaadmin, password: uaa-password, tag: admin}

- name: router
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: gorouter, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.27]  # <--- ECS內網IP
  properties:
    dropsonde: {enabled: true}

- name: dea_ng
  instances: 1
  resource_pool: small_ecs
  templates:
  - {name: dea_next, release: cf}
  networks:
  - name: private
    static_ips: [10.0.0.26]  # <--- ECS內網IP

properties:
  networks: {apps: private}
  app_domains: [DOMAIN]  # <--- domain
  domain: DOMAIN  # <--- domain
  system_domain: DOMAIN  # <--- domain
  system_domain_organization: default_organization

  cc:
    allow_app_ssh_access: false
    bulk_api_password: PASSWORD
    db_encryption_key: PASSWORD
    default_running_security_groups: [public_networks, dns]
    default_staging_security_groups: [public_networks, dns]
    install_buildpacks:
    - {name: java_buildpack, package: buildpack_java}
    - {name: ruby_buildpack, package: buildpack_ruby}
    - {name: nodejs_buildpack, package: buildpack_nodejs}
    - {name: go_buildpack, package: buildpack_go}
    - {name: python_buildpack, package: buildpack_python}
    - {name: php_buildpack, package: buildpack_php}
    - {name: staticfile_buildpack, package: buildpack_staticfile}
    - {name: binary_buildpack, package: buildpack_binary}
    internal_api_password: PASSWORD
    quota_definitions:
      default:
        memory_limit: 102400
        non_basic_services_allowed: true
        total_routes: 1000
        total_services: -1
    security_group_definitions:
    - name: private
      rules: []
    srv_api_uri: https://api.DOMAIN  # <--- domain
    staging_upload_password: PASSWORD
    staging_upload_user: staging_upload_user

  ccdb:
    address: 10.0.0.22  # <--- postgres組件內網IP
    databases:
    - {name: ccdb, tag: cc}
    db_scheme: postgres
    port: 5524
    roles:
    - {name: ccadmin, password: cc-password, tag: admin}

  databases:
    databases:
    - {name: ccdb, tag: cc, citext: true}
    - {name: uaadb, tag: uaa, citext: true}
    port: 5524
    roles:
    - {name: ccadmin, password: cc-password, tag: admin}
    - {name: uaaadmin, password: uaa-password, tag: admin}
  dea_next:
    advertise_interval_in_seconds: 5
    heartbeat_interval_in_seconds: 10

  etcd:
    machines: [10.0.0.24]  # <--- hm9000組件內網IP

  hm9000:
    url: https://hm9000.DOMAIN  # <--- domain

  nats:
    machines: [10.0.0.20]  # <--- nats組件內網IP
    password: nats-password
    port: 4222
    user: nats

  nfs_server:
    no_root_squash: true
    address: 10.0.0.21  # <--- nfs組件內網IP
    allow_from_entries: [10.0.0.0/24]

  uaa:
    no_ssl: true
    clients:
      gorouter:
        authorities: clients.read,clients.write,clients.admin,route.admin,route.advertise
        authorized-grant-types: client_credentials,refresh_token
        scope: openid,cloud_controller_service_permissions.read
        secret: PASSWORD
      cloud_controller_username_lookup:
        authorities: scim.userids
        authorized-grant-types: client_credentials
        secret: PASSWORD
      login:
        authorities: oauth.login,scim.write,clients.read,notifications.write,critical_notifications.write,emails.write,scim.useridsassword.write
        authorized-grant-types: authorization_code,client_credentials,refresh_token
        override: true
        redirect-uri: https://login.DOMAIN:8080/auth/cloudfoundry/callback  # <--- domain
        scope: openid,oauth.approvals,cloud_controller.read,cloud_controller.write
        secret: PASSWORD

    url: https://uaa.DOMAIN  # <--- domain

6. 新建cf.yml對應的Deployment

bosh deployment cf.yml

7. 執行部署命令

bosh deploy

8. 驗證Cloud Foundry

部署完成之後，執行以下命令，查看Cloud Foundry部署詳情：
bosh vms

至此，Cloud Foundry在阿裏雲上部署成功。

最後更新：2017-06-16 16:01:51

開源PaaS平台Cloud Foundry在阿裏雲上部署實戰

Cloud Foundry介紹

Cloud Foundry解決了什麼問題？

Cloud Foundry組件（v2版本）

阿裏雲上部署Cloud Foundry實戰

環境說明：

部署流程

一、開通阿裏雲環境資源

創建專有網絡VPC

創建虛擬交換機

創建Access Key ID和Access Key Secret

二、部署Bosh

Bosh介紹

CPI

bosh-init

stemcell

release

Deployment

阿裏雲上部署Bosh

1. 創建ECS

2. 給bosh-init配置公網IP

3. 安裝bosh-init

4. 下載bosh-release

5. 下載cpi-release

6. 配置manifest

7. 部署Bosh

8. 驗證Bosh

安裝Bosh CLI

驗證Bosh

三、部署Cloud Foundry

基本概念

Bosh CLI

CPI

stemcell

cf-release

Deployment manifest

部署Cloud Foundry流程

1. 下載cf-release

2. 登陸到安裝有Bosh CLI的機器

3. 用Bosh CLI連接到部署好的Bosh

4. 上傳cf-release

5. 配置manifest

6. 新建cf.yml對應的Deployment

7. 執行部署命令

8. 驗證Cloud Foundry

上一篇： 雲加數支撐商業地產轉型——無止境的商業想象力測試

下一篇： 新零售時代供應鏈發展的五大趨勢

相關內容

熱門內容

最新內容

上一篇：雲加數支撐商業地產轉型——無止境的商業想象力測試

下一篇：新零售時代供應鏈發展的五大趨勢