Merge lp:~bbaqar/charms/bundles/plumgrid-ons/liberty into lp:~plumgrid-team/charms/bundles/plumgrid-ons/liberty

[Bilal Baqar]

Changes:

1. Divided the bundle into 3 different bundles inside the same file
2. All plumgrid config parameters are in the same place
3. Added support to pull packages from LCM
4. Added assign_tags.sh to assign tags to controller, compute and gateway nodes
5. Added setup_lcm.sh to enable pg-tools

[lp:~bbaqar/charms/bundles/plumgrid-ons/liberty]

11. By Bilal Baqar on 2016-02-10

1. Divided the bundle into 3 different bundles inside the same file
2. All plumgrid config parameters are in the same place
3. Added support to pull packages from LCM
4. Added assign_tags.sh to assign tags to controller, compute and gateway nodes
5. Added setup_lcm.sh to enable pg-tools

=== modified file 'README.md'
--- README.md	2015-08-24 20:08:26 +0000
+++ README.md	2016-02-10 16:35:43 +0000
@@ -6,7 +6,7 @@
 
 The distributed architecture can scale to multi-rack or geographically distributed OpenStack deployments while the high availability features ensure maximum service uptime while minimizing downtime during maintenance windows. The product suite also simplifies ongoing operations with built-in monitoring, analytics and troubleshooting tools. By providing a comprehensive suite of capabilities, PLUMgrid ONS for OpenStack overcomes the scale, availability, feature set and lifecycle limitations of OpenStack Networking.
 
-This bundle deploys a OpenStack Cloud (Kilo release) on Ubuntu 14.04 LTS, providing Dashboard, Compute, Network (PLUMgrid), Block Storage (Cinder), Identity and Image services.
+This bundle deploys a OpenStack Cloud (Liberty release) on Ubuntu 14.04 LTS, providing Dashboard, Compute, Network (PLUMgrid), Block Storage (Cinder), Identity and Image services.
 
 ## PLUMgrid Components
 
@@ -30,14 +30,15 @@
 
  - A minimum of 8GB of physical RAM.
  - Enough CPU cores to support your capacity requirements.
- - Two cabled network ports on eth0 and eth1 (see below).
+ - Two cabled network ports on eth0 and eth1 on all nodes and eth2 as well on the plumgrid-gateway node (see below).
  - MTU of atleast 1580 needs to be supported by the managment network of MAAS.
 
-Servers should have two physical network ports cabled; the first is used for general communication between services in the Cloud, the second is used for 'public' network traffic to and from instances (North/South traffic) running within the Cloud.
+Servers should have two physical network ports cabled; the first is used for general communication between services in the Cloud, the second is used overlay fabric connectivity and the third is required on the PLUMgrid gateway node for 'public' network traffic to and from instances (North/South traffic) running within the Cloud.
 
 ## Deployment
 
-    juju quickstart bundle:plumgrid-ons
+    juju quickstart -i
+    juju-deployer -c bundles.yaml plumgrid-liberty
     juju set neutron-api plumgrid-virtual-ip="<Any IP in subnet>"
     juju set plumgrid-director plumgrid-virtual-ip="<Same IP as above>"
 
@@ -60,9 +61,10 @@
 
 ## Components
 
- - 1 Node for PLUMgrid Director and Mysql, Rabbitmq Server, Keystone, Nova Cloud Controller, Glance, Horizon and Cinder under LXC containers.
- - 1 Node for Nova Compute and PLUMgrid Edge.
- - 1 Node for PLUMgrid Gateway.
+ - 1 Node for JUJU state server
+ - 1 Node for PLUMgrid Director and Mysql, Rabbitmq Server, Keystone, Nova Cloud Controller, Glance, Horizon and Cinder under LXC containers
+ - 2 Nodes for Nova Compute and PLUMgrid Edge
+ - 1 Node for PLUMgrid Gateway
 
 PLUMgrid Open Networking Suite (ONS) is built with the ability to scale and support high availability across the PLUMgrid Platform. PLUMgrid ONS supports high availability across multiple components within a PLUMgrid Zone. PLUMgrid Director, PLUMgrid edge running on Nova Compute and PLUMgrid Gateway are designed to be horizontally scalable.
 
@@ -86,7 +88,6 @@
 ##Contact Information
 
 Bilal Baqar <[e-mail]>
-Bilal Ahmad <[e-mail]>
 
 PLUMgrid, Inc.
 5155 Old Ironsides Dr, Suite 200, Santa Clara, CA 95054, U.S.A | Ph:[phone]

=== added file 'assign_tag.sh'
--- assign_tag.sh	1970-01-01 00:00:00 +0000
+++ assign_tag.sh	2016-02-10 16:35:43 +0000
@@ -0,0 +1,26 @@
+#!/bin/bash
+set +e
+
+function usage() {
+  cat <<DELIM__
+usage: $(basename $0) [Parameter 1] [Parameter 2] [Parameter 3]  
+  [Parameter 1]: Registered MAAS Profile Name
+  [Parameter 2]: Node FQDN
+  [Parameter 3]: Tag Name
+
+Example:
+  ./assign_tag.sh maas-cli node0.maas compute 
+DELIM__
+}
+
+if [[ $# -ne 3 ]]; then
+    echo "Incorrect input"
+    usage
+    exit 1
+fi
+
+dpkg -l "jq" > /dev/null || sudo apt-get install -y "jq"
+nodes_json=$(maas $1 nodes list)
+node_id=$(echo $nodes_json | jq '.[] | select(.hostname=="'$2'")' | jq .system_id | sed 's/"//g')
+maas $1 tags new name="$3" comment="$3 Tag"
+maas $1 tag update-nodes $3 add="$node_id"

=== modified file 'bundles.yaml'
--- bundles.yaml	2016-02-01 07:08:58 +0000
+++ bundles.yaml	2016-02-10 16:35:43 +0000
@@ -1,4 +1,35 @@
-plumgrid-ons:
+# This bundle file containts the following bundles:
+# 1. plumgrid-ons-phase1: 
+#        This bundle deploys 4 nodes in total:
+#            - 2 x Compute
+#            - 1 x Controller
+#            - 1 x Gateways 
+#        Please note that the first compute is on the JUJU Bootstrap node i.e. JUJU machine 0
+#        This bundle waits for all the nodes to be deployed before it exits
+#
+# 2. plumgrid-ons-phase2:
+#        This bundle deploys all the required charms to bring up OpenStack Liberty with PLUMgrid
+#
+# 3. plumgrid-liberty:
+#        This bundle overwrites the configs in plumgrid-ons-phase2
+
+plumgrid-ons-phase1:
+  series: trusty
+  services:
+    controller-node:
+      charm: "cs:trusty/ubuntu"
+      num_units: 1
+      constraints: tags=controller
+    compute-node:
+      charm: "cs:trusty/ubuntu"
+      num_units: 2
+      constraints: tags=compute
+    gateway-node:
+      charm: "cs:trusty/ubuntu"
+      num_units: 1
+      constraints: tags=gateway
+plumgrid-ons-phase2:
+  inherits: plumgrid-ons-phase1
   series: 'trusty'
   relations:
     - ['mysql', 'keystone']
@@ -34,18 +65,18 @@
       annotations:
         "gui-x": "-250"
         "gui-y": "250"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     rabbitmq-server:
-      charm: cs:trusty/rabbitmq-server
+      charm: cs:trusty/rabbitmq-server-43
       num_units: 1
       options:
         source: cloud:trusty-liberty
       annotations:
         "gui-x": "-750"
         "gui-y": "250"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     keystone:
-      charm: cs:trusty/keystone
+      charm: cs:trusty/keystone-33
       num_units: 1
       options:
         admin-password: plumgrid
@@ -53,9 +84,9 @@
       annotations:
         "gui-x": "0"
         "gui-y": "0"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     nova-cloud-controller:
-      charm: cs:trusty/nova-cloud-controller
+      charm: cs:trusty/nova-cloud-controller-66
       num_units: 1
       options:
         console-access-protocol: novnc
@@ -65,34 +96,34 @@
       annotations:
         "gui-x": "-500"
         "gui-y": "0"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     glance:
-      charm: cs:trusty/glance
+      charm: cs:trusty/glance-30
       num_units: 1
       options:
         openstack-origin: cloud:trusty-liberty
       annotations:
         "gui-x": "-250"
         "gui-y": "-250"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     openstack-dashboard:
-      charm: cs:trusty/openstack-dashboard
+      charm: cs:trusty/openstack-dashboard-21
       num_units: 1
       options:
         openstack-origin: cloud:trusty-liberty
       annotations:
         "gui-x": "0"
         "gui-y": "-500"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     cinder:
-      charm: cs:trusty/cinder
+      charm: cs:trusty/cinder-34
       num_units: 1
       options:
         openstack-origin: cloud:trusty-liberty
       annotations:
         "gui-x": "250"
         "gui-y": "-250"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     neutron-api:
       branch: lp:~bbaqar/charms/trusty/neutron-api/liberty
       num_units: 1
@@ -102,15 +133,13 @@
         openstack-origin: cloud:trusty-liberty
         plumgrid-password: plumgrid
         plumgrid-username: plumgrid
-        plumgrid-virtual-ip: 192.168.100.250
       annotations:
         "gui-x": "500"
         "gui-y": "0"
-      to: 'lxc:plumgrid-director=0'
+      to: 'lxc:controller-node=0'
     neutron-api-plumgrid:
       branch: lp:~bbaqar/charms/trusty/neutron-api-plumgrid/liberty
       options:
-        install_sources: "deb http://37.48.126.173/ build413b18/"
         enable-metadata: True
       annotations:
         "gui-x": "750"
@@ -118,15 +147,14 @@
     plumgrid-director:
       branch: lp:~bbaqar/charms/trusty/plumgrid-director/liberty
       num_units: 1
-      options:
-        install_sources: "deb http://37.48.126.173/ build413b18/"
-        plumgrid-virtual-ip: 192.168.100.250
       annotations:
         "gui-x": "0"
         "gui-y": "500"
+      to:
+        - "controller-node=0"
     nova-compute:
-      charm: cs:trusty/nova-compute
-      num_units: 1
+      charm: cs:trusty/nova-compute-36
+      num_units: 2
       options:
         enable-live-migration: true
         enable-resize: true
@@ -137,19 +165,45 @@
       annotations:
         "gui-x": "250"
         "gui-y": "250"
-      to: '0'
+      to:
+        - "compute-node=0"
+        - "compute-node=1"
     plumgrid-edge:
       branch: lp:~bbaqar/charms/trusty/plumgrid-edge/liberty
-      options:
-        install_sources: "deb http://37.48.126.173/ build413b18/"
       annotations:
         "gui-x": "500"
         "gui-y": "500"
     plumgrid-gateway:
       branch: lp:~bbaqar/charms/trusty/plumgrid-gateway/liberty
       num_units: 1
-      options:
-        install_sources: "deb http://37.48.126.173/ build413b18/"
       annotations:
         "gui-x": "-500"
         "gui-y": "500"
+      to:
+        - "gateway-node=0"
+
+plumgrid-liberty:
+  inherits: plumgrid-ons-phase2
+  overrides:
+    plumgrid-virtual-ip: 192.168.100.250
+    #plumgrid-license-key: "ENTER KEY CONTENT HERE"
+
+    # Provide apt repository for PLUMgrid packages
+    install_sources: "deb http://37.48.126.173/ build413b18/"
+
+    # Use this value if pulling packages from LCM
+    #install_sources: |
+    #      - "deb http://192.168.100.5:81/plumgrid plumgrid fuji-413"
+    #      - "deb http://192.168.100.5:81/plumgrid-images plumgrid fuji-413"
+
+    #plumgrid-build:
+    #iovisor-build:
+
+    # Interfaces on gateway node for external connectivity
+    #external-interfaces: '{"node2":"eth2"}'
+
+    # Uncomment for seperate fabric/data network
+    # Interfaces on all PLUMgrid nodes connected to fabric/data network
+    #fabric-interfaces: '{"DEFAULT":"eth1"}'
+    # Fabric/data network
+    #os-data-network: "192.168.102.0/24"

=== added file 'setup_lcm.sh'
--- setup_lcm.sh	1970-01-01 00:00:00 +0000
+++ setup_lcm.sh	2016-02-10 16:35:43 +0000
@@ -0,0 +1,118 @@
+#!/bin/bash
+set -e
+
+if [ "$EUID" -ne 0 ]
+  then echo "Please run script with sudo"
+  exit
+fi
+
+function usage() {
+  cat <<DELIM__
+usage: $(basename $0) [options]
+
+Options:
+  --clobber             Will clear stored configurations and do a fresh run
+DELIM__
+}
+
+TEMP=$(getopt -o c --long ,clobber -- "$@")
+if [[ $? -ne 0 ]]; then
+  usage
+  exit 1
+fi
+
+eval set -- "$TEMP"
+
+while true; do
+  case "$1" in
+    --clobber) clobber=1; shift ;;
+    --) shift; break ;;
+    *) usage; exit 1
+      ;;
+  esac
+done
+
+mkdir -p $HOME/.pginstall
+
+function read_raw() {
+  eval "local=\$${2}"
+  if [[ -n "$local" && "$local" =~ $3 ]]; then
+    ## the value stored in the destination variable is already OK
+    echo "Using provided value [${local}] for: $1"
+    return 0
+  fi
+
+  if [[ $# -eq 5 ]]; then
+    echo -n "${1} (default [${5}])"
+    dfl="$5"
+  else
+    echo -n "${1}"
+    dfl=""
+  fi
+
+  while true; do
+    local=""
+    echo -n ": "
+    read local
+    [[ $local == "" ]] && local="$dfl"
+    if [[ "$local" =~ $3 ]]; then
+      break
+    fi
+    echo "  Please try again.  [${local}] is not a valid value: $4"
+  done
+  eval $2="\"$local\""
+}
+
+function read_nonempty() {
+  if [[ $# -eq 3 ]]; then
+    read_raw "$1" "$2" "[^ ]+" "must be non-empty" "$3"
+  else
+    read_raw "$1" "$2" "[^ ]+" "must be non-empty"
+  fi
+}
+
+function read_alnum() {
+  if [[ $# -eq 3 ]]; then
+    read_raw "$1" "$2" "^[0-9a-zA-Z]+$" "must use alphanumeric characters only" "$3"
+  else
+    read_raw "$1" "$2" "^[0-9a-zA-Z]+$" "must use alphanumeric characters only"
+  fi
+}
+
+function read_url() {
+  read_raw "$1" "$2" "^(ftp|http|https)://[^ ]+" "invalid schema (e.g. http://IP)"
+}
+
+function write_default_file() {
+  cat > $HOME/.pginstall/config <<DELIM__
+MAAS_CONTROLLER_REGION_ADMIN_USERNAME="${MAAS_CONTROLLER_REGION_ADMIN_USERNAME}"
+plumgrid_repo_url="${plumgrid_repo_url}"
+lvm_keypath_content="${lvm_keypath_content}"
+zone_name="${zone_name}"
+DELIM__
+  return 0
+}
+
+function lcm_key_check() {
+  zone_name_=${zone_name}
+  while true; do
+    zone_name="${zone_name_}"
+    read_alnum 'Enter LCM Canonical zone name' zone_name
+    read_url "Enter LCM repository base url" plumgrid_repo_url
+    echo -n "Checking ssh key on LCM for your deployment..."
+    ret=$(curl -Lks ${plumgrid_repo_url}/files/ssh_keys/zones/$zone_name/id_rsa.pub -o /tmp/id_rsa.pub -w '%{http_code}' || true)
+    [ $ret -eq 200 ] && break
+    echo "Unable to get ssh key from LCM for your deployment."
+    echo "Check zone name or retry after running canonical-pg-refresh.sh --key-only script on LCM and make sure your deployment shows up."
+    zone_name_=""
+  done
+  echo "OK"
+  mkdir -p /var/lib/plumgrid/zones/$zone_name
+  mv /tmp/id_rsa.pub /var/lib/plumgrid/zones/$zone_name/id_rsa.pub
+  lvm_keypath_content="/var/lib/plumgrid/zones/${zone_name}/id_rsa.pub"
+}
+
+[[ -r $HOME/.pginstall/config && $clobber != 1 ]] && . $HOME/.pginstall/config
+read_nonempty "Username of MAAS Region Admin User" MAAS_CONTROLLER_REGION_ADMIN_USERNAME "root"
+lcm_key_check
+write_default_file