Merge bootstack-ops:create_patch_groups_using_AZ into bootstack-ops:master

[Paul Henien]

Added script called "create_patch_groups.py"

Script retrieves node details from maas and juju, sorts the nodes into patch groups depending on availability zone and creates a upgrade profile in landscape for each patch group.

Script has a --help flag for instructions on how to run the script.
Can be tested using the --dry-run flag which outputs the patch groups but doesn't create upgrade profiles in landscape.

Requirements:
landscape-api and yaml packages needs to be installed
yaml file containing juju status format yaml
yaml file called landscape_creds.yml which contains landscape credentials

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Giuseppe Petralia (peppepetra)]

Please see my comments inline.

review: Needs Fixing

Unmerged commits

bcd9d20... by Paul Henien on 2019-09-24

added comments

42312df... by Paul Henien on 2019-08-19

modified script to take jsfy file as argument

a864500... by Paul Henien on 2019-08-14

added create_patch_groups.py script

8000536... by James Hebden on 2019-06-24

Moved to using snapcraft bases and updated for snapcraft 3.0

Reviewed-on: https://code.launchpad.net/~canonical-bootstack/bootstack-ops/+git/bootstack-ops/+merge/367557
Reviewed-by: David O Neill <email address hidden>

3ec3503... by James <email address hidden> on 2019-05-17

Readded libc6

3759b88... by James <email address hidden> on 2019-05-17

Moved to using snapcraft bases and updated for snapcraft 3.0

93070a0... by Wouter van Bommel on 2019-05-09

Added a maas test script for network testing

Reviewed-on: https://code.launchpad.net/~woutervb/bootstack-ops/+git/bootstack-ops/+merge/364496
Reviewed-by: David O Neill <email address hidden>

906f248... by Giuseppe Petralia on 2019-05-09

Switch to controller model before launching juju create-backup

Reviewed-on: https://code.launchpad.net/~peppepetra86/bootstack-ops/+git/bootstack-ops/+merge/362453
Reviewed-by: David O Neill <email address hidden>
Reviewed-by: Alok G Singh <email address hidden>

58aaa04... by Giuseppe Petralia on 2019-05-09

Add sleep retry on MAAS connection errors

Reviewed-on: https://code.launchpad.net/~canonical-bootstack/bootstack-ops/+git/bootstack-ops/+merge/365904
Reviewed-by: David O Neill <email address hidden>

7a73b9b... by Drew Freiberger on 2019-05-08

Updated openstack_managed_upgrades.py to specify phases matching openstack upgrade timing template

Reviewed-on: https://code.launchpad.net/~afreiberger/bootstack-ops/+git/bootstack-ops/+merge/367075
Reviewed-by: James Hebden <email address hidden>

diff --git a/bootstack-ops/create_patch_groups.py b/bootstack-ops/create_patch_groups.py
new file mode 100755
index 0000000..b00a937
--- /dev/null
+++ b/bootstack-ops/create_patch_groups.py
@@ -0,0 +1,281 @@
+#!/usr/bin/env python2.7
+
+"""
+# create_patch_groups.py
+
+Retrieve maas machines and juju containers, sort into patch groups and create a upgrade profile in landscape for each patch group
+
+## Version History
+
+
+### 0.1.0
+
+* Initial release
+
+"""
+
+__version__ = "0.1.0"
+__author__ = "Paul Henien <[e-mail]>"
+
+import subprocess
+import json
+import os
+import sys
+import yaml
+import argparse
+import time
+# Check that landscape-api package is installed
+try:
+    from landscape_api.base import API, HTTPError
+except ImportError:
+    sys.exit("""Please install the package 'Landscape API Python module' before continuing.""")
+
+
+def get_nodes():
+    # Execute "maas list" to grab the username to interact with maas
+    try:
+        user_name = subprocess.check_output("maas list | awk '{print $1}'", shell=True).decode('ascii').strip()
+    except:
+        print("Please check user exists")
+    # Execute "maas list" to grab the api version
+    api_version = ((subprocess.check_output("maas list | awk '{print $2}'", shell=True).decode('ascii').strip()).split('/'))[5]
+    # Check if api version 1.0 or 2.0 is used and execute node retrieval
+    # command based on the api version
+    if api_version == '1.0':
+        nodes = subprocess.check_output('maas {} nodes list'.format(user_name), shell=True).decode('ascii').strip()
+    elif api_version == '2.0':
+        nodes = subprocess.check_output('maas {} machines read'.format(user_name), shell=True).decode('ascii').strip()
+    return nodes
+
+
+def nodes_in_az(nodes):
+    # Converts node data into json
+    nodes_json = json.loads(nodes)
+    # Initialise variable to store zones and the nodes assigned
+    zones = dict()
+    # Run through each node
+    for x in nodes_json:
+        # "key" is the zone name that is used as a key in the dictionary
+        key = x['zone']['resource_uri'].split('/')[5]
+        # setdefault checks if the key exists,
+        # if key doesn't exist in the dictionary it will add it,
+        # if not it will append the value to the existing key
+        zones.setdefault(key, [])
+        zones[key].append(x['hostname'])
+    return zones
+
+
+def landscape_auth():
+    # Authenticate to landscape using credentials from landscape_creds.yml and return authentication token
+    try: # Check if landscape_creds.yml file exists
+        with open("landscape_creds.yml", 'r') as ymlfile:
+            cfg = yaml.safe_load(ymlfile)
+    except IOError:
+        print("Please provide a valid landscape credentials file")
+        sys.exit(1)
+
+    api = API(cfg['uri'], cfg['key'], cfg['secret'], cfg['ca'])
+
+    return api
+
+
+def landscape_computers(api, nodes):
+    # Retrieve a list of the computers in landscaoe and append them to a list called ls_computers
+    try:
+        computers = api.get_computers()
+        pass
+    except HTTPError, e:
+        print ("\nGot server error:\n"
+                "Code: %d\n"
+                "Message: %s\n") % (e.code, e.message)
+        sys.exit(1)
+
+    ls_computers = []
+    for computer in computers:
+        ls_computers.append(computer['title'].encode("utf-8"))
+
+    print "Computers in landscape: {}".format(ls_computers)
+    print ""
+
+    return ls_computers
+
+
+def multiple_az_into_groups(YAML, args, PATCH_GROUPS, ls_computers):
+
+    if args.application is None: # Check if any applications are added in the excluded flag
+        excluded = []
+    else:
+        excluded = args.application.lower().split(',')
+        print("Excluded: {}".format(excluded))
+        print("")
+
+    APPS = YAML["applications"]
+
+    for APP in APPS:
+        if APP not in excluded: # If application hasn't been excluded, nodes are added to patch groups
+            try:
+                UNITS = []
+                counter = 0
+                for UNIT in APPS[APP]['units']:
+                    UNITS.append(UNIT)
+                print("{} : {}".format(APP, UNITS))
+            except:
+                print("{} : No units".format(APP))
+
+
+    for machine in YAML['machines']:
+        try:
+            for container in YAML['machines'][machine]['containers']:
+                for APP in APPS:
+                    if APP not in excluded:
+                        try:
+                            for UNIT in APPS[APP]['units']:
+                                if APPS[APP]['units'][UNIT]['machine'] == container:
+                                    a, zone = YAML['machines'][machine]['containers'][container]['hardware'].split("=")
+                                    if YAML['machines'][machine]['containers'][container]['instance-id'] in ls_computers:
+                                        PATCH_GROUPS[zone].append(YAML['machines'][machine]['containers'][container]['instance-id'])
+                        except:
+                            pass
+        except:
+            pass
+
+    PG = []
+
+    for key, value in PATCH_GROUPS.items():
+        counter = 0
+        GROUP = []
+        for i in value:
+            if i in ls_computers:
+                GROUP.append(i.encode("utf-8"))
+            counter += 1
+        PG.append(GROUP)
+
+    print ""
+    print "PG 1: {}".format(PG[0])
+    print "PG 2: {}".format(PG[1])
+    print "PG 3: {}".format(PG[2])
+
+    return PATCH_GROUPS
+
+
+def single_az_into_groups(YAML, args, nodes, ls_computers):
+    PATCH_GROUPS = [[], [], []]
+    counter = 0
+    for key, value in nodes.items():
+        for i in value:
+            if i in ls_computers: # Check if node exists in landscape
+                PATCH_GROUPS[counter % 3].append(i.encode("utf-8"))
+            counter += 1
+
+    if args.application is None:
+        excluded = []
+    else:
+        excluded = args.application.lower().split(',')
+        print("Excluded: {}".format(excluded))
+        print("")
+
+    APPS = YAML["applications"]
+
+    for APP in APPS:
+        if APP not in excluded:
+            try:
+                UNITS = []
+                counter = 0
+                for UNIT in APPS[APP]['units']:
+                    for machine in YAML['machines']:
+                        try:
+                            for container in YAML['machines'][machine]['containers']:
+                                if APPS[APP]['units'][UNIT]['machine'] == container:
+                                    HOSTNAME = YAML['machines'][machine]['containers'][container]['instance-id']
+                                    UNITS.append(HOSTNAME)
+                                    if HOSTNAME not in PATCH_GROUPS:
+                                        if HOSTNAME in ls_computers:
+                                            PATCH_GROUPS[counter % 3].append(HOSTNAME) # Distributes hosts in application over three patch groups
+                                            counter += 1
+                        except:
+                            pass
+                print("{} : {}".format(APP, UNITS))
+            except:
+                print("{} : No units".format(APP))
+
+    print ""
+    print "PATCH_GROUP1: {}".format(PATCH_GROUPS[0])
+    print "PATCH_GROUP2: {}".format(PATCH_GROUPS[1])
+    print "PATCH_GROUP3: {}".format(PATCH_GROUPS[2])
+
+    return PATCH_GROUPS
+
+
+"""
+  def remove_duplicates(PATCH_GROUPS):
+    PG = []
+    for group in PATCH_GROUPS:
+        PG.append(list(dict.fromkeys(group)))
+
+    print ""
+    print "PATCH_GROUP1: {}".format(PG[0])
+    print "PATCH_GROUP2: {}".format(PG[1])
+    print "PATCH_GROUP3: {}".format(PG[2])
+
+    return PG
+"""
+
+
+def create_upgrade_profile(api, PATCH_GROUPS):
+    # Takes patch groups and creates upgrade profiles in landscape for each one
+    upgrade_profiles = api.get_upgrade_profiles()
+    profiles = []
+    for profile in upgrade_profiles:
+        profiles.append(profile["name"])
+
+    counter = 1
+    for group in PATCH_GROUPS.keys():
+        try:
+            if "patch_group{}".format(counter) in profiles:
+                up_pro = "patch_group{}".format(counter)
+                api.remove_upgrade_profile(name=up_pro)
+
+            api.create_upgrade_profile(title="PATCH_GROUP{}".format(counter),every="week",on_days=["we","th"],at_hour=20,at_minute=0,upgrade_type="all")
+            for machine in PATCH_GROUPS[group]:
+                api.add_tags_to_computers(query=machine, tags="PATCH_GROUP{}".format(counter))
+            api.associate_upgrade_profile(name="patch_group{}".format(counter),tags="PATCH_GROUP{}".format(counter))
+        except HTTPError, e:
+            print ("\nGot server error:\n"
+             "Code: %d\n"
+             "Message: %s\n") % (e.code, e.message)
+            sys.exit(1)
+
+        counter += 1
+
+
+def main():
+    parser = argparse.ArgumentParser(description="Create patch groups from maas kvms and juju lxds")
+    parser.add_argument("--exclude", help="exclude juju application. Format: app1,app2,app3", dest="application", type=str, required=False)
+    parser.add_argument("--dry-run", help="only lists patch groups but doesn't create them", action="store_true", default=False)
+    parser.add_argument("--jsfy-file", help="juju status format yaml file path (run juju status --format yaml > jsfy to create file) ", dest="jsfy", required=True)
+    args = parser.parse_args()
+    try:
+        with open(args.jsfy) as myjsfy:
+            YAML = yaml.safe_load(myjsfy)
+    except IOError:
+        print("Please provide a valid jsfy file")
+        sys.exit(1)
+
+    nodes = get_nodes()
+    az_node_assignment = nodes_in_az(nodes)
+    api = landscape_auth()
+    ls_computers = landscape_computers(api, az_node_assignment)
+
+    # Check if cloud has multiple AZs
+    if len(az_node_assignment) > 1:
+        PATCH_GROUPS = multiple_az_into_groups(YAML, args, az_node_assignment, ls_computers)
+    else:
+        PATCH_GROUPS = single_az_into_groups(YAML, args, az_node_assignment, ls_computers)
+    # PATCH_GROUPS = remove_duplicates(PATCH_GROUPS)
+    # If dry_run flag is not set, create patch groups
+    if args.dry_run == False:
+        create_upgrade_profile(api, PATCH_GROUPS)
+
+
+if __name__ == "__main__":
+    main()