Merge ~smoser/cloud-init:cleanup/net-convert-add-debug into cloud-init:master

[Scott Moser]

tools: add '--debug' to tools/net-convert.py

In order to see some of the WARNING messages added by bug 1774666
I wanted logging output of tools/net-convert. This does:

a.) add '--debug' and make it print the network state and read yaml only
    if --debug is provided.
b.) set up basic logging so warnings goes to console by default and
    debug goes to console if --debug is provided.

see commit message

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:89df355378d78072e9576810d531c14b7efebdc2
https://jenkins.ubuntu.com/server/job/cloud-init-ci/164/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/164/rebuild

review: Needs Fixing (continuous-integration)

[Ryan Harper (raharper)]

Thanks. I sort of got used to the noisy output, but better to have it via --debug.

review: Approve

[Server Team CI bot (server-team-bot)]

FAILED: Autolanding.
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/cloud-init-autoland-test/7/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:9bc7bc6e5ed1bc716fbfde963fa2b12603359486
https://jenkins.ubuntu.com/server/job/cloud-init-ci/166/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    SUCCESS: MAAS Compatability Testing
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/166/rebuild

review: Approve (continuous-integration)

diff --git a/tools/net-convert.py b/tools/net-convert.py
index 68559cb..d1a4a64 100755
--- a/tools/net-convert.py
+++ b/tools/net-convert.py
@@ -4,11 +4,13 @@
 import argparse
 import json
 import os
+import sys
 import yaml
 
 from cloudinit.sources.helpers import openstack
 
 from cloudinit.net import eni
+from cloudinit import log
 from cloudinit.net import netplan
 from cloudinit.net import network_state
 from cloudinit.net import sysconfig
@@ -29,14 +31,23 @@ def main():
                         metavar="name,mac",
                         action='append',
                         help="interface name to mac mapping")
+    parser.add_argument("--debug", action='store_true',
+                        help='enable debug logging to stderr.')
     parser.add_argument("--output-kind", "-ok",
                         choices=['eni', 'netplan', 'sysconfig'],
                         required=True)
     args = parser.parse_args()
 
+    if not args.directory.endswith("/"):
+        args.directory += "/"
+
     if not os.path.isdir(args.directory):
         os.makedirs(args.directory)
 
+    if args.debug:
+        log.setupBasicLogging(level=log.DEBUG)
+    else:
+        log.setupBasicLogging(level=log.WARN)
     if args.mac:
         known_macs = {}
         for item in args.mac:
@@ -53,8 +64,10 @@ def main():
         pre_ns = yaml.load(net_data)
         if 'network' in pre_ns:
             pre_ns = pre_ns.get('network')
-        print("Input YAML")
-        print(yaml.dump(pre_ns, default_flow_style=False, indent=4))
+        if args.debug:
+            sys.stderr.write('\n'.join(
+                ["Input YAML",
+                 yaml.dump(pre_ns, default_flow_style=False, indent=4), ""]))
         ns = network_state.parse_net_config_data(pre_ns)
     else:
         pre_ns = openstack.convert_net_json(
@@ -65,8 +78,10 @@ def main():
         raise RuntimeError("No valid network_state object created from"
                            "input data")
 
-    print("\nInternal State")
-    print(yaml.dump(ns, default_flow_style=False, indent=4))
+    if args.debug:
+        sys.stderr.write('\n'.join([
+            "", "Internal State",
+            yaml.dump(ns, default_flow_style=False, indent=4), ""]))
     if args.output_kind == "eni":
         r_cls = eni.Renderer
     elif args.output_kind == "netplan":
@@ -75,6 +90,11 @@ def main():
         r_cls = sysconfig.Renderer
 
     r = r_cls()
+    sys.stderr.write(''.join([
+        "Read input format '%s' from '%s'.\n" % (
+            args.kind, args.network_data.name),
+        "Wrote output format '%s' to '%s'\n" % (
+            args.output_kind, args.directory)]) + "\n")
     r.render_network_state(network_state=ns, target=args.directory)