Merge lp:~jaypipes/glance/bug742190 into lp:~glance-coresec/glance/cactus-trunk

[Jay Pipes]

Adds --config-file option to common options processing.

Updates glance-manage to use configuration files as well
as command line options.

Completes documentation on configuring Glance's servers
and CLI tools.

[Rick Harris (rconradharris)]

Overall looks good, but looks like test needs to be updated for new code.

Failed test: http://paste.openstack.org/show/1115/

review: Needs Fixing

[Jay Pipes (jaypipes)]

ah, cheers. I'll fix that up. not sure how that slipped past me ...

[Jay Pipes (jaypipes)]

ok, should be fixed. pls re-review.

[Devin Carlen (devcamcar)]

excellent, lgtm!

review: Approve

[Rick Harris (rconradharris)]

Looks great, thanks Jay.

review: Approve

=== modified file 'bin/glance-manage'
--- bin/glance-manage	2011-03-17 14:39:50 +0000
+++ bin/glance-manage	2011-04-07 18:56:35 +0000
@@ -52,9 +52,9 @@
 
     :param parser: The option parser
     """
-    glance.registry.db.add_options(parser)
     config.add_common_options(parser)
     config.add_log_options(parser)
+    glance.registry.db.add_options(parser)
 
 
 def do_db_version(options, args):
@@ -119,7 +119,11 @@
     (options, args) = config.parse_options(oparser)
 
     try:
-        config.setup_logging(options, {})
+        # We load the glance-registry config section because
+        # sql_connection is only part of the glance registry.
+        conf_file, conf = config.load_paste_config('glance-registry',
+                                                   options, args)
+        config.setup_logging(options, conf)
     except RuntimeError, e:
         sys.exit("ERROR: %s" % e)
 
@@ -127,6 +131,9 @@
         oparser.print_usage()
         sys.exit(1)
 
+    if conf.get('sql_connection') and not options['sql_connection']:
+        options['sql_connection'] = conf.get('sql_connection')
+
     dispatch_cmd(options, args)
 
 

=== modified file 'doc/source/configuring.rst'
--- doc/source/configuring.rst	2011-03-08 21:52:38 +0000
+++ doc/source/configuring.rst	2011-04-07 18:56:35 +0000
@@ -17,14 +17,57 @@
 Configuring Glance
 ==================
 
-.. todo:: Complete details of configuration with paste.deploy config files
+In addition to this documentation page, you can check the
+``etc/glance.conf.sample`` sample configuration file distributed with Glance
+for an example configuration file with detailed comments on what each options
+does.
+
+Common Configuration Options in Glance
+--------------------------------------
+
+Glance has a few command-line options that are common to all Glance programs:
+
+* ``--verbose``
+
+Optional. Default: ``False``
+
+Can be specified on the command line and in configuration files.
+
+Turns on the INFO level in logging and prints more verbose command-line
+interface printouts.
+
+* ``--debug``
+
+Optional. Default: ``False``
+
+Can be specified on the command line and in configuration files.
+
+Turns on the DEBUG level in logging.
+
+* ``--config-file=PATH``
+
+Optional. Default: ``None``
+
+Specified on the command line only.
+
+Takes a path to a configuration file to use when running the program. If this
+CLI option is not specified, then we check to see if the first argument is a
+file. If it is, then we try to use that as the configuration file. If there is
+no file or there were no arguments, we search for a configuration file in the
+following order:
+
+  - ./glance.conf
+  - ~/glance.conf
+  - ~/.glance/glance.conf
+  - /etc/glance/glance.conf
+  - /etc/glance.conf
 
 Configuring Logging in Glance
 -----------------------------
 
 There are a number of configuration options in Glance that control how Glance
-servers log messages. The configuration options are specified in the
-``glance.conf`` config file.
+servers log messages. The configuration options can be specified both on the
+command line and in the ``glance.conf`` config file.
 
 * ``--log-config=PATH``
 
@@ -36,8 +79,8 @@
 
 * ``--log-format``
 
-*Because of a bug in the PasteDeploy package, this option is only available
-on the command line.*
+`Because of a bug in the PasteDeploy package, this option is only available
+on the command line.`
 
 Optional. Default: ``%(asctime)s %(levelname)8s [%(name)s] %(message)s``
 
@@ -64,3 +107,119 @@
 Defaults to ``%Y-%m-%d %H:%M:%S``. See the
 `logging module <http://docs.python.org/library/logging.html>`_ documentation for
 more information on setting this format string.
+
+Configuring Glance Storage Backends
+-----------------------------------
+
+There are a number of configuration options in Glance that control how Glance
+stores disk images. These configuration options are specified in the
+``glance.conf`` config file `in the section [app:glance-api]`.
+
+* ``default_store=STORE``
+
+Optional. Default: ``file``
+
+Can only be specified in configuration files.
+
+Sets the storage backend to use by default when storing images in Glance.
+Available options for this option are (``file``, ``swift``, or ``s3``).
+
+* ``filesystem_store_datadir=PATH``
+
+Optional. Default: ``/var/lib/glance/images/``
+
+Can only be specified in configuration files.
+
+`This option is specific to the filesystem storage backend.`
+
+Sets the path where the filesystem storage backend write disk images. Note that
+the filesystem storage backend will attempt to create this directory if it does
+not exist. Ensure that the user that ``glance-api`` runs under has write
+permissions to this directory.
+
+* ``swift_store_auth_address=URL``
+
+Required when using the Swift storage backend.
+
+Can only be specified in configuration files.
+
+`This option is specific to the Swift storage backend.`
+
+Sets the authentication URL supplied to Swift when making calls to its storage
+system. For more information about the Swift authentication system, please
+see the `Swift auth <http://swift.openstack.org/overview_auth.html>`_ 
+documentation and the
+`overview of Swift authentication <http://docs.openstack.org/openstack-object-storage/admin/content/ch02s02.html>`_.
+
+* ``swift_store_user=USER``
+
+Required when using the Swift storage backend.
+
+Can only be specified in configuration files.
+
+`This option is specific to the Swift storage backend.`
+
+Sets the user to authenticate against the ``swift_store_auth_address`` with.
+
+* ``swift_store_key=KEY``
+
+Required when using the Swift storage backend.
+
+Can only be specified in configuration files.
+
+`This option is specific to the Swift storage backend.`
+
+Sets the authentication key to authenticate against the
+``swift_store_auth_address`` with for the user ``swift_store_user``.
+
+* ``swift_store_container=CONTAINER``
+
+Optional. Default: ``glance``
+
+Can only be specified in configuration files.
+
+`This option is specific to the Swift storage backend.`
+
+Sets the name of the container to use for Glance images in Swift.
+
+* ``swift_store_create_container_on_put``
+
+Optional. Default: ``False``
+
+Can only be specified in configuration files.
+
+`This option is specific to the Swift storage backend.`
+
+If true, Glance will attempt to create the container ``swift_store_container``
+if it does not exist.
+
+Configuring the Glance Registry
+-------------------------------
+
+Glance ships with a default, reference implementation registry server. There
+are a number of configuration options in Glance that control how this registry
+server operates. These configuration options are specified in the
+``glance.conf`` config file `in the section [app:glance-registry]`.
+
+* ``sql_connection=CONNECTION_STRING`` (``--sql-connection`` when specified
+  on command line)
+
+Optional. Default: ``None``
+
+Can be specified in configuration files. Can also be specified on the
+command-line for the ``glance-manage`` program.
+
+Sets the SQLAlchemy connection string to use when connecting to the registry
+database. Please see the documentation for
+`SQLAlchemy connection strings <http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html>`_
+online.
+
+* ``sql_timeout=SECONDS``
+  on command line)
+
+Optional. Default: ``3600``
+
+Can only be specified in configuration files.
+
+Sets the number of seconds after which SQLAlchemy should reconnect to the
+datastore if no activity has been made on the connection.

=== modified file 'etc/glance.conf.sample'
--- etc/glance.conf.sample	2011-03-05 00:02:26 +0000
+++ etc/glance.conf.sample	2011-04-07 18:56:35 +0000
@@ -34,9 +34,7 @@
 # ============ Swift Store Options =============================
 
 # Address where the Swift authentication service lives
-# The auth address should be in the form:
-#   <DOMAIN>[:<PORT>]/<VERSION>/<ACCOUNT>
-swift_store_auth_address = 127.0.0.1:8080/v1.0/glance-account
+swift_store_auth_address = 127.0.0.1:8080/v1.0/
 
 # User to authenticate against the Swift authentication service
 swift_store_user = jdoe

=== modified file 'glance/common/config.py'
--- glance/common/config.py	2011-03-08 21:52:38 +0000
+++ glance/common/config.py	2011-04-07 18:56:35 +0000
@@ -77,6 +77,12 @@
     group.add_option('-d', '--debug', default=False, dest="debug",
                      action="store_true",
                      help="Print debugging output")
+    group.add_option('--config-file', default=None, metavar="PATH",
+                     help="Path to the config file to use. When not specified "
+                          "(the default), we generally look at the first "
+                          "argument specified to be a config file, and if "
+                          "that is also missing, we search standard "
+                          "directories for a config file.")
     parser.add_option_group(group)
 
 
@@ -183,9 +189,9 @@
     """
 
     fix_path = lambda p: os.path.abspath(os.path.expanduser(p))
-    if getattr(options, 'config', None):
-        if os.path.exists(options.config_file):
-            return fix_path(getattr(options, 'config'))
+    if options.get('config_file'):
+        if os.path.exists(options['config_file']):
+            return fix_path(options['config_file'])
     elif args:
         if os.path.exists(args[0]):
             return fix_path(args[0])
@@ -203,6 +209,43 @@
             return cfg_file
 
 
+def load_paste_config(app_name, options, args):
+    """
+    Looks for a config file to use for an app and returns the
+    config file path and a configuration mapping from a paste config file.
+
+    We search for the paste config file in the following order:
+    * If --config-file option is used, use that
+    * If args[0] is a file, use that
+    * Search for glance.conf in standard directories:
+        * .
+        * ~.glance/
+        * ~
+        * /etc/glance
+        * /etc
+
+    :param app_name: Name of the application to load config for, or None.
+                     None signifies to only load the [DEFAULT] section of
+                     the config file.
+    :param options: Set of typed options returned from parse_options()
+    :param args: Command line arguments from argv[1:]
+    :retval Tuple of (conf_file, conf)
+
+    :raises RuntimeError when config file cannot be located or there was a
+            problem loading the configuration file.
+    """
+    conf_file = find_config_file(options, args)
+    if not conf_file:
+        raise RuntimeError("Unable to locate any configuration file. "
+                            "Cannot load application %s" % app_name)
+    try:
+        conf = deploy.appconfig("config:%s" % conf_file, name=app_name)
+        return conf_file, conf
+    except Exception, e:
+        raise RuntimeError("Error trying to load config %s: %s"
+                           % (conf_file, e))
+
+
 def load_paste_app(app_name, options, args):
     """
     Builds and returns a WSGI app from a paste config file.
@@ -224,13 +267,9 @@
     :raises RuntimeError when config file cannot be located or application
             cannot be loaded from config file
     """
-    conf_file = find_config_file(options, args)
-    if not conf_file:
-        raise RuntimeError("Unable to locate any configuration file. "
-                            "Cannot load application %s" % app_name)
+    conf_file, conf = load_paste_config(app_name, options, args)
+
     try:
-        conf = deploy.appconfig("config:%s" % conf_file, name=app_name)
-
         # Setup logging early, supplying both the CLI options and the
         # configuration mapping from the config file
         setup_logging(options, conf)

=== modified file 'glance/registry/db/__init__.py'
--- glance/registry/db/__init__.py	2011-02-02 16:55:07 +0000
+++ glance/registry/db/__init__.py	2011-04-07 18:56:35 +0000
@@ -33,7 +33,7 @@
     group = optparse.OptionGroup(parser, "Registry Database Options",
                                  help_text)
     group.add_option('--sql-connection', metavar="CONNECTION",
-                     default='sqlite:///glance.sqlite',
+                     default=None,
                      help="A valid SQLAlchemy connection string for the "
                           "registry database. Default: %default")
     parser.add_option_group(group)

=== modified file 'tests/unit/test_config.py'
--- tests/unit/test_config.py	2011-02-09 23:57:05 +0000
+++ tests/unit/test_config.py	2011-04-07 18:56:35 +0000
@@ -30,7 +30,7 @@
         config.add_common_options(parser)
         self.assertEquals(1, len(parser.option_groups))
 
-        expected_options = ['--verbose', '--debug']
+        expected_options = ['--verbose', '--debug', '--config-file']
         for e in expected_options:
             self.assertTrue(parser.option_groups[0].get_option(e),
                             "Missing required common option: %s" % e)
@@ -42,7 +42,8 @@
         config.add_common_options(parser)
         parsed_options, args = config.parse_options(parser)
 
-        expected_options = {'verbose': False, 'debug': False}
+        expected_options = {'verbose': False, 'debug': False,
+                            'config_file': None}
         self.assertEquals(expected_options, parsed_options)
 
         # test non-empty args and that parse_options() returns a mapping
@@ -51,7 +52,8 @@
         config.add_common_options(parser)
         parsed_options, args = config.parse_options(parser, ['--verbose'])
 
-        expected_options = {'verbose': True, 'debug': False}
+        expected_options = {'verbose': True, 'debug': False,
+                            'config_file': None}
         self.assertEquals(expected_options, parsed_options)
 
         # test non-empty args that contain unknown options raises