Merge lp:~ensoft-opensource/ensoft-sextant/minor_bugfixes into lp:ensoft-sextant

[Patrick Stevens]

Make Sextant Web a lot less chatty.

Make Sextant Web a lot less chatty.
Variety of other small bugfixes.

[Patrick Stevens (patrickas)]

review: Approve

[Martin Morrison (isoschiz)]

The prerequisite lp:~ensoft-opensource/ensoft-sextant/auditing has not yet been merged into lp:ensoft-sextant.

[ChrisD (gingerchris)]

Did we work out why info logging was coming out?
As discussed I thought logging defaulted to the level above info.

[Patrick Stevens (patrickas)]

I think it's because of Phil's addition:

logging.config.dictConfig({
         "version": 1,
         "handlers": {
             "console": {
                 "class": "logging.StreamHandler",
                 "level": logging.INFO,
                 "stream": "ext://sys.stderr",
             },
         },
         "root": {
             "level": logging.DEBUG,
             "handlers": ["console"],
         },
     })

which appears in sextant.environment or sextant.__main__, depending on
which revision you're looking at.

Patrick

On 26/08/14 15:02, Chris wrote:
> Did we work out why info logging was coming out?
> As discussed I thought logging defaulted to the level above info.

[ChrisD (gingerchris)]

Ok, I'd remove that "level" line - I doubt we want debug level on by
default.

I'd suggest we do as the @@@ suggests and add something like

        parser.add_argument('-v', '--verbose',
                            action='store_true', help="Show debug")
...
        if args.verbose:
            logging.setLevel(logging.DEBUG)

On 26 August 2014 15:06, Patrick Stevens <email address hidden> wrote:

> I think it's because of Phil's addition:
>
> logging.config.dictConfig({
> "version": 1,
> "handlers": {
> "console": {
> "class": "logging.StreamHandler",
> "level": logging.INFO,
> "stream": "ext://sys.stderr",
> },
> },
> "root": {
> "level": logging.DEBUG,
> "handlers": ["console"],
> },
> })
>
> which appears in sextant.environment or sextant.__main__, depending on
> which revision you're looking at.
>
> Patrick
>
> On 26/08/14 15:02, Chris wrote:
> > Did we work out why info logging was coming out?
> > As discussed I thought logging defaulted to the level above info.
>
>
> --
>
> https://code.launchpad.net/~ensoft-opensource/ensoft-sextant/minor_bugfixes/+merge/232219
> Your team Ensoft Open Source is subscribed to branch
> lp:~ensoft-opensource/ensoft-sextant/auditing.
>

[lp:~ensoft-opensource/ensoft-sextant/minor_bugfixes]

6. By James Harkin <email address hidden> on 2014-08-26

remove usage: __main__.py changed to SEXTANT

7. By James Harkin <email address hidden> on 2014-08-26

merge

8. By James Harkin <email address hidden> on 2014-08-26

merge

[Patrick Stevens (patrickas)]

By the way, the argparse-parent-parsers thing was why I didn't implement
this.

On 26/08/14 15:39, Chris wrote:
> Ok, I'd remove that "level" line - I doubt we want debug level on by
> default.
>
> I'd suggest we do as the @@@ suggests and add something like
>
> parser.add_argument('-v', '--verbose',
> action='store_true', help="Show debug")
> ...
> if args.verbose:
> logging.setLevel(logging.DEBUG)
>
>
>
>
>
> On 26 August 2014 15:06, Patrick Stevens <email address hidden> wrote:
>
>> I think it's because of Phil's addition:
>>
>> logging.config.dictConfig({
>> "version": 1,
>> "handlers": {
>> "console": {
>> "class": "logging.StreamHandler",
>> "level": logging.INFO,
>> "stream": "ext://sys.stderr",
>> },
>> },
>> "root": {
>> "level": logging.DEBUG,
>> "handlers": ["console"],
>> },
>> })
>>
>> which appears in sextant.environment or sextant.__main__, depending on
>> which revision you're looking at.
>>
>> Patrick
>>
>> On 26/08/14 15:02, Chris wrote:
>>> Did we work out why info logging was coming out?
>>> As discussed I thought logging defaulted to the level above info.
>>
>> --
>>
>> https://code.launchpad.net/~ensoft-opensource/ensoft-sextant/minor_bugfixes/+merge/232219
>> Your team Ensoft Open Source is subscribed to branch
>> lp:~ensoft-opensource/ensoft-sextant/auditing.
>>

[lp:~ensoft-opensource/ensoft-sextant/minor_bugfixes]

9. By Patrick Stevens <email address hidden> on 2014-08-29

Merge personal minor_bugfixes

10. By Patrick Stevens <email address hidden> on 2014-09-01

Merge trunk (SSH branch)

11. By Patrick Stevens <email address hidden> on 2014-09-01

Fix logging level ('warning' -> 'info')

[Patrick Stevens (patrickas)]

Is it OK for this to be approved without the --verbose or --quiet options? They seem more like feature requests than bugfixes anyway. Currently invoking Sextant from the command line prints out which config file we're using, then if we're in Sextant Web it prints which port we're serving on, while in sextant add-program it prints "Objdump has parsed", "Sending <num> objects to server" and "Sending complete! Exiting".

It also erroneously prints "Exit request sent" on completion of every command when ssh was used (that's because of a bug in ssh, which is patched in Debian but not Ubuntu). That's a future bugfix, I think.

[Patrick Stevens (patrickas)]

One comment

[ChrisD (gingerchris)]

Yes fine to approve without those options, I agree they are outside the scope of this task.

[lp:~ensoft-opensource/ensoft-sextant/minor_bugfixes]

12. By Patrick Stevens <email address hidden> on 2014-09-01

Handle an error where the user specified a non-existent place to SSH to

13. By Patrick Stevens <email address hidden> on 2014-09-01

SEXTANT -> sextant in usage message

=== modified file 'setup.py'
--- setup.py	2014-08-29 13:10:17 +0000
+++ setup.py	2014-09-01 15:05:19 +0000
@@ -18,7 +18,11 @@
     packages=['sextant', 'sextant.web', 'resources', 'etc'],
     package_dir={'sextant': 'src/sextant', 'resources': 'resources', 'etc': 'etc'},
     scripts=['bin/sextant'],
+    # neo4jrestclient has no python-neo4jrestclient package on Ubuntu 12,
+    # so we remove its dependency here. You *must* install neo4jrestclient
+    # somehow before using Sextant; we recommend pip.
     install_requires=['twisted'],
+    #install_requires=['neo4jrestclient', 'twisted'],
     package_data={'resources': ['sextant/web/*'], 'etc': ['*.conf']},
 )
 

=== modified file 'src/sextant/__main__.py'
--- src/sextant/__main__.py	2014-08-29 14:18:22 +0000
+++ src/sextant/__main__.py	2014-09-01 15:05:19 +0000
@@ -82,7 +82,7 @@
             st = ('Program {progname} with {numfuncs} functions '
                   'uploaded by {uploader} (id {uploaderid}) on {date}.')
             print(st.format(progname=program.program_name,
-                            numfuncs=program.number,
+                            numfuncs=program.number_of_funcs,
                             uploader=program.uploader,
                             uploaderid=program.uploader_id,
                             date=program.date))
@@ -164,7 +164,7 @@
 
     """
 
-    argumentparser = argparse.ArgumentParser(description="Invoke part of the SEXTANT program")
+    argumentparser = argparse.ArgumentParser(prog='sextant', usage='sextant', description="Invoke part of the SEXTANT program")
     subparsers = argumentparser.add_subparsers(title="subcommands")
 
     #set what will be defined as a "common function"
@@ -368,8 +368,16 @@
 
 
 def _is_localhost(host, port):
-    """Checks whether a host is an alias to localhost."""
-    return socket.getaddrinfo(host, port)[0][4][0] in ('127.0.0.1', '::1')
+    """
+    Checks whether a host is an alias to localhost.
+
+    Raises socket.gaierror if the host was not found.
+
+    """
+
+    addr = socket.getaddrinfo(host, port)[0][4][0]
+
+    return addr in ('127.0.0.1', '::1')
 
 
 def main():
@@ -380,7 +388,13 @@
 
         remotehost, remoteport = _get_host_and_port(args.remote_neo4j)
 
-        if _is_localhost(remotehost, remoteport):
+        try:
+            is_loc = _is_localhost(remotehost, remoteport)
+        except socket.gaierror:
+            logging.error('Server {} not found.'.format(remotehost))
+            return
+
+        if is_loc:
             # we are attempting to connect to localhost anyway, so we won't
             # bother to SSH to it.
             # There may be some ways the user can trick us into trying to SSH

=== modified file 'src/sextant/db_api.py'
--- src/sextant/db_api.py	2014-08-22 14:23:32 +0000
+++ src/sextant/db_api.py	2014-09-01 15:05:19 +0000
@@ -374,7 +374,8 @@
 
     ProgramWithMetadata = namedtuple('ProgramWithMetadata',
                                      ['uploader', 'uploader_id',
-                                      'program_name', 'date', 'number'])
+                                      'program_name', 'date', 
+                                      'number_of_funcs'])
 
     def __init__(self, url):
         self.url = url
@@ -459,10 +460,13 @@
     def programs_with_metadata(self):
         """
         Returns a set of namedtuples which represent the current database.
+        
         The namedtuples have .uploader, .uploader_id, .program_name, .date,
-        .number (of functions)
+        .number_of_funcs.
         :return: set of namedtuples
+       
         """
+        
         q = ("MATCH (base) WHERE base.type = 'program' "
              "MATCH (base)-[:subject]->(n)"
              "RETURN base.uploader, base.uploader_id, base.name, base.date, count(n)")

=== modified file 'src/sextant/environment.py'
--- src/sextant/environment.py	2014-08-29 13:19:53 +0000
+++ src/sextant/environment.py	2014-09-01 15:05:19 +0000
@@ -75,7 +75,7 @@
         "handlers": {
             "console": {
                 "class": "logging.StreamHandler",
-                "level": logging.WARNING,
+                "level": logging.INFO,
                 "stream": "ext://sys.stderr",
             },
         },

=== modified file 'src/sextant/web/server.py'
--- src/sextant/web/server.py	2014-08-26 11:04:04 +0000
+++ src/sextant/web/server.py	2014-09-01 15:05:19 +0000
@@ -84,7 +84,6 @@
             request.finish()
             defer.returnValue(None)
 
-        logging.info('enter')
         name = request.args["program_name"][0]
 
         try:
@@ -106,16 +105,14 @@
             defer.returnValue(None)
             neo4jconnection = None  # to silence the "referenced before assignment" warnings later
 
-        logging.info('created')
         exists = yield deferToThread(self.check_program_exists, neo4jconnection, name)
         if not exists:
             request.setResponseCode(404)
-            logging.info('returning nonexistent')
+            logging.debug('returning nonexistent')
             request.write("Name %s not found." % (escape(name)))
             request.finish()
             defer.returnValue(None)
 
-        logging.info('done created')
         allowed_queries = ("whole_program", "functions_calling", "functions_called_by", "call_paths", "shortest_path")
 
         if "query" not in request.args:
@@ -198,8 +195,7 @@
             request.finish()
             defer.returnValue(None)
 
-        logging.info('getting plot')
-        logging.info(program)
+        logging.debug('getting plot')
         if not program.functions:  # we got an empty program back:  the program is in the Sextant but has no functions
             request.setResponseCode(204)
             request.finish()
@@ -208,7 +204,7 @@
                                      remove_self_calls=False)
         request.setHeader("content-type", "image/svg+xml")
 
-        logging.info('SVG: return')
+        logging.debug('SVG: return')
         request.write(output)
         request.finish()
 
@@ -242,8 +238,6 @@
 
         query = request.args['query'][0]
 
-        logging.info('Properties: about to get_connection')
-
         try:
             neo4j_connection = yield deferToThread(self._get_connection)
         except Exception:
@@ -253,8 +247,6 @@
             defer.returnValue(None)
             neo4j_connection = None  # just to silence the "referenced before assignment" warnings
 
-        logging.info('got connection')
-
         if query == 'programs':
             request.setHeader("content-type", "application/json")
             prognames = yield deferToThread(self._get_program_names, neo4j_connection)