Merge lp:~jml/pkgme-devportal/more-fingerprints into lp:pkgme-devportal

[Jonathan Lange]

Even more stub backends

I said I wasn't going to do this, but it's pretty easy and will make writing
the weekly report less of a pain.

More backends, some refactoring. Not all of the new ones have tests, and
there are no integration tests.

[James Westby (james-w)]

Could unity-webapps perhaps also use your new want_single_file_with_metadata?

Thanks,

James

review: Approve

[lp:~jml/pkgme-devportal/more-fingerprints]

135. By Jonathan Lange on 2012-09-19

Move to central location.

136. By Jonathan Lange on 2012-09-19

Use the thing for unity as well.

[Jonathan Lange (jml)]

On 19 September 2012 16:39, James Westby <email address hidden> wrote:
> Review: Approve
>
> Could unity-webapps perhaps also use your new want_single_file_with_metadata?

Good call. Have done so, making it use the extension thing. I don't
think the factoring is ideal, btw, but I think it's an improvement on
what we've got.

jml

[James Westby (james-w)]

Great, thanks, I'd been wanting to do something like that when writing the webapps
backend, but couldn't figure out a very nice interface, so I'm glad you did it.

Thanks,

James

review: Approve

=== added directory 'devportalbinary/backends/desktop'
=== added file 'devportalbinary/backends/desktop/all_info'
--- devportalbinary/backends/desktop/all_info	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/desktop/all_info	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import DesktopBackend
+
+
+if __name__ == '__main__':
+    sys.exit(DesktopBackend.all_info_script())

=== added file 'devportalbinary/backends/desktop/want'
--- devportalbinary/backends/desktop/want	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/desktop/want	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import DesktopBackend
+
+
+if __name__ == '__main__':
+    sys.exit(DesktopBackend.want_script())

=== added directory 'devportalbinary/backends/diff'
=== added file 'devportalbinary/backends/diff/all_info'
--- devportalbinary/backends/diff/all_info	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/diff/all_info	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import DiffBackend
+
+
+if __name__ == '__main__':
+    sys.exit(DiffBackend.all_info_script())

=== added file 'devportalbinary/backends/diff/want'
--- devportalbinary/backends/diff/want	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/diff/want	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import DiffBackend
+
+
+if __name__ == '__main__':
+    sys.exit(DiffBackend.want_script())

=== added directory 'devportalbinary/backends/exe'
=== added file 'devportalbinary/backends/exe/all_info'
--- devportalbinary/backends/exe/all_info	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/exe/all_info	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ExeBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ExeBackend.all_info_script())

=== added file 'devportalbinary/backends/exe/want'
--- devportalbinary/backends/exe/want	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/exe/want	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ExeBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ExeBackend.want_script())

=== added directory 'devportalbinary/backends/image'
=== added file 'devportalbinary/backends/image/all_info'
--- devportalbinary/backends/image/all_info	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/image/all_info	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ImageBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ImageBackend.all_info_script())

=== added file 'devportalbinary/backends/image/want'
--- devportalbinary/backends/image/want	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/image/want	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ImageBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ImageBackend.want_script())

=== added directory 'devportalbinary/backends/shell'
=== added file 'devportalbinary/backends/shell/all_info'
--- devportalbinary/backends/shell/all_info	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/shell/all_info	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ShellScriptBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ShellScriptBackend.all_info_script())

=== added file 'devportalbinary/backends/shell/want'
--- devportalbinary/backends/shell/want	1970-01-01 00:00:00 +0000
+++ devportalbinary/backends/shell/want	2012-09-19 16:09:21 +0000
@@ -0,0 +1,11 @@
+#!/usr/bin/env python
+# Copyright 2011 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import sys
+
+from devportalbinary.stubs import ShellScriptBackend
+
+
+if __name__ == '__main__':
+    sys.exit(ShellScriptBackend.want_script())

=== modified file 'devportalbinary/metadata.py'
--- devportalbinary/metadata.py	2012-09-18 14:34:23 +0000
+++ devportalbinary/metadata.py	2012-09-19 16:09:21 +0000
@@ -266,6 +266,47 @@
     return metadata, None
 
 
+def want_single_file_with_metadata(backend, path, metadata, predicate,
+                                   match_score,
+                                   mismatch_message, too_many_message):
+    excluded_files = get_excluded_package_files(backend, metadata)
+    files = list(get_package_files(path, excluded_files))
+    # By default, we don't want it and give no reason.  Sane default in
+    # case of buggy code below.
+    score = 0
+    reason = None
+    if len(files) == 0:
+        reason = 'No files found, just metadata'
+    elif len(files) == 1:
+        filename = files[0]
+        if predicate(filename):
+            score = match_score
+        else:
+            reason = '%s: %r' % (mismatch_message, filename)
+    else:
+        reason = '%s: %r' % (too_many_message, sorted(files))
+    return {'score': score, 'reason': reason}
+
+
+def _has_extensions(extensions):
+    def _filename_has_extensions(filename):
+        for ext in extensions:
+            if filename.endswith(ext):
+                return True
+        return False
+    return _filename_has_extensions
+
+
+def want_single_file_by_extension(cls, path, metadata, extensions, name, score=20):
+    return want_single_file_with_metadata(
+        cls, path, metadata,
+        _has_extensions(extensions),
+        score,
+        "File is not %s" % (name,),
+        "More files than just %s" % (name,),
+        )
+
+
 class MetadataBackend(object):
     """A backend that is mostly powered by metadata from MyApps."""
 

=== modified file 'devportalbinary/pdf.py'
--- devportalbinary/pdf.py	2012-09-12 15:04:26 +0000
+++ devportalbinary/pdf.py	2012-09-19 16:09:21 +0000
@@ -4,11 +4,10 @@
 import os
 
 from devportalbinary.metadata import (
-    get_excluded_package_files,
-    get_package_files,
     make_all_info_fn,
     make_want_fn,
     MetadataBackend,
+    want_single_file_with_metadata,
     )
 
 
@@ -50,23 +49,10 @@
 
     @classmethod
     def want_with_metadata(cls, path, metadata):
-        excluded_files = get_excluded_package_files(cls, metadata)
-        files = list(get_package_files(path, excluded_files))
-        # By default, we don't want it and give no reason.  Sane default in
-        # case of buggy code below.
-        score = 0
-        reason = None
-        if len(files) == 0:
-            reason = 'No files found, just metadata'
-        elif len(files) == 1:
-            filename = files[0]
-            if filename.endswith(".pdf"):
-                score = 20
-            else:
-                reason = 'File is not a PDF: %r' % (filename,)
-        else:
-            reason = 'More files than just a PDF: %r' % (sorted(files),)
-        return {'score': score, 'reason': reason}
+        return want_single_file_with_metadata(
+            cls, path, metadata,
+            lambda filename: filename.endswith('.pdf'),
+            20, "File is not a PDF", "More files than just a PDF")
 
 
 want = make_want_fn(PdfBackend)

=== modified file 'devportalbinary/stubs.py'
--- devportalbinary/stubs.py	2012-09-12 17:27:24 +0000
+++ devportalbinary/stubs.py	2012-09-19 16:09:21 +0000
@@ -22,6 +22,7 @@
     make_all_info_fn,
     make_want_fn,
     MetadataBackend,
+    want_single_file_by_extension,
     )
 
 
@@ -33,14 +34,14 @@
             "%s not implemented yet" % (backend_name,))
 
 
-class UnimplementableBackend(PkgmeError):
+class UnimplementableBackendError(PkgmeError):
     """Raised by backends we'll never implement."""
 
     def __init__(self, backend_name, reason=None):
         msg = '%s will never be implemented' % (backend_name,)
         if reason:
             msg = ': '.join([msg, reason])
-        super(UnimplementableBackend, self).__init__(msg)
+        super(UnimplementableBackendError, self).__init__(msg)
 
 
 class StubBackend(MetadataBackend):
@@ -68,6 +69,14 @@
         return make_all_info_fn(cls)(path)
 
 
+class UnimplementableBackend(StubBackend):
+
+    reason = None
+
+    def get_info(self):
+        raise UnimplementableBackendError(self.name, self.reason)
+
+
 class JarBackend(StubBackend):
 
     name = 'Java binary'
@@ -131,31 +140,35 @@
         return {'reason': reason, 'score': score}
 
 
-class TextBackend(StubBackend):
+class TextBackend(UnimplementableBackend):
 
     name = 'Text'
-
-    @classmethod
-    def want_with_metadata(cls, path, metadata):
-        excluded_files = get_excluded_package_files(cls, metadata)
-        files = list(get_package_files(path, excluded_files))
-        score = 0
-        reason = None
-        if len(files) == 0:
-            reason = 'No files found, just metadata'
-        elif len(files) == 1:
-            filename = files[0]
-            if filename.endswith(".txt"):
-                score = 20
-            else:
-                reason = 'File is not a .txt: %r' % (filename,)
-        else:
-            reason = 'More files than just a .txt: %r' % (sorted(files),)
-        return {'score': score, 'reason': reason}
-
-    def get_info(self):
-        raise UnimplementableBackend(
-            self.name, "You can't package a text file")
+    reason = "You can't package a text file"
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.txt'], "a .txt")
+
+
+class ImageBackend(UnimplementableBackend):
+
+    name = 'Image'
+    reason = "You can't package an image"
+
+    EXTENSIONS = [
+        '.bmp',
+        '.ico',
+        '.jpeg',
+        '.jpg',
+        '.png',
+        '.svg',
+        ]
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, cls.EXTENSIONS, "an image")
 
 
 class PythonBackend(StubBackend):
@@ -179,3 +192,45 @@
             else:
                 reason = "No setup.py found"
         return {'reason': reason, 'score': score}
+
+
+class ShellScriptBackend(StubBackend):
+
+    name = 'Shell script'
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.sh'], "a .sh", score=5)
+
+
+class DesktopBackend(UnimplementableBackend):
+
+    name = 'Desktop file'
+    reason = "Can't package a .desktop file"
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.desktop'], "a desktop file")
+
+
+class DiffBackend(UnimplementableBackend):
+
+    name = 'Diff'
+    reason = "Can't package a .diff"
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.diff'], "a diff")
+
+
+class ExeBackend(StubBackend):
+
+    name = 'EXE'
+
+    @classmethod
+    def want_with_metadata(cls, path, metadata):
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.exe'], "a .exe", score=5)

=== modified file 'devportalbinary/tests/test_stubs.py'
--- devportalbinary/tests/test_stubs.py	2012-09-12 17:49:31 +0000
+++ devportalbinary/tests/test_stubs.py	2012-09-19 16:09:21 +0000
@@ -9,11 +9,13 @@
     BackendNotImplemented,
     DebianBinaryBackend,
     DebianSourceBackend,
+    ImageBackend,
     JarBackend,
     StubBackend,
     PythonBackend,
+    ShellScriptBackend,
     TextBackend,
-    UnimplementableBackend,
+    UnimplementableBackendError,
     )
 from ..testing import BackendTests
 
@@ -35,11 +37,11 @@
 class UnimplementableTests(TestCase):
 
     def test_str(self):
-        e = UnimplementableBackend('backend_name')
+        e = UnimplementableBackendError('backend_name')
         self.assertEqual('backend_name will never be implemented', str(e))
 
     def test_str_with_reason(self):
-        e = UnimplementableBackend(
+        e = UnimplementableBackendError(
             'backend_name', "You can't package a text file!")
         self.assertEqual(
             ("backend_name will never be implemented: You can't package "
@@ -47,7 +49,7 @@
             str(e))
 
     def test_user_friendly(self):
-        e = UnimplementableBackend('backend_name')
+        e = UnimplementableBackendError('backend_name')
         self.assertIsInstance(e, PkgmeError)
 
 
@@ -158,7 +160,72 @@
 
     def test_get_info(self):
         backend = self.make_backend()
-        self.assertRaises(UnimplementableBackend, backend.get_info)
+        self.assertRaises(UnimplementableBackendError, backend.get_info)
+
+
+class ShellScriptTests(BackendTests):
+
+    BACKEND = ShellScriptBackend
+
+    def test_want_with_only_metadata(self):
+        self.assertEqual(
+            {'score': 0, 'reason': 'No files found, just metadata'},
+            self.want([VALID_METADATA_FILE]))
+
+    def test_want_with_only_shell(self):
+        filename = self.getUniqueString() + '.sh'
+        self.assertEqual(
+            {'score': 5, 'reason': None},
+            self.want([VALID_METADATA_FILE, filename]))
+
+    def test_want_with_only_non_shell(self):
+        filename = self.getUniqueString() + '.txt'
+        self.assertEqual(
+            {'score': 0, 'reason': "File is not a .sh: %r" % (filename,)},
+            self.want([VALID_METADATA_FILE, filename]))
+
+
+class ImageTests(BackendTests):
+
+    BACKEND = ImageBackend
+
+    def test_want_with_only_metadata(self):
+        self.assertEqual(
+            {'score': 0, 'reason': 'No files found, just metadata'},
+            self.want([VALID_METADATA_FILE]))
+
+    def test_want_with_random_files(self):
+        filename = self.getUniqueString() + '.data'
+        self.assertEqual(
+            {'score': 0, 'reason': "File is not an image: %r" % (filename,)},
+            self.want([VALID_METADATA_FILE, filename]))
+
+    def test_want_with_only_jpg(self):
+        # Do all image types
+        filename = self.getUniqueString() + '.jpg'
+        self.assertEqual(
+            {'score': 20, 'reason': None},
+            self.want([VALID_METADATA_FILE, filename]))
+
+    def test_want_with_only_jpeg(self):
+        # Do all image types
+        filename = self.getUniqueString() + '.jpeg'
+        self.assertEqual(
+            {'score': 20, 'reason': None},
+            self.want([VALID_METADATA_FILE, filename]))
+
+    def test_want_with_more_than_image(self):
+        img_file = self.getUniqueString() + '.jpg'
+        other_file = self.getUniqueString() + '.exe'
+        self.assertEqual(
+            {'score': 0,
+             'reason': ('More files than just an image: %r' %
+                        (sorted([img_file, other_file]),))},
+            self.want([VALID_METADATA_FILE, img_file, other_file]))
+
+    def test_get_info(self):
+        backend = self.make_backend()
+        self.assertRaises(UnimplementableBackendError, backend.get_info)
 
 
 class PythonTests(BackendTests):

=== modified file 'devportalbinary/unity_webapps.py'
--- devportalbinary/unity_webapps.py	2012-09-17 20:37:06 +0000
+++ devportalbinary/unity_webapps.py	2012-09-19 16:09:21 +0000
@@ -2,13 +2,12 @@
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 from .metadata import (
-    get_excluded_package_files,
     get_icon_install_map,
     get_install_file,
-    get_package_files,
     make_all_info_fn,
     make_want_fn,
     MetadataBackend,
+    want_single_file_by_extension,
     )
 
 
@@ -99,23 +98,8 @@
 
     @classmethod
     def want_with_metadata(cls, path, metadata):
-        excluded_files = get_excluded_package_files(cls, metadata)
-        files = list(get_package_files(path, excluded_files))
-        # By default, we don't want it and give no reason.  Sane default in
-        # case of buggy code below.
-        score = 0
-        reason = None
-        if len(files) == 0:
-            reason = 'No files found, just metadata'
-        elif len(files) == 1:
-            filename = files[0]
-            if filename.endswith(".user.js"):
-                score = 20
-            else:
-                reason = 'File is not *.user.js: %r' % (filename,)
-        else:
-            reason = 'More files than just one *.user.js: %r' % (sorted(files),)
-        return {'score': score, 'reason': reason}
+        return want_single_file_by_extension(
+            cls, path, metadata, ['.user.js'], '*.user.js')
 
 
 want = make_want_fn(UnityWebappsBackend)