Merge ~pieq/oem-qa-autosummary:fix-1945724-image-url into oem-qa-autosummary:master

[Pierre Equoy]

See issue lp:1945724 for more context.

With this MR, if auto-summary cannot find the mail.content file that contains the image name and the image sha256, it will ask user to manually enter the full URL to that image, and use it to compute the WebDAV URI to download the associated sha256 file that contains the image sha256 value.

This is a bit more flexible than the current situation where user can only input the image name, and not the URL.

To see it in action, check the following screen recording:

https://drive.google.com/file/d/1-ufHaCMXlMpfABC-i4gkpq957l6wVOrd/view?usp=sharing

1. The carlsbad project is selected, with the x03 milestone.
2. The mail.content file cannot be found, so user manually inputs full URL to the image name
3. The webapp computes the WebDAV URI to point to the file containing the sha256 value and use it in the generated summary.

NOTE: In the above video, I use the x03 milestone, but then paste the URL to a x02 milestone image. This is for demonstration purpose only, because the x03 milestone for that project is actually not out yet.

[StanleyHuang (stanley31)]

LGTM +1

review: Approve

[Nara Huang (narahuang)]

LGTM +1

review: Approve

diff --git a/autosummary/summary.py b/autosummary/summary.py
index 9a08276..6117361 100644
--- a/autosummary/summary.py
+++ b/autosummary/summary.py
@@ -12,23 +12,19 @@ WEBDAV_CREDENTIALS = os.environ.get("WEBDAV_CREDENTIALS")
 
 class MailContentNotFoundError(FileNotFoundError):
     """Raised when the mail.content file is not found on the WebDAV server"""
-    logging.error("the mail.content file is not found on the WebDAV server")
-    pass
 
 
 class Sha256FileNotFoundError(FileNotFoundError):
     """Raised when the sha256sum file for a given image is not found on the WebDAV server"""
-    logging.error(
-        "sha256sum file for a given image is not found on the WebDAV server")
-    pass
 
 
 class Summary:
-    def __init__(self, launchpad, project_name, milestone_name, image_name=""):
+    def __init__(self, launchpad, project_name, milestone_name, image_url=""):
         logging.info("Initailizing Summary object")
         self.project_name = project_name
         self.milestone_name = milestone_name
         self.launchpad = launchpad
+        self.image_url = image_url
 
         logging.info((f"Preparing launchpad API for {self.project_name} "
                       f"- {self.milestone_name}"))
@@ -45,11 +41,14 @@ class Summary:
         logging.info(f"Preparing WebDAV API as user {user}")
         self.webdav_client = Client(webdav_options)
 
-        if not image_name:
-            self.image_name, self.image_sha256 = self.get_image_info_from_mail_content()
-        else:
-            self.image_name = image_name
+        if self.image_url:
+            # Image name is the last element of the URL, e.g.:
+            # https://oem-share.canonical.com/partners/project/share/uc20/uc20-alpha/iot-blabla-x01-20201228-9.img.xz
+            # -> iot-blabla-x01-20201228-9.img.xz
+            self.image_name = self.image_url.split("/")[-1]
             self.image_sha256 = self.get_image_sha256()
+        else:
+            self.image_name, self.image_sha256 = self.get_image_info_from_mail_content()
         self.image_date = get_date_from_image_name(self.image_name)
         self.new_bugs = self.get_new_bugs()
         self.milestone_bugs = self.lp_milestone.searchTasks()
@@ -136,7 +135,7 @@ class Summary:
 
             image_name = image_path.strip().split("/")[-1]
         except Exception:
-            raise MailContentNotFoundError()
+            raise MailContentNotFoundError("the mail.content file is not found on the WebDAV server")
 
         logging.info(f"Image: {image_name}, SHA256: {image_sha256}")
         return image_name, image_sha256
@@ -144,7 +143,7 @@ class Summary:
     def get_image_sha256(self):
         """
         If the image information cannot be automatically parsed from mail.content,
-        user can manually input the image name, and this function will try to parse
+        user can manually input the image URL, and this function will try to parse
         the corresponding sha256sum file to retrieve the sha256 value.
         """
         buff = io.BytesIO()
@@ -156,20 +155,28 @@ class Summary:
             content = buff.read()
             sha256 = content.decode().split()[0]
         except Exception:
-            raise Sha256FileNotFoundError()
+            raise Sha256FileNotFoundError("sha256sum file for a given image is not found on the WebDAV server")
 
         return sha256
 
     def get_milestone_uri(self):
-        # For projects not using series (i.e. using the default "trunk"
-        # series), the default is to put the releases in a `releases` directory
-        # on WebDAV.
-        if self.series_name == "trunk":
-            uri = f"/share/{self.project_name}/releases/{self.milestone_name}/"
+        """Generate WebDAV URI for the selected milestone"""
+        # If the image URL is manually provided by user, we recreate the WebDAV URI from it, e.g.:
+        # https://oem-share.canonical.com/partners/project/share/uc-20/uc20-x01/iot-blabla-x01-20201228-9.img.xz
+        # -> /share/project/uc20/uc20-x01
+        if self.image_url:
+            url_list = self.image_url.split("/")
+            uri = f"/share/{self.project_name}/{url_list[-3]}/{url_list[-2]}/"
         else:
-            uri = (
-                f"/share/{self.project_name}/{self.series_name}/{self.milestone_name}/"
-            )
+            # For projects not using series (i.e. using the default "trunk"
+            # series), the default is to put the releases in a `releases` directory
+            # on WebDAV.
+            if self.series_name == "trunk":
+                uri = f"/share/{self.project_name}/releases/{self.milestone_name}/"
+            else:
+                uri = (
+                    f"/share/{self.project_name}/{self.series_name}/{self.milestone_name}/"
+                )
         return uri
 
 
diff --git a/webapp/__init__.py b/webapp/__init__.py
index befaea5..755a829 100644
--- a/webapp/__init__.py
+++ b/webapp/__init__.py
@@ -119,7 +119,7 @@ def generate_summary():
     logging.info("Route to summary page")
     project = request.form.get("project")
     milestone = request.form.get("milestone")
-    image_name = request.form.get("image_name")
+    image_url = request.form.get("image_url")
     hardware = request.form.get("hardware")
     stress_test_reports = request.form.get("stress_test_reports")
     full_test_reports = request.form.get("full_test_reports")
@@ -130,7 +130,7 @@ def generate_summary():
         return redirect(url_for("index"))
 
     try:
-        report = summary.Summary(launchpad, project, milestone, image_name=image_name)
+        report = summary.Summary(launchpad, project, milestone, image_url=image_url)
         critical_new_bugs = [b for b in report.new_bugs if b.importance == "Critical"]
         high_new_bugs = [b for b in report.new_bugs if b.importance == "High"]
         critical_reopened_bugs = [
@@ -156,7 +156,7 @@ def generate_summary():
         )
     except summary.Sha256FileNotFoundError:
         error = (
-            f"Could not find sha256 for image named {image_name}. "
+            f"Could not find sha256 for image at {image_url}. "
             f"Please make sure this image is available on oem-share."
         )
         logging.error(error)
diff --git a/webapp/static/style.css b/webapp/static/style.css
index 3ef3f96..b45b806 100644
--- a/webapp/static/style.css
+++ b/webapp/static/style.css
@@ -11,7 +11,7 @@ main {
 input, textarea, select {
     line-height: 1.5em;
     font-size: 1em;
-    width: 40em;
+    width: 50em;
     padding: 0.5em;
 }
 
diff --git a/webapp/templates/image_info.html b/webapp/templates/image_info.html
index 6828be6..eb76be1 100644
--- a/webapp/templates/image_info.html
+++ b/webapp/templates/image_info.html
@@ -17,15 +17,15 @@
   <p class="warning"><strong>Error:</strong> {{ error }}</p>
   {% endif %}
   <p>Auto-Summary relies on the release e-mail (stored on oem-share as a <code>mail.content</code> file) to retrieve the image name and its SHA256 value.</p>
-  <p>If you want, you can manually specify the image name, and Auto-Summary will try to figure out the rest.</p>
+  <p>If you want, you can manually specify the image URL, and Auto-Summary will try to figure out the rest.</p>
   <p>Note: In the normal process, a release e-mail should always be sent, and the <code>mail.content</code> file should always be available in the project's oem-share. If it is not the case, please contact the Software Engineering team to review the process.</p>
 
   <hr>
 <form action="{{ url_for('generate_summary') }}" method="post">
   <ul>
     <li>
-      <label for="html_report">Image name:</label>
-      <input type="text" id="image_name" name="image_name" placeholder="e.g. iot-fisterra-nitrogen6x-classic-server-2004-x01-20201228-9.img.xz">
+      <label for="html_report">Image URL:</label>
+      <input type="text" id="image_url" name="image_url" placeholder="e.g. https://oem-share.canonical.com/partners/project/share/uc20/uc20-x01/iot-project-uc20-x01-20201228-9.img.xz">
     </li>
     <li class="button">
       <button type="submit">Generate Summary</button>