Merge lp:~brian.curtin/ubuntuone-windows-installer/build_installer into lp:ubuntuone-windows-installer

[Brian Curtin]

- Automate the building and packaging of the Windows installer

This branch adds build_installer.py which fully automates the creation of the BitRock installer. It includes the typical setup.py steps of fetch, prepare, and py2exe, then it runs the BitRock autoupdate and installer creation. Upon creation of the installer, it timestamps the installer name so we can identify which installers were created by the automation on which date.

Also included in the branch are changes which make the automation easier. The VistaLib DLLs are downloaded from my U1 rather than forcing a user to create an account on CodeProject, sign in, then download the zip file from there. The files are placed directly into dist/ where they are needed, along with the required license file.

A change was made to the setup.py file to package the CRT alongside the binaries rather than requiring the vcredist installer to be packaged in the installer and run. It makes the installer a bit quicker and it makes automation much easier.

[Roberto Alsina (ralsina)]

+1 code review

review: Approve

[Manuel de la Peña (mandel)]

Here are some comments about the branch:

1. Imports are not in order, I personally don't care but I do know that members of the team like them to be in alphabetical order etc... can you please set them that way (I know, I know...)

2. _find_bitrock is a little fragile. The method is assuming that the package is going to be build in a x64 machine which might not be the case (I do not know the arch of the ec2 instance that runs jenkins). I think that a more 'robust' way to find bitrock will be looking in the registry. After installation I've noticed that you can find the installation path under HKEY_LOCAL_MACHINE\Software\BitRock\BitRock InstallBuilder Enterprise\Locatin (same path + the Wow6432Node in x64 machines). I think it is better approach over looking at the ProgramFiles(x86) path.

The rest of the code looks great to me (I'm really looking foward to add a +1 to this and get automated builds in jenkins).

review: Needs Fixing

[lp:~brian.curtin/ubuntuone-windows-installer/build_installer]

117. By Brian Curtin on 2012-05-17

Turn renaming off because Jenkins doesn't need a more unique path, just a predictable one. Removing the rename option also makes it easier to upload the resulting installer to a beta channel or something like that.

118. By Brian Curtin on 2012-05-17

Reorder imports

119. By Brian Curtin on 2012-05-17

Use the registry to determine BitRock's installation path rather than trying to discover it

[Manuel de la Peña (mandel)]

Superb!

review: Approve

=== added file 'scripts/build_installer.py'
--- scripts/build_installer.py	1970-01-01 00:00:00 +0000
+++ scripts/build_installer.py	2012-05-17 15:00:27 +0000
@@ -0,0 +1,99 @@
+from __future__ import print_function
+from collections import namedtuple
+import os
+import shutil
+import subprocess
+import sys
+import time
+
+try:
+    import _winreg as winreg
+except ImportError:
+    import winreg
+
+def _which_python():
+    for dir in os.environ.get("Path").split(os.pathsep):
+        full_path = os.path.join(dir, "python.exe")
+        if os.path.exists(full_path):
+            return full_path
+    return None
+
+BRCommand = namedtuple("BRCommand", ["bin", "args", "output"])
+
+# Since buildout's python.exe is really a script passed to the
+# system-installed Python, we need to get that same environment by starting
+# Python in the same manner.
+# Ex. ['c:\\Python27\\python.exe', 'C:\\u1\\bin\\python-script.py']
+PYTHON = [_which_python()]
+
+def _find_bitrock():
+    with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,
+            r"Software\BitRock\BitRock InstallBuilder Professional", 0,
+            winreg.KEY_READ | winreg.KEY_WOW64_32KEY) as key:
+        path, reg_type = winreg.QueryValueEx(key, "Location")
+        if reg_type != winreg.REG_SZ:
+            raise Exception(
+                "Got unexpected Location information: {}".format(reg_type))
+    return path if os.path.exists(path) else None
+
+def _find_output_dir(type):
+    user_dir = os.getenv("USERPROFILE")
+    if not user_dir:
+        return None
+    output = os.path.join(user_dir, "Documents", type, "output")
+    return output if os.path.exists(output) else None
+
+def _get_last_file(dir):
+    """Get the last written file in this directory. We just wrote it."""
+    files = os.listdir(dir)
+    # Sort the files by modification time.
+    files.sort(key=lambda f: os.stat(os.path.join(dir, f)).st_mtime)
+    # We want the most recently modified, so take the last one.
+    return os.path.join(dir, files[-1])
+
+def setup_command(cmd):
+    """Run a command through the setup.py script"""
+    print("Calling", cmd)
+    rv = subprocess.Popen(PYTHON + ["setup.py", cmd]).wait()
+    if rv != 0:
+        raise Exception("{} returned {}".format(cmd, rv))
+    print("{} returned {}".format(cmd, rv))
+
+def bitrock_command(cmd, rename=False):
+    """Run one of the BitRock binaries with some arguments"""
+    print("Running BitRock", cmd.bin)
+    subprocess.check_output(
+            [os.path.join(_find_bitrock(), cmd.bin)] + cmd.args,
+            stderr=subprocess.STDOUT)
+
+    source = _get_last_file(cmd.output)
+    target = os.path.join(os.path.dirname(__file__) or os.getcwd(),
+                          os.path.basename(source))
+
+    # Copy the installer locally
+    shutil.copy(source, target)
+    if rename:
+        orig_target = target
+        # Rename the installer to be timestamped.
+        root, ext = os.path.splitext(target)
+        timestamp = time.strftime("%Y%m%d-%H%M%S", time.gmtime())
+        target = "{}-{}{}".format(root, timestamp, ext)
+        os.rename(orig_target, target)
+    print("Created", target)
+
+def main():
+    for cmd in ["fetch", "prepare", "py2exe"]:
+        setup_command(cmd)
+
+    autoupdate = BRCommand(os.path.join("autoupdate", "bin", "customize.exe"),
+                           ["build", "ubuntuone_autoupdate.xml", "windows"],
+                           _find_output_dir("AutoUpdate"))
+    installer = BRCommand(os.path.join("bin", "builder-cli.exe"),
+                          ["build", "ubuntuone.xml"],
+                          _find_output_dir("InstallBuilder"))
+    for cmd, rename in ((autoupdate, False), (installer, False)):
+        bitrock_command(cmd, rename)
+
+if __name__ == "__main__":
+    main()
+

=== modified file 'scripts/setup.py'
--- scripts/setup.py	2012-03-23 21:02:45 +0000
+++ scripts/setup.py	2012-05-17 15:00:27 +0000
@@ -20,10 +20,12 @@
 import shutil
 import subprocess
 import sys
+import urllib
 
 from distutils import log
 from distutils.cmd import Command
 from distutils.core import setup
+from glob import glob
 
 import conf
 
@@ -330,6 +332,33 @@
         console.extend(windows)
         windows = []
 
+    # From http://www.py2exe.org/index.cgi/Tutorial#Step521
+    # Bundle our own copy of the CRT so we don't need to depend on the
+    # vcredist_x86.exe for automation, or as a step of installer.
+    data_files = [("Microsoft.VC90.CRT",
+                   glob(r"C:\Program Files (x86)"
+                        r"\Microsoft Visual Studio 9.0\vc\redist"
+                        r"\x86\Microsoft.VC90.CRT\*.*"))]
+
+    if not os.path.exists("dist"):
+        os.mkdir("dist")
+
+    # The VistaLib license requires that we distribute this file.
+    shutil.copy("README_nonelevated.txt", "dist")
+
+    vistalib32 = "http://ubuntuone.com/2jcKAeBwIw8joW8mEu5JYX"
+    vistalib64 = "http://ubuntuone.com/16h3eDF4NPCLIUFJeBYxxc"
+
+    for url, name in ((vistalib32, "VistaLib32.dll"),
+                      (vistalib64, "VistaLib64.dll")):
+        response = urllib.urlopen(url)
+        data = response.read()
+        if not data:
+            raise Exception("Unable to download %s" % URL)
+
+        with open(os.path.join("dist", name), "wb") as f:
+            f.write(data)
+
     # Build bundles
     setup(
         name='ubuntuone',
@@ -346,6 +375,7 @@
         },
         console=console,
         windows=windows,
+        data_files=data_files,
         options={
             'py2exe': {
                 'includes': ['google.protobuf.descriptor',

=== modified file 'scripts/ubuntuone.xml'
--- scripts/ubuntuone.xml	2012-04-05 19:45:42 +0000
+++ scripts/ubuntuone.xml	2012-05-17 15:00:27 +0000
@@ -94,9 +94,6 @@
                             <origin>data</origin>
                         </distributionDirectory>
                         <distributionFile>
-                            <origin>vcredist_x86.exe</origin>
-                        </distributionFile>
-                        <distributionFile>
                             <origin>ubuntu_one.ico</origin>
                         </distributionFile>
                     </distributionFileList>
@@ -151,34 +148,6 @@
                 </startMenuFileShortcut>
             </startMenuShortcutList>
         </component>
-        <component>
-            <name>vcruntime</name>
-            <canBeEdited>1</canBeEdited>
-            <selected>1</selected>
-            <show>1</show>
-            <folderList>
-                <folder>
-                    <destination>${installdir}</destination>
-                    <name>vcfiles</name>
-                    <platforms>all</platforms>
-                    <actionList>
-                        <runProgram>
-                            <abortOnError>0</abortOnError>
-                            <program>${installdir}/vcredist_x86.exe</program>
-                            <programArguments>/q
-</programArguments>
-                            <progressText>Installing C++ runtime</progressText>
-                            <showMessageOnError>0</showMessageOnError>
-                        </runProgram>
-                    </actionList>
-                    <distributionFileList>
-                        <distributionFile>
-                            <origin>vcredist_x86.exe</origin>
-                        </distributionFile>
-                    </distributionFileList>
-                </folder>
-            </folderList>
-        </component>
     </componentList>
     <initializationActionList>
         <setInstallerVariable name="msiexec" value=""/>