Merge lp:~le-chi-thu/lava-test/enabled-file-cache into lp:lava-test/0.0

[Le Chi Thu]

When using out of tree tests in LAVA, the master image have network access and install the out-of-tree tests in form of json files. These files will be cached and when boot to test image, the cached file will be use and no network access is needed.

[Zygmunt Krynicki (zyga)]

21 - try:
22 - stream = open(os.path.join(self.cache_dir, key), mode)
23 - yield stream
24 - finally:
25 - stream.close()
26
27 + stream = open(os.path.join(self.cache_dir, key), mode)
28 + return stream
29 +

Why is this change needed?

review: Needs Information

[lp:~le-chi-thu/lava-test/enabled-file-cache]

123. By Zygmunt Krynicki on 2012-03-06

Bump version to 0.5 dev

124. By Zygmunt Krynicki on 2012-03-06

Allow the user to pre-define analyzer_assigned_uuid

[Le Chi Thu (le-chi-thu)]

The yield statement is call one level up. You will get method is missing on generator object otherwise.

[Zygmunt Krynicki (zyga)]

> The yield statement is call one level up. You will get method is missing on
> generator object otherwise.

Yes, but it was supposed to be a context manger via @contextlib.contextmanager decorator (then yield is actually feeding the decorator and everything works as expected). I'd rather fix it than remove this as it prevents dangling URL connection objects.

[lp:~le-chi-thu/lava-test/enabled-file-cache]

125. By Zygmunt Krynicki on 2012-03-06

Make lava-test usable inside a virtualenv.

[Le Chi Thu (le-chi-thu)]

Made Cache.open_cached protected method

review: Needs Resubmitting

[Zygmunt Krynicki (zyga)]

 ChiThu, one more change please
 ChiThu, add an entry to doc/changes.rst
 ChiThu, and maybe rebase on top of trunk to make this easier
 ChiThu, ok?

[lp:~le-chi-thu/lava-test/enabled-file-cache]

127. By Le Chi Thu <email address hidden> <email address hidden> on 2012-03-08

Made Cache.open_cached protected method

128. By Le Chi Thu <email address hidden> <email address hidden> on 2012-03-08

Updated changes.rst file

[Zygmunt Krynicki (zyga)]

+1

review: Approve

=== modified file 'doc/changes.rst'
--- doc/changes.rst	2012-03-06 17:54:26 +0000
+++ doc/changes.rst	2012-03-08 10:16:18 +0000
@@ -1,6 +1,13 @@
 Version History
 ***************
 
+.. _version_0_6:
+
+Version 0.6
+===========
+
+* Enabled file caching. Use when access the out-of-tree json tests.
+
 .. _version_0_5:
 
 Version 0.5

=== modified file 'lava_test/utils.py'
--- lava_test/utils.py	2011-09-12 09:19:10 +0000
+++ lava_test/utils.py	2012-03-08 10:16:18 +0000
@@ -20,6 +20,7 @@
 import shutil
 import urllib2
 import urlparse
+import sys
 
 _fake_files = None
 _fake_paths = None
@@ -181,28 +182,27 @@
             cls._instance = cls()
         return cls._instance
 
-    def open_cached(self, key, mode="r"):
+    def _open_cached(self, key, mode="r"):
         """
         Acts like open() but the pathname is relative to the
         lava_test-specific cache directory.
         """
+
         if "w" in mode and not os.path.exists(self.cache_dir):
             os.makedirs(self.cache_dir)
         if os.path.isabs(key):
             raise ValueError("key cannot be an absolute path")
-        try:
-            stream = open(os.path.join(self.cache_dir, key), mode)
-            yield stream
-        finally:
-            stream.close()
 
+        stream = open(os.path.join(self.cache_dir, key), mode)
+        return stream
+    
     def _key_for_url(self, url):
         return hashlib.sha1(url).hexdigest()
 
     def _refresh_url_cache(self, key, url):
         with contextlib.nested(
             contextlib.closing(urllib2.urlopen(url)),
-            self.open_cached(key, "wb")) as (in_stream, out_stream):
+            self._open_cached(key, "wb")) as (in_stream, out_stream):
             out_stream.write(in_stream.read())
 
     @contextlib.contextmanager
@@ -212,18 +212,15 @@
         """
         # Do not cache local files, this is not what users would expect
 
-        # workaround - not using cache at all.
-        # TODO: fix this and use the cache
-        # if url.startswith("file://"):
-        if True:
+        if url.startswith("file://"):
             stream = urllib2.urlopen(url)
         else:
             key = self._key_for_url(url)
             try:
-                stream = self.open_cached(key, "rb")
+                stream = self._open_cached(key, "rb")
             except IOError:
                 self._refresh_url_cache(key, url)
-                stream = self.open_cached(key, "rb")
+                stream = self._open_cached(key, "rb")
         try:
             yield stream
         finally: