Merge lp:~jose/charms/precise/owncloud/port-change+repo+ssl-support into lp:charms/owncloud

Proposed by José Antonio Rey
Status: Merged
Merged at revision: 26
Proposed branch: lp:~jose/charms/precise/owncloud/port-change+repo+ssl-support
Merge into: lp:charms/owncloud
Diff against target: 2050 lines (+1592/-251)
18 files modified
README.md (+110/-33)
charm-helpers.yaml (+5/-0)
config.yaml (+28/-0)
hooks/charmhelpers/contrib/ssl/__init__.py (+78/-0)
hooks/charmhelpers/contrib/ssl/service.py (+267/-0)
hooks/charmhelpers/core/hookenv.py (+401/-0)
hooks/charmhelpers/core/host.py (+297/-0)
hooks/config-changed (+153/-9)
hooks/db-relation-changed (+9/-0)
hooks/db-relation-departed (+13/-0)
hooks/install (+45/-23)
hooks/ssl (+19/-0)
hooks/start (+6/-2)
hooks/upgrade-charm (+43/-39)
hooks/website-relation-joined (+1/-2)
metadata.yaml (+1/-1)
tests/100-deploy.py (+116/-0)
tests/100_deploy.test (+0/-142)
To merge this branch: bzr merge lp:~jose/charms/precise/owncloud/port-change+repo+ssl-support
Reviewer Review Type Date Requested Status
Charles Butler (community) Approve
Matt Bruzek (community) Needs Fixing
Review via email: mp+215527@code.launchpad.net

Commit message

Added port changing and repository install support, as well as SSL support by default.

Description of the change

These are fixes to bug #1161894, bug #1306756, bug #1310164 and bug #1315091.

Support has been added considering that it will not break previous installs of ownCloud: if a person has an older version and upgrade-charm's the instance(s) will not break.

To post a comment you must log in.
Revision history for this message
Matt Bruzek (mbruzek) wrote :
Download full text (3.2 KiB)

José,

Thanks for working on adding ssh to the Owncloud charm! This is my first experience with Owncloud and I am really excited about this charm!

The command “charm proof” passes with no errors! That is excellent, but I have come to understand that you pay attention to details like that.

I see you added charmhelpers to generate the selfsigned certificate. That all looks good to me.

The README.md file was updated, it looks very good but I found a few small errors:

“If you do not know what it is, execute `juju status` to find out the public DNS.”

The command “juju status” does not return a Domain Name Server (DNS). What you might have meant “... to find out the public domain name of the deployed unit.”

“This password will be used in case you want to deo a setup of Shared Instances. If not provided, it will be randomly generated when a DB relation is joined.”
The word “do” is misspelled here.

The readme file asserts that the charm will not be set up without setting the domain configuration option, which is fine. I tested the server address before setting domain and got the default page:

  It works!
  This is the default web page for this server.
  The web server software is running but no content has been added, yet.

I understand this is the default apache page has not been disabled at this point. What would be really awesome if the default page could provide the user with instructions on what needs to be done. The hook could write a small index.html to the www root that tells the user what is needed before the charm will set up and configure. Something like:

  The owncloud service is not fully configured. Set the domain value in Juju to configure owncloud.

Bugs found unrelated to this merge proposal:

The admin user name and password are immutable, even after adding a mysql database. I realize this is not a bug introduced by this merge proposal so I created a bug here:
https://bugs.launchpad.net/charms/+source/owncloud/+bug/1315047

We can track that topic separately on that bug report.

Removing the database relation generates the following error:

[1044] SQLSTATE[42000] [1044] Access denied for user 'paethaicieniaju'@'%' to database 'owncloud'

I opened a bug here:
https://bugs.launchpad.net/charms/+source/owncloud/+bug/1315091

I am really excited to see that the charm included tests! That is outstanding not many merge propoals have tests. However the test that I ran did not work.

$ ./tests/100_deploy.test
Traceback (most recent call last):
  File "./tests/100_deploy.test", line 25, in <module>
    domain = d.sentry.unit['haproxy/0'].info['public-address']
AttributeError: 'NoneType' object has no attribute 'unit'

I took a look at the test and it looks like an error in the amulet test code. The sentry objects are not created until after d.setup(...). You should move that domain and d.configure() to after the d.setup(...) call.

Thank you so much for your submission! Based on this test failure I have moved this bug to Incomplete. Once you have addressed the issues above either with code changes or comments, and wish to have another review, simply move the bug back to either "New" or "Fix Committed" to have it ad...

Read more...

Revision history for this message
Matt Bruzek (mbruzek) wrote :

The 100_deploy.test fails.

review: Needs Fixing
Revision history for this message
José Antonio Rey (jose) wrote :

As revision 24 was merged in, tests no longer work due to many different factors. There is currently an open bug for it as they need to be updated.

Revision history for this message
Ted Gould (ted) wrote :

Not sure what is blocking this MR from the comments. I'd really like to use these features in the owncloud charm, any chance it could get included? Thanks!

Revision history for this message
José Antonio Rey (jose) wrote :

Ted,

Tests are blocking this MP from being approved. I'll be talking to the person in charge of them to see what can I do to speed up the process. Rest assured that this will be in the Charm Store soon.

Revision history for this message
Matt Bruzek (mbruzek) wrote :

José,

Thank you for following up with the Owncloud charm. Your fixes since my last review have noted and for the most part are great! Keep up the excellent work.

I took some time to review this Merge Proposal again today. Here is what I found.

The Amulet tests still do not pass but since the tests did not pass before this MP that is OK. The tests should pass eventually, so I created a bug to track that work here: https://bugs.launchpad.net/charms/+source/owncloud/+bug/1320324

I am still concerned about the immutable configuration options user and password that was not changed by this MP so the bug can be found here: https://bugs.launchpad.net/charms/+source/owncloud/+bug/1315047

hooks/website-relation-joined

There is an invalid bash variable in there $80. I assume you meant to always set the port to 80. Since the port is a configuration option I think you want to call config-get port to get the port and set it properly for the website relation.

Based on the error in the hook I can not approve this change at this time. Hopefully it is an easy fix and we can move to the next step quickly, as I am genuinely interested in getting owncloud in the charm store!

Thank you so much for the work here, and being patient with the review process. I have moved this proposal to incomplete as a result of the review. Once you have addressed the issue and want another review please set this proposal back to “needs review” and it will be added to the review queue.

review: Needs Fixing
Revision history for this message
Matt Bruzek (mbruzek) wrote :

The final paragraph of my last statement was incorrect. My apologies on this.

Once the problem has been addressed, lick on the “Request another review” link on this merge proposal. That way it will be added to the review queue properly.

If you have any questions/comments/concerns about the review contact us
in #juju on irc.freenode.net or email the mailing list
<email address hidden>

Revision history for this message
Matt Bruzek (mbruzek) wrote :

I took some time to review the latest submission. The amulet test failed with the error message “Unable to validate NFS config 10.0.3.163” on my local system. The tests need to run without errors before we can accept the merge proposal.

The upgrade-charm code needs some improvements. Please use https:// when ever possible with wget, that verifies the identity of the server that you are downloading binaries from. Please change or add https to all wget statements where possible.

The while loop in updgrade-charm has a couple of issues that we would like to see fixed. If you are hardcoding the version of owncloud, you should hardcode the md5sum in the charm code, this prevents Man In the Middle attacks from faking a payload and generating an md5sum with the right hash value of the payload. Also it is possible that the while loop would never exit if the md5 sum was incorrect or if the download was unavailable. The code should fail if wget fails to download the tar file, not loop forever.

In the config-changed hook it appears that the if-elif only sets the user OR the password not both. For instance if the user equals the contents of .admin the password code can not be executed. Please revisit this code and fix the either or situation.

Thanks again for the submission. I am going to put this MP to "needs fixing" and when you are ready for another review please click the "Request another review" button in the upper right hand corner of the commit message.

review: Needs Fixing
38. By José Antonio Rey

Fixed various bugs

Revision history for this message
José Antonio Rey (jose) wrote :

Matt,

All the issues have now been fixed, except the NFS one. This is due to NFS failing on the local provider, it's not test-specific.

On the other hand, I added a section on the README specifying to which websites and ports the charm needs to connect in order to successfully deploy, as this is not a fat charm.

Revision history for this message
Matt Bruzek (mbruzek) wrote :

José,

Thank you for your submission for owncloud! Your continued dedication to the owncloud charm is very much appreciated.

I took some time to review the latest submission. The amulet test failed on hp-cloud with the error message:

HTTPSConnectionPool(host='15.125.126.129', port=443): Max retries exceeded with url: /index.php

We spoke about this and I understand this is because the 'domain' configuration value is not set after the deployment. We should try to set that and re-run the tests.

The tests/00_setup.sh tests need the executable flag.

Thank you for fixing the bugs I pointed out in the previous review. Also I really like the way you wrote the README.md with the Internet Connection Requirements section! Very well done sir, I think this will be a model for future charm README files.

config-changed:

I noticed something very strange in the config-changed hook:
sed -i 's/;upload_max_filesize/;upload_max_filesize/g' /etc/php5/apache2/php.ini

This seems to be searching for a term and replacing the exact same term. I believe this to be a mistake and it should be searching for '^upload_max_filesize” and replacing with the commented out version.

I realize this is not in your MP but please fix this problem.

Thanks for your time and work on this owncloud submission! I am going to put this MP to "needs fixing" and when you are ready for another review please click the "Request another review" button in the upper right hand corner of the commit message.

review: Needs Fixing
39. By José Antonio Rey

Fixed default behaviour when domain wasn't set and made setup test +x

Revision history for this message
José Antonio Rey (jose) wrote :

Hey Matt,

All of this has been fixed as of the latest revision of the branch.

40. By José Antonio Rey

Updated the author's email address

41. By José Antonio Rey

Updated Release key sha1sum, updated source to version 6.0.4

42. By José Antonio Rey

Updated README, fixed idempotency on src option

Revision history for this message
Charles Butler (lazypower) wrote :

hey Jose,

Thanks for the continued effort on getting this merge prepared for the charm store.

I've had a chance to review, and per our feedback in IRC i see that you've added configuration options when using the deploy from source option. This will help aleviate additional stress on you as the charm maintainer when users want to run from more recent copies upstream, and the deployment routine hasn't changed.

I've taken some time to grease the wheels and deploy owncloud in it's various configurations. Ideally I'd like to see more tests against this charm with those various deployment models, such as generating SSH keys post deployment, deploying with SSH keys as a config option, and switching up the deployment model from DEBS to SOURCE, and back post deployment.

Otherwise I've run several deployments and they all came back good for me on my MAAS host and AWS.

I'm going to approve this, but caution you to break up future merges into smaller digestable chunks as this was a fairly large review that ultimately delayed the merge from happening as quickly as we would like.

Thanks again for such dedication and great work.

+1 LGTM

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'README.md'
2--- README.md 2014-04-11 13:44:05 +0000
3+++ README.md 2014-07-21 22:01:08 +0000
4@@ -1,7 +1,7 @@
5 # OwnCloud
6
7-- Author: Atul Jha <atul.jha@csscorp.com>
8-- Maintainer: Nathan Williams <nathan@nathanewilliams.com>
9+- Author: Atul Jha <koolhead17@gmail.com>
10+- Maintainer: José Antonio Rey <jose@ubuntu.com>
11
12 # Overview
13
14@@ -14,32 +14,76 @@
15 sqlite as a standalone server. Provides relationship hooks with NFS file
16 storage, MySQL Databases, and HAProxy reverse proxy charms.
17
18-## Preparation:
19-
20-1. the charm comes with port, user, password configuration options.
21-
22- if no user is provided, the administrator will be "owncloud".
23-
24- if no password is provided, a random one is chosen during the
25- db-relation-joined hook. if you do this, you can obtain the auth
26- credentials from the juju logs.
27-
28-# Usage:
29-
30-#### 1. Install
31+# Configuration
32+
33+This charm comes with different configuration options. Optional configuration
34+options include:
35+
36+`domain`: This is the domain or IP address for your ownCloud instance. If you
37+do not know what it is, execute `juju status` to find out the public address. If
38+not provided, the charm will refuse to configure.
39+
40+`downloadurl`: This is the download URL that the charm will use in case the src
41+option is set to `source`. It defaults to the 6.0.4 URL.
42+
43+`sha1sum`: This is the SHA1SUM for the `downloadurl` file. It defaults to the
44+SHA1SUM for the 6.0.4 file.
45+
46+`port`: This is the alias port that will be used for the ownCloud instance. It
47+will redirect to 443, which is the HTTPS port. It defaults to 80.
48+
49+`src`: This is the source from which the package will be installed. You can
50+choose between `repo`, which will install it from a repository built by
51+ownCloud, or `source`, which will download the tarball and extract it. It
52+defaults to `repo`.
53+
54+`customssl`: This charm provides default SSL support for ownCloud. This means
55+that if you do not provide a custom SSL key and certificate, a self-signed one
56+will be auto-generated for you. If you want to use a custom SSL certificate,
57+please set this option to `true`. It defaults to `false`.
58+
59+`sslkey`: If `customssl` is set to true, this is the SSL key that will be used.
60+If not provided and `customssl` is true, the charm will refuse to configure.
61+
62+`sslcert`: If `customssl` is set to true, this is the SSL cert that will be
63+used. If not provided and `customssl` is true, the charm will refuse to
64+configure.
65+
66+`user`: This user will be used in case you want to do a setup of Shared
67+Instances. If not provided, it will be defaulted to `ownCloud`.
68+
69+`password`: This password will be used in case you want to do a setup of
70+Shared Instances. If not provided, it will be randomly generated when a DB
71+relation is joined.
72+
73+You can put any of this options in a config.yaml file and specify it at the
74+moment of deploying. Otherwise, you can change them by executing:
75+
76+ juju set owncloud [option]=[value]
77+
78+# Usage
79+
80+## Standalone Instance
81+
82+First, bootstrap your environment:
83+
84+ juju bootstrap
85+
86+Then, deploy ownCloud by executing the following command:
87
88 juju deploy owncloud
89
90-#### 2. Expose
91+Finally, expose the service:
92
93 juju expose owncloud
94
95-#### 3a. Standalone Instance
96-
97- Access OwnCloud service directly, and complete the setup, providing user
98- credentials and initializing sqlite database.
99-
100-#### 3b. Shared Instances
101+Access OwnCloud service directly, and complete the setup, providing user
102+credentials and initializing sqlite database.
103+
104+## Shared Instances
105+
106+If you want to deploy shared instances, execute the following commands after
107+doing a Standalone Instance setup:
108
109 juju deploy mysql
110 juju add-relation mysql owncloud
111@@ -47,15 +91,14 @@
112 juju deploy nfs
113 juju add-relation nfs owncloud
114
115-#### 4. Access
116-
117-http://$owncloud-machine-addr/. To find out the public address of ownCloud,
118-look for it in the output of the `juju status` command.
119-
120-
121-## Scale out Usage
122-
123-
124+We're now done! To find out the address for your ownCloud instance, execute
125+`juju status` and navigate to it.
126+
127+# Scale out Usage
128+
129+In order to do a scalabe deploy of ownCloud, execute the following commands
130+
131+ juju bootstrap
132 juju deploy owncloud
133 juju deploy mysql
134 juju deploy haproxy
135@@ -63,13 +106,47 @@
136 juju add-relation haproxy owncloud
137 juju add-unit owncloud
138
139-
140-## Known Limitations
141+# Internet Connection Requirements
142+
143+This charm downloads files from the Internet, and requires Internet connectivity
144+in order to properly install. The requirements vary for each setup type.
145+
146+## When installing from the source
147+
148+When installing from the source packages available for download, this charm will
149+connect to the following Internet sites:
150+
151+ * download.owncloud.org with port 443
152+ * The Ubuntu repositories or a private mirror of them
153+
154+## When installing from the repository
155+
156+ownCloud offers the option to install from a repository. This is the default
157+configuration value for the charm. With this, the charm will connect to the
158+following Internet sites:
159+
160+ * download.opensuse.org with port 80
161+ * download.opensuse.org as a repository
162+ * The Ubuntu repositories or a private mirror of them
163+
164+# Known Limitations
165
166 If you have been using a standalone instance and want to migrate to a shared
167 instance, please note that adding the mysql relation will not preserve the file
168 structure in the database. This means that your file listing will not be
169 available. Make sure to have this in mind when doing the migration.
170
171+Also, if you leave the `customssl` option set to false or provide a self-signed
172+SSL certificate, ownCloud will throw a WebDAV error after creating the admin
173+username and password. Ignore this error as it does not affect the working of
174+ownCloud (it is silently fixed), and enter your website again.
175+
176+If port is different than 80, it looks like the instance throws an SSL
177+error when connecting. Recommendation is to set the `port` value to 80 to avoid
178+problems.
179+
180+Finally, on the tests side, the tests will fail on the local provider due to
181+NFS not being able to deploy correctly (this is an NFS-related issue).
182+
183 #TODO
184 Genericize shared-fs-relation-* for non-nfs shared-fs providers
185
186=== added file 'charm-helpers.yaml'
187--- charm-helpers.yaml 1970-01-01 00:00:00 +0000
188+++ charm-helpers.yaml 2014-07-21 22:01:08 +0000
189@@ -0,0 +1,5 @@
190+destination: hooks/charmhelpers
191+branch: lp:charm-helpers
192+include:
193+ - contrib.ssl
194+ - core
195
196=== modified file 'config.yaml'
197--- config.yaml 2014-01-28 15:41:38 +0000
198+++ config.yaml 2014-07-21 22:01:08 +0000
199@@ -1,4 +1,16 @@
200 options:
201+ domain:
202+ type: string
203+ default: ""
204+ description: the domain name for your owncloud server
205+ downloadurl:
206+ type: string
207+ default: "https://download.owncloud.org/community/owncloud-6.0.4.tar.bz2"
208+ description: url from which owncloud will be downloaded
209+ sha1sum:
210+ type: string
211+ default: "6e341aeba2cf99416de009c7862bf03d90d1b058"
212+ description: the sha1sum for the file in the download link
213 port:
214 type: int
215 default: 80
216@@ -11,3 +23,19 @@
217 type: string
218 default: ""
219 description: default administrative password
220+ src:
221+ type: string
222+ default: "repo"
223+ description: source from where the charm will install the package
224+ customssl:
225+ type: boolean
226+ default: false
227+ description: option to set if custom ssl certificates are going to be used
228+ sslcert:
229+ type: string
230+ default: ""
231+ description: ssl cert to be used if custom is on
232+ sslkey:
233+ type: string
234+ default: ""
235+ description: ssl key to be used if custom is on
236
237=== added directory 'hooks/charmhelpers'
238=== added file 'hooks/charmhelpers/__init__.py'
239=== added directory 'hooks/charmhelpers/contrib'
240=== added file 'hooks/charmhelpers/contrib/__init__.py'
241=== added directory 'hooks/charmhelpers/contrib/ssl'
242=== added file 'hooks/charmhelpers/contrib/ssl/__init__.py'
243--- hooks/charmhelpers/contrib/ssl/__init__.py 1970-01-01 00:00:00 +0000
244+++ hooks/charmhelpers/contrib/ssl/__init__.py 2014-07-21 22:01:08 +0000
245@@ -0,0 +1,78 @@
246+import subprocess
247+from charmhelpers.core import hookenv
248+
249+
250+def generate_selfsigned(keyfile, certfile, keysize="1024", config=None, subject=None, cn=None):
251+ """Generate selfsigned SSL keypair
252+
253+ You must provide one of the 3 optional arguments:
254+ config, subject or cn
255+ If more than one is provided the leftmost will be used
256+
257+ Arguments:
258+ keyfile -- (required) full path to the keyfile to be created
259+ certfile -- (required) full path to the certfile to be created
260+ keysize -- (optional) SSL key length
261+ config -- (optional) openssl configuration file
262+ subject -- (optional) dictionary with SSL subject variables
263+ cn -- (optional) cerfificate common name
264+
265+ Required keys in subject dict:
266+ cn -- Common name (eq. FQDN)
267+
268+ Optional keys in subject dict
269+ country -- Country Name (2 letter code)
270+ state -- State or Province Name (full name)
271+ locality -- Locality Name (eg, city)
272+ organization -- Organization Name (eg, company)
273+ organizational_unit -- Organizational Unit Name (eg, section)
274+ email -- Email Address
275+ """
276+
277+ cmd = []
278+ if config:
279+ cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
280+ "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
281+ "-keyout", keyfile,
282+ "-out", certfile, "-config", config]
283+ elif subject:
284+ ssl_subject = ""
285+ if "country" in subject:
286+ ssl_subject = ssl_subject + "/C={}".format(subject["country"])
287+ if "state" in subject:
288+ ssl_subject = ssl_subject + "/ST={}".format(subject["state"])
289+ if "locality" in subject:
290+ ssl_subject = ssl_subject + "/L={}".format(subject["locality"])
291+ if "organization" in subject:
292+ ssl_subject = ssl_subject + "/O={}".format(subject["organization"])
293+ if "organizational_unit" in subject:
294+ ssl_subject = ssl_subject + "/OU={}".format(subject["organizational_unit"])
295+ if "cn" in subject:
296+ ssl_subject = ssl_subject + "/CN={}".format(subject["cn"])
297+ else:
298+ hookenv.log("When using \"subject\" argument you must "
299+ "provide \"cn\" field at very least")
300+ return False
301+ if "email" in subject:
302+ ssl_subject = ssl_subject + "/emailAddress={}".format(subject["email"])
303+
304+ cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
305+ "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
306+ "-keyout", keyfile,
307+ "-out", certfile, "-subj", ssl_subject]
308+ elif cn:
309+ cmd = ["/usr/bin/openssl", "req", "-new", "-newkey",
310+ "rsa:{}".format(keysize), "-days", "365", "-nodes", "-x509",
311+ "-keyout", keyfile,
312+ "-out", certfile, "-subj", "/CN={}".format(cn)]
313+
314+ if not cmd:
315+ hookenv.log("No config, subject or cn provided,"
316+ "unable to generate self signed SSL certificates")
317+ return False
318+ try:
319+ subprocess.check_call(cmd)
320+ return True
321+ except Exception as e:
322+ print "Execution of openssl command failed:\n{}".format(e)
323+ return False
324
325=== added file 'hooks/charmhelpers/contrib/ssl/service.py'
326--- hooks/charmhelpers/contrib/ssl/service.py 1970-01-01 00:00:00 +0000
327+++ hooks/charmhelpers/contrib/ssl/service.py 2014-07-21 22:01:08 +0000
328@@ -0,0 +1,267 @@
329+import logging
330+import os
331+from os.path import join as path_join
332+from os.path import exists
333+import subprocess
334+
335+
336+log = logging.getLogger("service_ca")
337+
338+logging.basicConfig(level=logging.DEBUG)
339+
340+STD_CERT = "standard"
341+
342+# Mysql server is fairly picky about cert creation
343+# and types, spec its creation separately for now.
344+MYSQL_CERT = "mysql"
345+
346+
347+class ServiceCA(object):
348+
349+ default_expiry = str(365 * 2)
350+ default_ca_expiry = str(365 * 6)
351+
352+ def __init__(self, name, ca_dir, cert_type=STD_CERT):
353+ self.name = name
354+ self.ca_dir = ca_dir
355+ self.cert_type = cert_type
356+
357+ ###############
358+ # Hook Helper API
359+ @staticmethod
360+ def get_ca(type=STD_CERT):
361+ service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
362+ ca_path = os.path.join(os.environ['CHARM_DIR'], 'ca')
363+ ca = ServiceCA(service_name, ca_path, type)
364+ ca.init()
365+ return ca
366+
367+ @classmethod
368+ def get_service_cert(cls, type=STD_CERT):
369+ service_name = os.environ['JUJU_UNIT_NAME'].split('/')[0]
370+ ca = cls.get_ca()
371+ crt, key = ca.get_or_create_cert(service_name)
372+ return crt, key, ca.get_ca_bundle()
373+
374+ ###############
375+
376+ def init(self):
377+ log.debug("initializing service ca")
378+ if not exists(self.ca_dir):
379+ self._init_ca_dir(self.ca_dir)
380+ self._init_ca()
381+
382+ @property
383+ def ca_key(self):
384+ return path_join(self.ca_dir, 'private', 'cacert.key')
385+
386+ @property
387+ def ca_cert(self):
388+ return path_join(self.ca_dir, 'cacert.pem')
389+
390+ @property
391+ def ca_conf(self):
392+ return path_join(self.ca_dir, 'ca.cnf')
393+
394+ @property
395+ def signing_conf(self):
396+ return path_join(self.ca_dir, 'signing.cnf')
397+
398+ def _init_ca_dir(self, ca_dir):
399+ os.mkdir(ca_dir)
400+ for i in ['certs', 'crl', 'newcerts', 'private']:
401+ sd = path_join(ca_dir, i)
402+ if not exists(sd):
403+ os.mkdir(sd)
404+
405+ if not exists(path_join(ca_dir, 'serial')):
406+ with open(path_join(ca_dir, 'serial'), 'wb') as fh:
407+ fh.write('02\n')
408+
409+ if not exists(path_join(ca_dir, 'index.txt')):
410+ with open(path_join(ca_dir, 'index.txt'), 'wb') as fh:
411+ fh.write('')
412+
413+ def _init_ca(self):
414+ """Generate the root ca's cert and key.
415+ """
416+ if not exists(path_join(self.ca_dir, 'ca.cnf')):
417+ with open(path_join(self.ca_dir, 'ca.cnf'), 'wb') as fh:
418+ fh.write(
419+ CA_CONF_TEMPLATE % (self.get_conf_variables()))
420+
421+ if not exists(path_join(self.ca_dir, 'signing.cnf')):
422+ with open(path_join(self.ca_dir, 'signing.cnf'), 'wb') as fh:
423+ fh.write(
424+ SIGNING_CONF_TEMPLATE % (self.get_conf_variables()))
425+
426+ if exists(self.ca_cert) or exists(self.ca_key):
427+ raise RuntimeError("Initialized called when CA already exists")
428+ cmd = ['openssl', 'req', '-config', self.ca_conf,
429+ '-x509', '-nodes', '-newkey', 'rsa',
430+ '-days', self.default_ca_expiry,
431+ '-keyout', self.ca_key, '-out', self.ca_cert,
432+ '-outform', 'PEM']
433+ output = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
434+ log.debug("CA Init:\n %s", output)
435+
436+ def get_conf_variables(self):
437+ return dict(
438+ org_name="juju",
439+ org_unit_name="%s service" % self.name,
440+ common_name=self.name,
441+ ca_dir=self.ca_dir)
442+
443+ def get_or_create_cert(self, common_name):
444+ if common_name in self:
445+ return self.get_certificate(common_name)
446+ return self.create_certificate(common_name)
447+
448+ def create_certificate(self, common_name):
449+ if common_name in self:
450+ return self.get_certificate(common_name)
451+ key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
452+ crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
453+ csr_p = path_join(self.ca_dir, "certs", "%s.csr" % common_name)
454+ self._create_certificate(common_name, key_p, csr_p, crt_p)
455+ return self.get_certificate(common_name)
456+
457+ def get_certificate(self, common_name):
458+ if not common_name in self:
459+ raise ValueError("No certificate for %s" % common_name)
460+ key_p = path_join(self.ca_dir, "certs", "%s.key" % common_name)
461+ crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
462+ with open(crt_p) as fh:
463+ crt = fh.read()
464+ with open(key_p) as fh:
465+ key = fh.read()
466+ return crt, key
467+
468+ def __contains__(self, common_name):
469+ crt_p = path_join(self.ca_dir, "certs", "%s.crt" % common_name)
470+ return exists(crt_p)
471+
472+ def _create_certificate(self, common_name, key_p, csr_p, crt_p):
473+ template_vars = self.get_conf_variables()
474+ template_vars['common_name'] = common_name
475+ subj = '/O=%(org_name)s/OU=%(org_unit_name)s/CN=%(common_name)s' % (
476+ template_vars)
477+
478+ log.debug("CA Create Cert %s", common_name)
479+ cmd = ['openssl', 'req', '-sha1', '-newkey', 'rsa:2048',
480+ '-nodes', '-days', self.default_expiry,
481+ '-keyout', key_p, '-out', csr_p, '-subj', subj]
482+ subprocess.check_call(cmd)
483+ cmd = ['openssl', 'rsa', '-in', key_p, '-out', key_p]
484+ subprocess.check_call(cmd)
485+
486+ log.debug("CA Sign Cert %s", common_name)
487+ if self.cert_type == MYSQL_CERT:
488+ cmd = ['openssl', 'x509', '-req',
489+ '-in', csr_p, '-days', self.default_expiry,
490+ '-CA', self.ca_cert, '-CAkey', self.ca_key,
491+ '-set_serial', '01', '-out', crt_p]
492+ else:
493+ cmd = ['openssl', 'ca', '-config', self.signing_conf,
494+ '-extensions', 'req_extensions',
495+ '-days', self.default_expiry, '-notext',
496+ '-in', csr_p, '-out', crt_p, '-subj', subj, '-batch']
497+ log.debug("running %s", " ".join(cmd))
498+ subprocess.check_call(cmd)
499+
500+ def get_ca_bundle(self):
501+ with open(self.ca_cert) as fh:
502+ return fh.read()
503+
504+
505+CA_CONF_TEMPLATE = """
506+[ ca ]
507+default_ca = CA_default
508+
509+[ CA_default ]
510+dir = %(ca_dir)s
511+policy = policy_match
512+database = $dir/index.txt
513+serial = $dir/serial
514+certs = $dir/certs
515+crl_dir = $dir/crl
516+new_certs_dir = $dir/newcerts
517+certificate = $dir/cacert.pem
518+private_key = $dir/private/cacert.key
519+RANDFILE = $dir/private/.rand
520+default_md = default
521+
522+[ req ]
523+default_bits = 1024
524+default_md = sha1
525+
526+prompt = no
527+distinguished_name = ca_distinguished_name
528+
529+x509_extensions = ca_extensions
530+
531+[ ca_distinguished_name ]
532+organizationName = %(org_name)s
533+organizationalUnitName = %(org_unit_name)s Certificate Authority
534+
535+
536+[ policy_match ]
537+countryName = optional
538+stateOrProvinceName = optional
539+organizationName = match
540+organizationalUnitName = optional
541+commonName = supplied
542+
543+[ ca_extensions ]
544+basicConstraints = critical,CA:true
545+subjectKeyIdentifier = hash
546+authorityKeyIdentifier = keyid:always, issuer
547+keyUsage = cRLSign, keyCertSign
548+"""
549+
550+
551+SIGNING_CONF_TEMPLATE = """
552+[ ca ]
553+default_ca = CA_default
554+
555+[ CA_default ]
556+dir = %(ca_dir)s
557+policy = policy_match
558+database = $dir/index.txt
559+serial = $dir/serial
560+certs = $dir/certs
561+crl_dir = $dir/crl
562+new_certs_dir = $dir/newcerts
563+certificate = $dir/cacert.pem
564+private_key = $dir/private/cacert.key
565+RANDFILE = $dir/private/.rand
566+default_md = default
567+
568+[ req ]
569+default_bits = 1024
570+default_md = sha1
571+
572+prompt = no
573+distinguished_name = req_distinguished_name
574+
575+x509_extensions = req_extensions
576+
577+[ req_distinguished_name ]
578+organizationName = %(org_name)s
579+organizationalUnitName = %(org_unit_name)s machine resources
580+commonName = %(common_name)s
581+
582+[ policy_match ]
583+countryName = optional
584+stateOrProvinceName = optional
585+organizationName = match
586+organizationalUnitName = optional
587+commonName = supplied
588+
589+[ req_extensions ]
590+basicConstraints = CA:false
591+subjectKeyIdentifier = hash
592+authorityKeyIdentifier = keyid:always, issuer
593+keyUsage = digitalSignature, keyEncipherment, keyAgreement
594+extendedKeyUsage = serverAuth, clientAuth
595+"""
596
597=== added directory 'hooks/charmhelpers/core'
598=== added file 'hooks/charmhelpers/core/__init__.py'
599=== added file 'hooks/charmhelpers/core/hookenv.py'
600--- hooks/charmhelpers/core/hookenv.py 1970-01-01 00:00:00 +0000
601+++ hooks/charmhelpers/core/hookenv.py 2014-07-21 22:01:08 +0000
602@@ -0,0 +1,401 @@
603+"Interactions with the Juju environment"
604+# Copyright 2013 Canonical Ltd.
605+#
606+# Authors:
607+# Charm Helpers Developers <juju@lists.ubuntu.com>
608+
609+import os
610+import json
611+import yaml
612+import subprocess
613+import sys
614+import UserDict
615+from subprocess import CalledProcessError
616+
617+CRITICAL = "CRITICAL"
618+ERROR = "ERROR"
619+WARNING = "WARNING"
620+INFO = "INFO"
621+DEBUG = "DEBUG"
622+MARKER = object()
623+
624+cache = {}
625+
626+
627+def cached(func):
628+ """Cache return values for multiple executions of func + args
629+
630+ For example:
631+
632+ @cached
633+ def unit_get(attribute):
634+ pass
635+
636+ unit_get('test')
637+
638+ will cache the result of unit_get + 'test' for future calls.
639+ """
640+ def wrapper(*args, **kwargs):
641+ global cache
642+ key = str((func, args, kwargs))
643+ try:
644+ return cache[key]
645+ except KeyError:
646+ res = func(*args, **kwargs)
647+ cache[key] = res
648+ return res
649+ return wrapper
650+
651+
652+def flush(key):
653+ """Flushes any entries from function cache where the
654+ key is found in the function+args """
655+ flush_list = []
656+ for item in cache:
657+ if key in item:
658+ flush_list.append(item)
659+ for item in flush_list:
660+ del cache[item]
661+
662+
663+def log(message, level=None):
664+ """Write a message to the juju log"""
665+ command = ['juju-log']
666+ if level:
667+ command += ['-l', level]
668+ command += [message]
669+ subprocess.call(command)
670+
671+
672+class Serializable(UserDict.IterableUserDict):
673+ """Wrapper, an object that can be serialized to yaml or json"""
674+
675+ def __init__(self, obj):
676+ # wrap the object
677+ UserDict.IterableUserDict.__init__(self)
678+ self.data = obj
679+
680+ def __getattr__(self, attr):
681+ # See if this object has attribute.
682+ if attr in ("json", "yaml", "data"):
683+ return self.__dict__[attr]
684+ # Check for attribute in wrapped object.
685+ got = getattr(self.data, attr, MARKER)
686+ if got is not MARKER:
687+ return got
688+ # Proxy to the wrapped object via dict interface.
689+ try:
690+ return self.data[attr]
691+ except KeyError:
692+ raise AttributeError(attr)
693+
694+ def __getstate__(self):
695+ # Pickle as a standard dictionary.
696+ return self.data
697+
698+ def __setstate__(self, state):
699+ # Unpickle into our wrapper.
700+ self.data = state
701+
702+ def json(self):
703+ """Serialize the object to json"""
704+ return json.dumps(self.data)
705+
706+ def yaml(self):
707+ """Serialize the object to yaml"""
708+ return yaml.dump(self.data)
709+
710+
711+def execution_environment():
712+ """A convenient bundling of the current execution context"""
713+ context = {}
714+ context['conf'] = config()
715+ if relation_id():
716+ context['reltype'] = relation_type()
717+ context['relid'] = relation_id()
718+ context['rel'] = relation_get()
719+ context['unit'] = local_unit()
720+ context['rels'] = relations()
721+ context['env'] = os.environ
722+ return context
723+
724+
725+def in_relation_hook():
726+ """Determine whether we're running in a relation hook"""
727+ return 'JUJU_RELATION' in os.environ
728+
729+
730+def relation_type():
731+ """The scope for the current relation hook"""
732+ return os.environ.get('JUJU_RELATION', None)
733+
734+
735+def relation_id():
736+ """The relation ID for the current relation hook"""
737+ return os.environ.get('JUJU_RELATION_ID', None)
738+
739+
740+def local_unit():
741+ """Local unit ID"""
742+ return os.environ['JUJU_UNIT_NAME']
743+
744+
745+def remote_unit():
746+ """The remote unit for the current relation hook"""
747+ return os.environ['JUJU_REMOTE_UNIT']
748+
749+
750+def service_name():
751+ """The name service group this unit belongs to"""
752+ return local_unit().split('/')[0]
753+
754+
755+def hook_name():
756+ """The name of the currently executing hook"""
757+ return os.path.basename(sys.argv[0])
758+
759+
760+@cached
761+def config(scope=None):
762+ """Juju charm configuration"""
763+ config_cmd_line = ['config-get']
764+ if scope is not None:
765+ config_cmd_line.append(scope)
766+ config_cmd_line.append('--format=json')
767+ try:
768+ return json.loads(subprocess.check_output(config_cmd_line))
769+ except ValueError:
770+ return None
771+
772+
773+@cached
774+def relation_get(attribute=None, unit=None, rid=None):
775+ """Get relation information"""
776+ _args = ['relation-get', '--format=json']
777+ if rid:
778+ _args.append('-r')
779+ _args.append(rid)
780+ _args.append(attribute or '-')
781+ if unit:
782+ _args.append(unit)
783+ try:
784+ return json.loads(subprocess.check_output(_args))
785+ except ValueError:
786+ return None
787+ except CalledProcessError, e:
788+ if e.returncode == 2:
789+ return None
790+ raise
791+
792+
793+def relation_set(relation_id=None, relation_settings={}, **kwargs):
794+ """Set relation information for the current unit"""
795+ relation_cmd_line = ['relation-set']
796+ if relation_id is not None:
797+ relation_cmd_line.extend(('-r', relation_id))
798+ for k, v in (relation_settings.items() + kwargs.items()):
799+ if v is None:
800+ relation_cmd_line.append('{}='.format(k))
801+ else:
802+ relation_cmd_line.append('{}={}'.format(k, v))
803+ subprocess.check_call(relation_cmd_line)
804+ # Flush cache of any relation-gets for local unit
805+ flush(local_unit())
806+
807+
808+@cached
809+def relation_ids(reltype=None):
810+ """A list of relation_ids"""
811+ reltype = reltype or relation_type()
812+ relid_cmd_line = ['relation-ids', '--format=json']
813+ if reltype is not None:
814+ relid_cmd_line.append(reltype)
815+ return json.loads(subprocess.check_output(relid_cmd_line)) or []
816+ return []
817+
818+
819+@cached
820+def related_units(relid=None):
821+ """A list of related units"""
822+ relid = relid or relation_id()
823+ units_cmd_line = ['relation-list', '--format=json']
824+ if relid is not None:
825+ units_cmd_line.extend(('-r', relid))
826+ return json.loads(subprocess.check_output(units_cmd_line)) or []
827+
828+
829+@cached
830+def relation_for_unit(unit=None, rid=None):
831+ """Get the json represenation of a unit's relation"""
832+ unit = unit or remote_unit()
833+ relation = relation_get(unit=unit, rid=rid)
834+ for key in relation:
835+ if key.endswith('-list'):
836+ relation[key] = relation[key].split()
837+ relation['__unit__'] = unit
838+ return relation
839+
840+
841+@cached
842+def relations_for_id(relid=None):
843+ """Get relations of a specific relation ID"""
844+ relation_data = []
845+ relid = relid or relation_ids()
846+ for unit in related_units(relid):
847+ unit_data = relation_for_unit(unit, relid)
848+ unit_data['__relid__'] = relid
849+ relation_data.append(unit_data)
850+ return relation_data
851+
852+
853+@cached
854+def relations_of_type(reltype=None):
855+ """Get relations of a specific type"""
856+ relation_data = []
857+ reltype = reltype or relation_type()
858+ for relid in relation_ids(reltype):
859+ for relation in relations_for_id(relid):
860+ relation['__relid__'] = relid
861+ relation_data.append(relation)
862+ return relation_data
863+
864+
865+@cached
866+def relation_types():
867+ """Get a list of relation types supported by this charm"""
868+ charmdir = os.environ.get('CHARM_DIR', '')
869+ mdf = open(os.path.join(charmdir, 'metadata.yaml'))
870+ md = yaml.safe_load(mdf)
871+ rel_types = []
872+ for key in ('provides', 'requires', 'peers'):
873+ section = md.get(key)
874+ if section:
875+ rel_types.extend(section.keys())
876+ mdf.close()
877+ return rel_types
878+
879+
880+@cached
881+def relations():
882+ """Get a nested dictionary of relation data for all related units"""
883+ rels = {}
884+ for reltype in relation_types():
885+ relids = {}
886+ for relid in relation_ids(reltype):
887+ units = {local_unit(): relation_get(unit=local_unit(), rid=relid)}
888+ for unit in related_units(relid):
889+ reldata = relation_get(unit=unit, rid=relid)
890+ units[unit] = reldata
891+ relids[relid] = units
892+ rels[reltype] = relids
893+ return rels
894+
895+
896+@cached
897+def is_relation_made(relation, keys='private-address'):
898+ '''
899+ Determine whether a relation is established by checking for
900+ presence of key(s). If a list of keys is provided, they
901+ must all be present for the relation to be identified as made
902+ '''
903+ if isinstance(keys, str):
904+ keys = [keys]
905+ for r_id in relation_ids(relation):
906+ for unit in related_units(r_id):
907+ context = {}
908+ for k in keys:
909+ context[k] = relation_get(k, rid=r_id,
910+ unit=unit)
911+ if None not in context.values():
912+ return True
913+ return False
914+
915+
916+def open_port(port, protocol="TCP"):
917+ """Open a service network port"""
918+ _args = ['open-port']
919+ _args.append('{}/{}'.format(port, protocol))
920+ subprocess.check_call(_args)
921+
922+
923+def close_port(port, protocol="TCP"):
924+ """Close a service network port"""
925+ _args = ['close-port']
926+ _args.append('{}/{}'.format(port, protocol))
927+ subprocess.check_call(_args)
928+
929+
930+@cached
931+def unit_get(attribute):
932+ """Get the unit ID for the remote unit"""
933+ _args = ['unit-get', '--format=json', attribute]
934+ try:
935+ return json.loads(subprocess.check_output(_args))
936+ except ValueError:
937+ return None
938+
939+
940+def unit_private_ip():
941+ """Get this unit's private IP address"""
942+ return unit_get('private-address')
943+
944+
945+class UnregisteredHookError(Exception):
946+ """Raised when an undefined hook is called"""
947+ pass
948+
949+
950+class Hooks(object):
951+ """A convenient handler for hook functions.
952+
953+ Example:
954+ hooks = Hooks()
955+
956+ # register a hook, taking its name from the function name
957+ @hooks.hook()
958+ def install():
959+ ...
960+
961+ # register a hook, providing a custom hook name
962+ @hooks.hook("config-changed")
963+ def config_changed():
964+ ...
965+
966+ if __name__ == "__main__":
967+ # execute a hook based on the name the program is called by
968+ hooks.execute(sys.argv)
969+ """
970+
971+ def __init__(self):
972+ super(Hooks, self).__init__()
973+ self._hooks = {}
974+
975+ def register(self, name, function):
976+ """Register a hook"""
977+ self._hooks[name] = function
978+
979+ def execute(self, args):
980+ """Execute a registered hook based on args[0]"""
981+ hook_name = os.path.basename(args[0])
982+ if hook_name in self._hooks:
983+ self._hooks[hook_name]()
984+ else:
985+ raise UnregisteredHookError(hook_name)
986+
987+ def hook(self, *hook_names):
988+ """Decorator, registering them as hooks"""
989+ def wrapper(decorated):
990+ for hook_name in hook_names:
991+ self.register(hook_name, decorated)
992+ else:
993+ self.register(decorated.__name__, decorated)
994+ if '_' in decorated.__name__:
995+ self.register(
996+ decorated.__name__.replace('_', '-'), decorated)
997+ return decorated
998+ return wrapper
999+
1000+
1001+def charm_dir():
1002+ """Return the root directory of the current charm"""
1003+ return os.environ.get('CHARM_DIR')
1004
1005=== added file 'hooks/charmhelpers/core/host.py'
1006--- hooks/charmhelpers/core/host.py 1970-01-01 00:00:00 +0000
1007+++ hooks/charmhelpers/core/host.py 2014-07-21 22:01:08 +0000
1008@@ -0,0 +1,297 @@
1009+"""Tools for working with the host system"""
1010+# Copyright 2012 Canonical Ltd.
1011+#
1012+# Authors:
1013+# Nick Moffitt <nick.moffitt@canonical.com>
1014+# Matthew Wedgwood <matthew.wedgwood@canonical.com>
1015+
1016+import os
1017+import pwd
1018+import grp
1019+import random
1020+import string
1021+import subprocess
1022+import hashlib
1023+
1024+from collections import OrderedDict
1025+
1026+from hookenv import log
1027+
1028+
1029+def service_start(service_name):
1030+ """Start a system service"""
1031+ return service('start', service_name)
1032+
1033+
1034+def service_stop(service_name):
1035+ """Stop a system service"""
1036+ return service('stop', service_name)
1037+
1038+
1039+def service_restart(service_name):
1040+ """Restart a system service"""
1041+ return service('restart', service_name)
1042+
1043+
1044+def service_reload(service_name, restart_on_failure=False):
1045+ """Reload a system service, optionally falling back to restart if reload fails"""
1046+ service_result = service('reload', service_name)
1047+ if not service_result and restart_on_failure:
1048+ service_result = service('restart', service_name)
1049+ return service_result
1050+
1051+
1052+def service(action, service_name):
1053+ """Control a system service"""
1054+ cmd = ['service', service_name, action]
1055+ return subprocess.call(cmd) == 0
1056+
1057+
1058+def service_running(service):
1059+ """Determine whether a system service is running"""
1060+ try:
1061+ output = subprocess.check_output(['service', service, 'status'])
1062+ except subprocess.CalledProcessError:
1063+ return False
1064+ else:
1065+ if ("start/running" in output or "is running" in output):
1066+ return True
1067+ else:
1068+ return False
1069+
1070+
1071+def adduser(username, password=None, shell='/bin/bash', system_user=False):
1072+ """Add a user to the system"""
1073+ try:
1074+ user_info = pwd.getpwnam(username)
1075+ log('user {0} already exists!'.format(username))
1076+ except KeyError:
1077+ log('creating user {0}'.format(username))
1078+ cmd = ['useradd']
1079+ if system_user or password is None:
1080+ cmd.append('--system')
1081+ else:
1082+ cmd.extend([
1083+ '--create-home',
1084+ '--shell', shell,
1085+ '--password', password,
1086+ ])
1087+ cmd.append(username)
1088+ subprocess.check_call(cmd)
1089+ user_info = pwd.getpwnam(username)
1090+ return user_info
1091+
1092+
1093+def add_user_to_group(username, group):
1094+ """Add a user to a group"""
1095+ cmd = [
1096+ 'gpasswd', '-a',
1097+ username,
1098+ group
1099+ ]
1100+ log("Adding user {} to group {}".format(username, group))
1101+ subprocess.check_call(cmd)
1102+
1103+
1104+def rsync(from_path, to_path, flags='-r', options=None):
1105+ """Replicate the contents of a path"""
1106+ options = options or ['--delete', '--executability']
1107+ cmd = ['/usr/bin/rsync', flags]
1108+ cmd.extend(options)
1109+ cmd.append(from_path)
1110+ cmd.append(to_path)
1111+ log(" ".join(cmd))
1112+ return subprocess.check_output(cmd).strip()
1113+
1114+
1115+def symlink(source, destination):
1116+ """Create a symbolic link"""
1117+ log("Symlinking {} as {}".format(source, destination))
1118+ cmd = [
1119+ 'ln',
1120+ '-sf',
1121+ source,
1122+ destination,
1123+ ]
1124+ subprocess.check_call(cmd)
1125+
1126+
1127+def mkdir(path, owner='root', group='root', perms=0555, force=False):
1128+ """Create a directory"""
1129+ log("Making dir {} {}:{} {:o}".format(path, owner, group,
1130+ perms))
1131+ uid = pwd.getpwnam(owner).pw_uid
1132+ gid = grp.getgrnam(group).gr_gid
1133+ realpath = os.path.abspath(path)
1134+ if os.path.exists(realpath):
1135+ if force and not os.path.isdir(realpath):
1136+ log("Removing non-directory file {} prior to mkdir()".format(path))
1137+ os.unlink(realpath)
1138+ else:
1139+ os.makedirs(realpath, perms)
1140+ os.chown(realpath, uid, gid)
1141+
1142+
1143+def write_file(path, content, owner='root', group='root', perms=0444):
1144+ """Create or overwrite a file with the contents of a string"""
1145+ log("Writing file {} {}:{} {:o}".format(path, owner, group, perms))
1146+ uid = pwd.getpwnam(owner).pw_uid
1147+ gid = grp.getgrnam(group).gr_gid
1148+ with open(path, 'w') as target:
1149+ os.fchown(target.fileno(), uid, gid)
1150+ os.fchmod(target.fileno(), perms)
1151+ target.write(content)
1152+
1153+
1154+def mount(device, mountpoint, options=None, persist=False):
1155+ """Mount a filesystem at a particular mountpoint"""
1156+ cmd_args = ['mount']
1157+ if options is not None:
1158+ cmd_args.extend(['-o', options])
1159+ cmd_args.extend([device, mountpoint])
1160+ try:
1161+ subprocess.check_output(cmd_args)
1162+ except subprocess.CalledProcessError, e:
1163+ log('Error mounting {} at {}\n{}'.format(device, mountpoint, e.output))
1164+ return False
1165+ if persist:
1166+ # TODO: update fstab
1167+ pass
1168+ return True
1169+
1170+
1171+def umount(mountpoint, persist=False):
1172+ """Unmount a filesystem"""
1173+ cmd_args = ['umount', mountpoint]
1174+ try:
1175+ subprocess.check_output(cmd_args)
1176+ except subprocess.CalledProcessError, e:
1177+ log('Error unmounting {}\n{}'.format(mountpoint, e.output))
1178+ return False
1179+ if persist:
1180+ # TODO: update fstab
1181+ pass
1182+ return True
1183+
1184+
1185+def mounts():
1186+ """Get a list of all mounted volumes as [[mountpoint,device],[...]]"""
1187+ with open('/proc/mounts') as f:
1188+ # [['/mount/point','/dev/path'],[...]]
1189+ system_mounts = [m[1::-1] for m in [l.strip().split()
1190+ for l in f.readlines()]]
1191+ return system_mounts
1192+
1193+
1194+def file_hash(path):
1195+ """Generate a md5 hash of the contents of 'path' or None if not found """
1196+ if os.path.exists(path):
1197+ h = hashlib.md5()
1198+ with open(path, 'r') as source:
1199+ h.update(source.read()) # IGNORE:E1101 - it does have update
1200+ return h.hexdigest()
1201+ else:
1202+ return None
1203+
1204+
1205+def restart_on_change(restart_map, stopstart=False):
1206+ """Restart services based on configuration files changing
1207+
1208+ This function is used a decorator, for example
1209+
1210+ @restart_on_change({
1211+ '/etc/ceph/ceph.conf': [ 'cinder-api', 'cinder-volume' ]
1212+ })
1213+ def ceph_client_changed():
1214+ ...
1215+
1216+ In this example, the cinder-api and cinder-volume services
1217+ would be restarted if /etc/ceph/ceph.conf is changed by the
1218+ ceph_client_changed function.
1219+ """
1220+ def wrap(f):
1221+ def wrapped_f(*args):
1222+ checksums = {}
1223+ for path in restart_map:
1224+ checksums[path] = file_hash(path)
1225+ f(*args)
1226+ restarts = []
1227+ for path in restart_map:
1228+ if checksums[path] != file_hash(path):
1229+ restarts += restart_map[path]
1230+ services_list = list(OrderedDict.fromkeys(restarts))
1231+ if not stopstart:
1232+ for service_name in services_list:
1233+ service('restart', service_name)
1234+ else:
1235+ for action in ['stop', 'start']:
1236+ for service_name in services_list:
1237+ service(action, service_name)
1238+ return wrapped_f
1239+ return wrap
1240+
1241+
1242+def lsb_release():
1243+ """Return /etc/lsb-release in a dict"""
1244+ d = {}
1245+ with open('/etc/lsb-release', 'r') as lsb:
1246+ for l in lsb:
1247+ k, v = l.split('=')
1248+ d[k.strip()] = v.strip()
1249+ return d
1250+
1251+
1252+def pwgen(length=None):
1253+ """Generate a random pasword."""
1254+ if length is None:
1255+ length = random.choice(range(35, 45))
1256+ alphanumeric_chars = [
1257+ l for l in (string.letters + string.digits)
1258+ if l not in 'l0QD1vAEIOUaeiou']
1259+ random_chars = [
1260+ random.choice(alphanumeric_chars) for _ in range(length)]
1261+ return(''.join(random_chars))
1262+
1263+
1264+def list_nics(nic_type):
1265+ '''Return a list of nics of given type(s)'''
1266+ if isinstance(nic_type, basestring):
1267+ int_types = [nic_type]
1268+ else:
1269+ int_types = nic_type
1270+ interfaces = []
1271+ for int_type in int_types:
1272+ cmd = ['ip', 'addr', 'show', 'label', int_type + '*']
1273+ ip_output = subprocess.check_output(cmd).split('\n')
1274+ ip_output = (line for line in ip_output if line)
1275+ for line in ip_output:
1276+ if line.split()[1].startswith(int_type):
1277+ interfaces.append(line.split()[1].replace(":", ""))
1278+ return interfaces
1279+
1280+
1281+def set_nic_mtu(nic, mtu):
1282+ '''Set MTU on a network interface'''
1283+ cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
1284+ subprocess.check_call(cmd)
1285+
1286+
1287+def get_nic_mtu(nic):
1288+ cmd = ['ip', 'addr', 'show', nic]
1289+ ip_output = subprocess.check_output(cmd).split('\n')
1290+ mtu = ""
1291+ for line in ip_output:
1292+ words = line.split()
1293+ if 'mtu' in words:
1294+ mtu = words[words.index("mtu") + 1]
1295+ return mtu
1296+
1297+
1298+def get_nic_hwaddr(nic):
1299+ cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
1300+ ip_output = subprocess.check_output(cmd)
1301+ hwaddr = ""
1302+ words = ip_output.split()
1303+ if 'link/ether' in words:
1304+ hwaddr = words[words.index('link/ether') + 1]
1305+ return hwaddr
1306
1307=== modified file 'hooks/config-changed'
1308--- hooks/config-changed 2012-07-21 19:08:52 +0000
1309+++ hooks/config-changed 2014-07-21 22:01:08 +0000
1310@@ -1,25 +1,153 @@
1311 #!/bin/bash
1312
1313-set -e
1314+set -ex
1315+
1316+DOMAIN=`config-get domain`
1317+
1318+if [ -z "$DOMAIN" ]; then
1319+ DOMAIN=`unit-get public-address`
1320+fi
1321+
1322+if [ `cat .src` != `config-get src` ]; then
1323+ if [ `cat .src` == source ]; then
1324+ apt-get remove charm-helper-sh apache2 libapache2-mod-php5 php5-gd php5-mysql php5-sqlite curl libcurl3 php5-curl rpcbind nfs-common mysql-client -y
1325+ elif [ `cat .src` == repo ]; then
1326+ apt-get remove owncloud mysql-client -y
1327+ apt-get autoremove -y
1328+ fi
1329+ hooks/install
1330+fi
1331+
1332+if [ `config-get customssl` == True ] && [ -z `config-get sslkey` ]; then
1333+ juju-log "If you want to use a custom SSL cert, the key needs to be set. Exiting silently."
1334+ exit 0
1335+elif [ `config-get customssl` == True ] && [ -z `config-get sslcert` ]; then
1336+ juju-log "If you want to use a custom SSL cert, the cert needs to be set. Exiting silently."
1337+ exit 0
1338+fi
1339+
1340 # Write the apache config
1341 owncloud_vhost="/etc/apache2/sites-available/owncloud"
1342 port=`config-get port`
1343 juju-log "Writing apache config file: ${owncloud_vhost}"
1344 cat > $owncloud_vhost <<EOF
1345-<VirtualHost *:$port>
1346- DocumentRoot /var/www/owncloud
1347- Options All
1348- ErrorLog /var/log/apache2/owncloud-error.log
1349- TransferLog /var/log/apache2/owncloud-access.log
1350- RewriteEngine On
1351-</VirtualHost>
1352+<VirtualHost *:$port>
1353+RewriteEngine on
1354+ReWriteCond %{SERVER_PORT} !^443$
1355+RewriteRule ^/(.*) https://%{HTTP_HOST}/$1 [NC,R,L]
1356+DocumentRoot /var/www/owncloud/
1357+<Directory /var/www/owncloud>
1358+ Options Indexes FollowSymLinks MultiViews
1359+ AllowOverride All
1360+</Directory>
1361+</VirtualHost>
1362+
1363+<VirtualHost *:443>
1364+SSLEngine on
1365+SSLCertificateFile /home/ubuntu/$DOMAIN.cert
1366+SSLCertificateKeyFile /home/ubuntu/$DOMAIN.key
1367+DocumentRoot /var/www/owncloud/
1368+<Directory /var/www/owncloud>
1369+ Options Indexes FollowSymLinks MultiViews
1370+ AllowOverride All
1371+</Directory>
1372+RewriteEngine on
1373+</VirtualHost>
1374+
1375 EOF
1376
1377+if [ -f .customssl ] && [ `cat .customssl` != `config-get customssl` ]; then
1378+ rm /home/ubuntu/*.cert
1379+ rm /home/ubuntu/*.key
1380+ rm .customssl
1381+fi
1382+
1383+if [ -f .domain ] && [ `cat .domain` != "$DOMAIN" ]; then
1384+ rm /home/ubuntu/*.cert
1385+ rm /home/ubuntu/*.key
1386+ rm .customssl
1387+fi
1388+
1389+if [ ! -f .customssl ] && [ `config-get customssl` == "False" ]; then
1390+ hooks/ssl
1391+ echo "False" > .customssl
1392+elif [ `config-get customssl` == "True" ]; then
1393+ CERT=`config-get sslcert`
1394+ KEY=`config-get sslkey`
1395+ echo "$CERT" > /home/ubuntu/$DOMAIN.cert
1396+ echo "$KEY" > /home/ubuntu/$DOMAIN.key
1397+ echo "True" > .customssl
1398+fi
1399+
1400+echo "$DOMAIN" > .domain
1401+
1402+
1403 chmod 0644 $owncloud_vhost
1404
1405+if [ "$port" != "80" ] && [ ! -f .not80 ]; then
1406+ sed -i "s/NameVirtualHost\ \*\:80/NameVirtualHost\ \*\:$port/" /etc/apache2/ports.conf
1407+ sed -i "s/Listen\ 80/Listen\ $port/" /etc/apache2/ports.conf
1408+ echo "$port" > .not80
1409+elif [ "$port" != "80" ] && [ -f .not80 ]; then
1410+ not=`cat .not80`
1411+ sed -i "s/NameVirtualHost\ \*\:$not/NameVirtualHost\ \*\:$port/" /etc/apache2/ports.conf
1412+ sed -i "s/Listen\ $not/Listen\ $port/" /etc/apache2/ports.conf
1413+ echo "$port" > .not80
1414+elif [ "$port" == 80 ] && [ -f .not80 ]; then
1415+ not=`cat .not80`
1416+ sed -i "s/NameVirtualHost\ \*\:$not/NameVirtualHost\ \*\:80/" /etc/apache2/ports.conf
1417+ sed -i "s/Listen\ $not/Listen\ 80/" /etc/apache2/ports.conf
1418+ rm .not80
1419+fi
1420+
1421+if [ -f .port ] && [ `cat .port` != "$port" ]; then
1422+ currentport=`cat .port`
1423+ juju-log "Closing old port, opening new port"
1424+ close-port "$currentport"/tcp
1425+ open-port "$port"/tcp
1426+ echo "$port" > .port
1427+elif [ ! -f .port ]; then
1428+ open-port $port/tcp
1429+ echo "$port" > .port
1430+fi
1431+
1432+if [ -f .dbset ]; then
1433+ USER=`cat .user`
1434+ PASSWORD=`cat .password`
1435+ HOST=`cat .host`
1436+ DB=`cat .database`
1437+ if [ `cat .admin` != `config-get user` ]; then
1438+ OLDADMIN=`cat .admin`
1439+ ADMIN=`config-get user`
1440+ mysql -u $USER -p$PASSWORD -h $HOST $DB -e "UPDATE oc_users SET uid = '$ADMIN' WHERE uid = '$OLDADMIN';"
1441+ mv /var/www/owncloud/data/$OLDADMIN /var/www/owncloud/data/$ADMIN
1442+ echo "$ADMIN" > .admin
1443+ fi
1444+ if [ `cat .adm_pass` != `config-get password` ]; then
1445+ NEWPASS=`config-get password`
1446+ OCUSER=`config-get user`
1447+ if [ ! -f .passwordchanged ]; then
1448+ SALT=`cat /var/www/owncloud/config/config.php | grep passwordsalt | awk '{print $3}'`
1449+ sed -i "s/$t_hasher = new PasswordHash(8, FALSE)/$t_hasher = new PasswordHash(8, CRYPT_BLOWFISH!=1)/" /var/www/owncloud/3rdparty/phpass/test.php
1450+ sed -i "s/'test12345';/'$NEWPASS'.$SALT/" /var/www/owncloud/3rdparty/phpass/test.php
1451+ sed -i "s/',/';/" /var/www/owncloud/3rdparty/phpass/test.php
1452+ HASH=`php /var/www/owncloud/3rdparty/phpass/test.php | awk 'NR<2{ print $2 }'`
1453+ mysql -u $USER -p$PASSWORD -h $HOST $DB -e "UPDATE oc_users SET password = '$HASH' WHERE uid = '$OCUSER';"
1454+ echo "$NEWPASS" > .adm_pass
1455+ touch .passwordchanged
1456+ else
1457+ OLDPASS=`cat .adm_pass`
1458+ sed -i "s/'$OLDPASS'./'$NEWPASS'./" /var/www/owncloud/3rdparty/phpass/test.php
1459+ HASH=`php /var/www/owncloud/3rdparty/phpass/test.php | awk 'NR<2{ print $2 }'`
1460+ mysql -u $USER -p$PASSWORD -h $HOST $DB -e "UPDATE oc_users SET password = '$HASH' WHERE uid = '$OCUSER';"
1461+ echo "$NEWPASS" > .adm_pass
1462+ fi
1463+ fi
1464+fi
1465+
1466+
1467 # Optimize PHP for large files
1468 juju-log "Increasing upload limit and maximum POST size"
1469-sed -i 's/;upload_max_filesize/;upload_max_filesize/g' /etc/php5/apache2/php.ini
1470 sed -i 's/^post_max_size/;post_max_size/g' /etc/php5/apache2/php.ini
1471
1472 cat > /etc/php5/conf.d/owncloud.ini <<EOF
1473@@ -31,6 +159,7 @@
1474 juju-log "Enabling Apache modules: rewrite, headers, vhost_alias"
1475 a2enmod rewrite
1476 a2enmod headers
1477+a2enmod ssl
1478
1479 # Enable Apache vhost
1480 juju-log "Enabling ownCloud Apache vhost"
1481@@ -40,3 +169,18 @@
1482 juju-log "Reloading Apache configuration"
1483 service apache2 start || :
1484 service apache2 reload
1485+
1486+
1487+
1488+if [ ! -f .443 ]; then
1489+ open-port 443
1490+ touch .443
1491+fi
1492+
1493+if [ ! -f .configured ]; then
1494+ touch .configured
1495+fi
1496+
1497+if [ ! -f .started ]; then
1498+ hooks/start
1499+fi
1500
1501=== modified file 'hooks/db-relation-changed'
1502--- hooks/db-relation-changed 2012-07-21 16:20:15 +0000
1503+++ hooks/db-relation-changed 2014-07-21 22:01:08 +0000
1504@@ -12,6 +12,11 @@
1505 password=`relation-get password`
1506 host=`relation-get private-address`
1507
1508+echo "$database" > .database
1509+echo "$password" > .password
1510+echo "$user" > .user
1511+echo "$host" > .host
1512+
1513 # Prepare for sqlite->mysql migration if configured
1514 if [ -f "/var/www/owncloud/config/config.php" ]; then
1515 if grep -qs "sqlite" /var/www/owncloud/config/config.php; then
1516@@ -35,6 +40,9 @@
1517 adm_pass=`cat /dev/urandom | tr -dc [:alnum:] | head -c 10`
1518 fi
1519
1520+echo "$admin" > .admin
1521+echo "$adm_pass" > .adm_pass
1522+
1523 if [ -f "/var/www/owncloud/config/config.php" ]; then
1524 # Update config settings
1525 cat > /var/www/owncloud/config/config.php <<EOF
1526@@ -80,3 +88,4 @@
1527 fi
1528
1529 chmod 0644 /var/www/owncloud/config/*
1530+touch .dbset
1531
1532=== added file 'hooks/db-relation-departed'
1533--- hooks/db-relation-departed 1970-01-01 00:00:00 +0000
1534+++ hooks/db-relation-departed 2014-07-21 22:01:08 +0000
1535@@ -0,0 +1,13 @@
1536+#!/bin/bash
1537+
1538+set -eux
1539+
1540+mv /var/www/owncloud/config/config.php{,.bakmysql}
1541+
1542+if [ -f "/var/www/owncloud/config/config.bak" ]; then
1543+ if grep -qs "mysql" /var/www/owncloud/config/config.bak; then
1544+ mv /var/www/owncloud/config/config.bak{,.php}
1545+ fi
1546+fi
1547+
1548+chmod 0644 /var/www/owncloud/config/*
1549
1550=== modified file 'hooks/install'
1551--- hooks/install 2014-04-11 03:48:08 +0000
1552+++ hooks/install 2014-07-21 22:01:08 +0000
1553@@ -1,30 +1,52 @@
1554 #!/bin/bash
1555
1556-set -eu # -x for verbose logging to ensemble debug-log
1557+set -eu
1558
1559 juju-log "Installing all components"
1560
1561-add-apt-repository -y ppa:charmers/charm-helpers
1562-apt-get update
1563-apt-get install -qqy charm-helper-sh apache2 libapache2-mod-php5 \
1564- php5-gd php5-mysql php5-sqlite curl libcurl3 \
1565- php5-curl rpcbind nfs-common
1566-
1567-juju-log "Downloading and validating ownCloud"
1568-# Load charm helper and download owncloud
1569-. /usr/share/charm-helper/sh/net.sh
1570-LURL="http://download.owncloud.org/community/owncloud-6.0.2.tar.bz2"
1571-
1572-LHASH="a5a194ad07fca7cbf158b660cc098c6364590bdd15d086069221faf4386b713f"
1573-source_file=`ch_get_file $LURL $LHASH`
1574-
1575-juju-log "Downloaded"
1576-
1577-# Prepare files
1578-juju-log "Creating ownCloud DocRoot"
1579-tar -xjf $source_file --directory /var/www
1580+if [ `config-get src` == "source" ]; then
1581+
1582+ add-apt-repository -y ppa:charmers/charm-helpers
1583+ apt-get update
1584+ apt-get install -qqy charm-helper-sh apache2 libapache2-mod-php5 php5-gd php5-mysql php5-sqlite curl libcurl3 php5-curl rpcbind nfs-common mysql-client
1585+
1586+ juju-log "Downloading and validating ownCloud"
1587+ # Load charm helper and download owncloud
1588+ . /usr/share/charm-helper/sh/net.sh
1589+ LURL=`config-get downloadurl`
1590+
1591+ LHASH=`config-get sha1sum`
1592+ source_file=`ch_get_file $LURL $LHASH`
1593+
1594+ juju-log "Downloaded"
1595+
1596+ # Prepare files
1597+ juju-log "Creating ownCloud DocRoot"
1598+ tar -xjf $source_file --directory /var/www
1599+
1600+elif [ `config-get src` == "repo" ]; then
1601+
1602+ apt-get install -y mysql-client
1603+
1604+ sh -c "echo 'deb http://download.opensuse.org/repositories/isv:/ownCloud:/community/xUbuntu_12.04/ /' >> /etc/apt/sources.list.d/owncloud.list"
1605+ wget http://download.opensuse.org/repositories/isv:/ownCloud:/community/xUbuntu_12.04/Release.key
1606+
1607+ if [ `sha1sum Release.key | awk '{print $1}'` != "c76c49ca044d9547648f1d8313777ed7d55df6b2" ]; then
1608+ juju-log "Apt key verification failed, failing hook."
1609+ exit 1
1610+ fi
1611+
1612+ apt-key add - < Release.key
1613+ apt-get update
1614+ apt-get install -y owncloud
1615+
1616+fi
1617+
1618 chown www-data:www-data -R /var/www/owncloud
1619
1620-#Make ownCloud publicly visible
1621-port=`config-get port`
1622-open-port $port/tcp
1623+sed -i "s/ \$curlSettings = array(/ \$curlSettings = array(\n CURLOPT_SSL_VERIFYPEER => 0,\n CURLOPT_SSL_VERIFYHOST => 0,/" /var/www/owncloud/3rdparty/Sabre/DAV/Client.php
1624+
1625+SRC=`config-get src`
1626+echo $SRC > .src
1627+
1628+service apache2 restart
1629
1630=== added file 'hooks/ssl'
1631--- hooks/ssl 1970-01-01 00:00:00 +0000
1632+++ hooks/ssl 2014-07-21 22:01:08 +0000
1633@@ -0,0 +1,19 @@
1634+#!/usr/bin/python
1635+
1636+import os
1637+import sys
1638+
1639+sys.path.insert(0, os.environ['CHARM_DIR'])
1640+
1641+from charmhelpers.contrib import ssl
1642+from charmhelpers.core import hookenv
1643+from charmhelpers.core.hookenv import unit_get
1644+
1645+DOMAIN = hookenv.config('domain')
1646+
1647+if DOMAIN == "":
1648+ DOMAIN = unit_get('public-address')
1649+
1650+key_file = '/home/ubuntu/' + DOMAIN + '.key'
1651+cert_file = '/home/ubuntu/' + DOMAIN + '.cert'
1652+ssl.generate_selfsigned(key_file, cert_file, cn=DOMAIN)
1653
1654=== modified file 'hooks/start'
1655--- hooks/start 2012-07-21 00:54:38 +0000
1656+++ hooks/start 2014-07-21 22:01:08 +0000
1657@@ -1,4 +1,8 @@
1658 #!/bin/bash
1659 set -eu
1660-juju-log "Starting apache for owncloud"
1661-service apache2 start || service apache2 restart
1662+
1663+if [ -f .configured ]; then
1664+ juju-log "Starting apache for owncloud"
1665+ service apache2 start || service apache2 restart
1666+ touch .started
1667+fi
1668
1669=== modified file 'hooks/upgrade-charm'
1670--- hooks/upgrade-charm 2014-04-11 03:47:03 +0000
1671+++ hooks/upgrade-charm 2014-07-21 22:01:08 +0000
1672@@ -5,43 +5,47 @@
1673 # Pause during upgrade
1674 hooks/stop
1675
1676-#we are under active migration, dont do anything more than once
1677-if [ ! -f $CHARM_DIR/.migrating ]; then
1678-
1679- # Excise old installation data
1680- find /var/www -mindepth 2 -maxdepth 2 -type d -name 'data'\
1681- -exec mv {} /var/tmp/ \;
1682- find /var/www -mindepth 2 -maxdepth 2 -type d -name 'config'\
1683- -exec mv {} /var/tmp/ \;
1684- touch $CHARM_DIR/.migrating
1685-
1686+if [ -f /usr/share/charm-helper/sh/net.sh ]; then
1687+ #we are under active migration, dont do anything more than once
1688+ if [ ! -f $CHARM_DIR/.migrating ]; then
1689+
1690+ # Excise old installation data
1691+ find /var/www -mindepth 2 -maxdepth 2 -type d -name 'data'\
1692+ -exec mv {} /var/tmp/ \;
1693+ find /var/www -mindepth 2 -maxdepth 2 -type d -name 'config'\
1694+ -exec mv {} /var/tmp/ \;
1695+ touch $CHARM_DIR/.migrating
1696+
1697+ fi
1698+
1699+ # Deleting all traces of the old ownCloud install
1700+ rm -rf /var/www/owncloud/
1701+
1702+ cd /var/www/
1703+
1704+ # Getting new ownCloud version and MD5 checking it
1705+ wget download.owncloud.org/community/owncloud-6.0.3.tar.bz2
1706+
1707+ if [ `sha1sum owncloud-6.0.3.tar.bz2 | awk '{print $1}'` != "df1e272c208376117a8c00619079cee25a22f784" ]; then
1708+ juju-log "Download verification failed. Exiting."
1709+ exit 1
1710+ fi
1711+
1712+ tar xfj owncloud-6.0.3.tar.bz2
1713+ cd $CHARM_DIR
1714+
1715+ # Migrating data
1716+ mkdir /var/www/owncloud/data
1717+ rm -rf /var/www/owncloud/config
1718+ mkdir /var/www/owncloud/config
1719+ cp -pr /var/tmp/data/* /var/www/owncloud/data/
1720+ cp -pr /var/tmp/config/* /var/www/owncloud/config/
1721+ sudo chown -R www-data:www-data /var/www/owncloud
1722+ rm $CHARM_DIR/.migrating
1723+
1724+ # Finish upgrade
1725+ hooks/config-changed
1726+else
1727+ apt-get update
1728+ apt-get upgrade -y
1729 fi
1730-
1731-# Deleting all traces of the old ownCloud install
1732-rm -rf /var/www/owncloud/
1733-cd /var/www/
1734-
1735-# Getting new ownCloud version and MD5 checking it
1736-wget download.owncloud.org/community/owncloud-6.0.2.tar.bz2
1737-wget download.owncloud.org/community/owncloud-6.0.2.tar.bz2.md5
1738-
1739-while [ `md5sum owncloud-6.0.2.tar.bz2 | awk '{print $1}'` != `cat owncloud-6.0.2.tar.bz2.md5 | awk '{print $1}'` ]; do
1740- rm owncloud-6.0.2.tar.bz2
1741- wget download.owncloud.org/community/owncloud-6.0.2.tar.bz2
1742-done
1743-
1744-tar xfj owncloud-6.0.2.tar.bz2
1745-cd $CHARM_DIR
1746-
1747-# Migrating data
1748-mkdir /var/www/owncloud/data
1749-rm -rf /var/www/owncloud/config
1750-mkdir /var/www/owncloud/config
1751-cp -pr /var/tmp/data/* /var/www/owncloud/data/
1752-cp -pr /var/tmp/config/* /var/www/owncloud/config/
1753-sudo chown -R www-data:www-data /var/www/owncloud
1754-
1755-# Finish upgrade
1756-hooks/config-changed
1757-
1758-rm $CHARM_DIR/.migrating
1759
1760=== modified file 'hooks/website-relation-joined'
1761--- hooks/website-relation-joined 2012-07-21 16:20:15 +0000
1762+++ hooks/website-relation-joined 2014-07-21 22:01:08 +0000
1763@@ -1,4 +1,3 @@
1764 #!/bin/sh
1765-port=`config-get port`
1766 pub_addr=`unit-get private-address`
1767-relation-set port=$port hostname=$pub_addr
1768+relation-set port=443 hostname=$pub_addr
1769
1770=== modified file 'metadata.yaml'
1771--- metadata.yaml 2014-01-24 16:01:17 +0000
1772+++ metadata.yaml 2014-07-21 22:01:08 +0000
1773@@ -6,7 +6,7 @@
1774 storage, MySQL Databases, and HAProxy reverse proxy charms.
1775 categories:
1776 - file-servers
1777-maintainer: "Nathan Williams <nathan@nathanewilliams.com>"
1778+maintainer: "Jose Antonio Rey <jose@ubuntu.com>"
1779 requires:
1780 db:
1781 interface: mysql
1782
1783=== modified file 'tests/00_setup.sh' (properties changed: -x to +x)
1784=== added file 'tests/100-deploy.py'
1785--- tests/100-deploy.py 1970-01-01 00:00:00 +0000
1786+++ tests/100-deploy.py 2014-07-21 22:01:08 +0000
1787@@ -0,0 +1,116 @@
1788+#!/usr/bin/env python3
1789+# The format of this test was shamelessly ripped from Cory Johns awesome
1790+# tests that resemble nosetests in the Apache Allura charm. <3
1791+# http://bazaar.launchpad.net/~johnsca/charms/precise/apache-allura/
1792+
1793+import amulet
1794+import requests
1795+
1796+
1797+class TestDeploy(object):
1798+
1799+ def __init__(self, time=2500):
1800+ # Attempt to load the deployment topology from a bundle.
1801+ self.deploy = amulet.Deployment()
1802+
1803+ # If something errored out, attempt to continue by
1804+ # manually specifying a standalone deployment
1805+ self.deploy.add('owncloud')
1806+ self.deploy.add('mysql')
1807+ self.deploy.add('nfs')
1808+ self.deploy.configure('owncloud', {'user': 'tom',
1809+ 'password': 'swordfish'})
1810+ self.deploy.relate('owncloud:db', 'mysql:db')
1811+ self.deploy.relate('owncloud:shared-fs', 'nfs:nfs')
1812+ self.deploy.expose('owncloud')
1813+
1814+ try:
1815+ self.deploy.setup(time)
1816+ self.deploy.sentry.wait(time)
1817+ except:
1818+ amulet.raise_status(amulet.FAIL, msg="Environment standup timeout")
1819+ sentry = self.deploy.sentry
1820+
1821+ self.domain = sentry.unit['owncloud/0'].info['public-address']
1822+ self.mysql_unit = self.deploy.sentry.unit['mysql/0']
1823+ self.nfs_unit = self.deploy.sentry.unit['nfs/0']
1824+ self.app_unit = self.deploy.sentry.unit['owncloud/0']
1825+
1826+ def run(self):
1827+ for test in dir(self):
1828+ if test.startswith('test_'):
1829+ getattr(self, test)()
1830+
1831+ def test_standalone_http(self):
1832+
1833+ # r = requests.get("http://%s/" % domain, data, headers=h)
1834+ r = requests.get("https://%s/index.php" % self.domain, verify=False)
1835+ r.raise_for_status()
1836+
1837+ search_string = 'web services under your control'
1838+ if r.text.find(search_string) is -1:
1839+ amulet.raise_status(amulet.FAIL, msg="Unable to verify login page")
1840+
1841+ def test_database_relationship(self):
1842+
1843+ sent_config = self.mysql_unit.relation('db', 'owncloud:db')
1844+ # I'm sorry this is gross, and repeats itself. This is a dirty hack
1845+ # around owncloud not normalizing quotations, and field names.
1846+ try:
1847+ filepath = "/var/www/owncloud/config/autoconfig.php"
1848+ cfg = self.deploy.sentry.unit['owncloud/0'].file_contents(filepath)
1849+ cfg_to_check = {'dbuser': 'user', 'dbpass': 'password',
1850+ 'dbhost': 'host'}
1851+ for c_key, j_key in cfg_to_check.items():
1852+ if cfg.find('"%s" => "%s"' % (c_key, sent_config[j_key])) == -1:
1853+ amulet.raise_status(amulet.FAIL,
1854+ msg="Unable to validate db cfg %s" % j_key)
1855+ except:
1856+ filepath = "/var/www/owncloud/config/config.php"
1857+ cfg = self.deploy.sentry.unit['owncloud/0'].file_contents(filepath)
1858+ cfg_to_check = {'dbuser': 'user', 'dbpassword': 'password',
1859+ 'dbhost': 'host'}
1860+ for c_key, j_key in cfg_to_check.items():
1861+ if cfg.find("'%s' => '%s'" % (c_key, sent_config[j_key])) == -1:
1862+ amulet.raise_status(amulet.FAIL,
1863+ msg="Unable to validate db cfg %s" % j_key)
1864+
1865+ def term_search(self, output, term):
1866+ if output.find(term) == -1:
1867+ amulet.raise_status(amulet.FAIL,
1868+ msg="Unable to validate NFS config %s" % term)
1869+
1870+ def test_nfs_relationship(self):
1871+ # Cache Relationship details
1872+ nfs_relation = self.nfs_unit.relation('nfs', 'owncloud:shared-fs')
1873+
1874+ # Leverage Amulet's exception handling if directory doesnt exist
1875+ # to check for the directory, as a cheap quick fail test.
1876+ try:
1877+ self.app_unit.directory('/var/lib/owncloud')
1878+ except:
1879+ amulet.raise_status(amulet.FAIL, msg="NFS Directory not found")
1880+
1881+ #Fetch the contents of mtab for data validation
1882+ mtab_contents = self.app_unit.file_contents('/etc/mtab')
1883+
1884+ self.term_search(mtab_contents, nfs_relation['private-address'])
1885+ self.term_search(mtab_contents, nfs_relation['fstype'])
1886+ self.term_search(mtab_contents, nfs_relation['mountpoint'])
1887+ self.term_search(mtab_contents, nfs_relation['options'])
1888+
1889+ def test_nfs_write_pipeline(self):
1890+ # Validate file write pipeline
1891+ #Build a $block_size file, and ship it via NFS
1892+ cmd_builder = "dd if=/dev/zero of=/var/lib/owncloud/amulet-file-test bs=8M count=1"
1893+ self.app_unit.run(cmd_builder)
1894+
1895+ filepath = '/srv/data/relation-sentry/amulet-file-test'
1896+ file_test = self.nfs_unit.file(filepath)
1897+ if file_test['size'] < 8000000:
1898+ amulet.raise_status(amulet.FAIL, 'File size constraint not met')
1899+
1900+
1901+if __name__ == '__main__':
1902+ runner = TestDeploy()
1903+ runner.run()
1904
1905=== removed file 'tests/100_deploy.test'
1906--- tests/100_deploy.test 2014-02-10 21:49:21 +0000
1907+++ tests/100_deploy.test 1970-01-01 00:00:00 +0000
1908@@ -1,142 +0,0 @@
1909-#!/usr/bin/env python3
1910-
1911-import amulet
1912-import requests
1913-
1914-##########################################
1915-# Config Options
1916-##########################################
1917-scale = 2
1918-seconds = 1200
1919-user = 'tom'
1920-password = 'swordfish'
1921-block_size = "8M"
1922-verify_size = 8000000
1923-
1924-###########################################################
1925-#Deployment Setup
1926-############################################################
1927-d = amulet.Deployment()
1928-
1929-d.add('owncloud', units=scale)
1930-d.add('mysql')
1931-d.add('haproxy')
1932-d.add('nfs')
1933-d.configure('owncloud', {'user': user, 'password': password})
1934-d.relate('owncloud:db', 'mysql:db')
1935-d.relate('owncloud:website', 'haproxy:reverseproxy')
1936-d.relate('owncloud:shared-fs', 'nfs:nfs')
1937-d.expose('owncloud')
1938-d.expose('haproxy')
1939-
1940-
1941-#perform deployment
1942-try:
1943- d.setup(timeout=seconds)
1944-except amulet.helpers.TimeoutError:
1945- message = 'The environment did not setup in %d seconds.', seconds
1946- amulet.raise_status(amulet.SKIP, msg=message)
1947-except:
1948- raise
1949-
1950-
1951-#############################################################
1952-# Check presence of HTTP services
1953-#############################################################
1954-def validate_status_interface():
1955- h = {'User-Agent': 'Mozilla/5.0 Gecko/20100101 Firefox/12.0',
1956- 'Content-Type': 'application/x-www-form-urlencoded',
1957- 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*',
1958- 'Accept-Encoding': 'gzip, deflate'}
1959-
1960- data = {'user': 'tom', 'password': 'swordfish'}
1961-
1962- #validate the Owncloud login screen is present
1963- r = requests.post("http://{}".format(
1964- d.sentry.unit['owncloud/0'].info['public-address']),
1965- data, headers=h)
1966- r.raise_for_status()
1967-
1968- #validate the HAProxy daemon is forwarding as expected
1969- r = requests.get("http://{}".format(
1970- d.sentry.unit['haproxy/0'].info['public-address']))
1971- r.raise_for_status
1972-
1973-
1974-#############################################################
1975-# Validate that each service is running
1976-#############################################################
1977-def validate_running_services():
1978- output, code = d.sentry.unit['owncloud/0'].run('service apache2 status')
1979- if code != 0:
1980- message = "Failed to find running Apache instance on host owncloud/0"
1981- amulet.raise_status(amulet.SKIP, msg=message)
1982- output, code = d.sentry.unit['mysql/0'].run('service mysql status')
1983- if code != 0:
1984- message = "Failed to find running MYSQL instance on host mysql/0"
1985- amulet.raise_status(amulet.SKIP, msg=message)
1986-
1987-
1988-#############################################################
1989-# Validate that each service is running
1990-#############################################################
1991-def validate_owncloud_files():
1992- index_status = d.sentry.unit['owncloud/0'].file_stat('/var/www/owncloud/index.php')
1993- if index_status['size'] <= 0:
1994- message = "Failed to find owncloud index.php"
1995- amulet.raise_status(amulet.SKIP, msg=message)
1996-
1997-
1998-#############################################################
1999-# Validate database relationship
2000-#############################################################
2001-def validate_database_relationship():
2002- #Connect to the sentrys and fetch the transmitted details
2003- sent_config = d.sentry.unit['mysql/0'].relation('db', 'owncloud:db')
2004- #Connect to owncloud's sentry and read the configuration PHP file
2005- prod_config = d.sentry.unit['owncloud/0'].file_contents('/var/www/owncloud/config/config.php')
2006- cfg_to_check = {'dbuser': 'user', 'dbpassword': 'password', 'dbhost': 'host'}
2007-
2008- #Search the return string of the config for the transmit values
2009- for cfg_file_key, juju_cfg_key in cfg_to_check.items():
2010- if prod_config.find("'%s' => '%s'" % (cfg_file_key, sent_config[juju_cfg_key])) == -1:
2011- amulet.raise_status(amulet.SKIP, msg="Unable to validate db sent %s" % juju_cfg_key)
2012-
2013-
2014-# Utility Method for searching output
2015-def nfs_term_search(output, term):
2016- if output.find(term) == -1:
2017- amulet.raise_status(amulet.FAIL, msg="Unable to validate NFS export mounted with %s" % term)
2018-
2019-
2020-###########################################################
2021-# Validate NFS FileSystem Existence
2022-###########################################################
2023-def validate_nfs_relationship():
2024- # Cache Relationship details
2025- nfs_relation = d.sentry.unit['nfs/0'].relation('nfs', 'owncloud:shared-fs')
2026- # Raises an error if the directory does not exist
2027- d.sentry.unit['owncloud/0'].directory('/var/lib/owncloud')
2028- #Fetch the contents of mtab for data validation
2029- mtab_contents = d.sentry.unit['owncloud/0'].file_contents('/etc/mtab')
2030-
2031- nfs_term_search(mtab_contents, nfs_relation['private-address'])
2032- nfs_term_search(mtab_contents, nfs_relation['fstype'])
2033- nfs_term_search(mtab_contents, nfs_relation['mountpoint'])
2034- nfs_term_search(mtab_contents, nfs_relation['options'])
2035-
2036- # Validate file write pipeline
2037- #Build a $block_size file, and ship it via NFS
2038- cmd_builder = "dd if=/dev/zero of=/var/lib/owncloud/amulet-file-test bs=%s count=1" % block_size
2039- d.sentry.unit['owncloud/0'].run(cmd_builder)
2040-
2041- file_test = d.sentry.unit['nfs/0'].file('/srv/data/relation-sentry/amulet-file-test')
2042- if file_test['size'] < verify_size:
2043- amulet.raise_status(amulet.FAIL, 'File size constraint not met')
2044-
2045-
2046-validate_status_interface()
2047-validate_running_services()
2048-validate_owncloud_files()
2049-validate_database_relationship()
2050-validate_nfs_relationship()

Subscribers

People subscribed via source and target branches

to all changes:
to status/vote changes: