Merge lp:~tasdomas/juju-core/charm-store-auth into lp:~go-bot/juju-core/trunk
- charm-store-auth
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Roger Peppe |
Approved revision: | no longer in the source branch. |
Merged at revision: | 2114 |
Proposed branch: | lp:~tasdomas/juju-core/charm-store-auth |
Merge into: | lp:~go-bot/juju-core/trunk |
Diff against target: |
283 lines (+114/-11) 6 files modified
charm/export_test.go (+1/-1) charm/repo.go (+31/-7) cmd/juju/deploy.go (+16/-1) cmd/juju/upgradecharm.go (+9/-0) environs/config/config.go (+19/-2) environs/config/config_test.go (+38/-0) |
To merge this branch: | bzr merge lp:~tasdomas/juju-core/charm-store-auth |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Juju Engineering | Pending | ||
Review via email: mp+194130@code.launchpad.net |
Commit message
Charm store auth token support.
Juju client (deploy and upgrade subcommands) should send an authentication token
to the charm store with all requests, if the token was specified for the environment
under the charm-store-auth key.
Description of the change
Charm store auth token support.
Juju client (deploy and upgrade subcommands) should send an authentication token
to the charm store with all requests, if the token was specified for the environment
under the charm-store-auth key.
William Reade (fwereade) wrote : | # |
Roger Peppe (rogpeppe) wrote : | # |
Thanks for this. A few thoughts below.
https:/
File charm/repo.go (right):
https:/
charm/repo.go:69: func (s *CharmStore) SetAuthToken(token string) {
This isn't quite right.
Store is a global variable, so this method can alter global state
without any mutual exclusion.
Better might be just to export the authToken field.
Then it's clear to callers that they need to copy
Store (or create a new one) before adding an auth token.
Alternatively we could add a method to CharmStore, say:
func (cs *CharmStore) WithAuth(authToken string) Repository
which would return a new repository value with authentication
attached.
https:/
charm/repo.go:80: req.Header.
Is it ok to be adding our own http headers like this?
Might it not be better to include the auth token in the URL?
https:/
File charm/repo.go (right):
https:/
charm/repo.go:69: func (s *CharmStore) setAuthToken(token string) {
I'm not sure about this either - this means that the only way of getting
an authenticated connection is by using InferRepository, but we might
not want to do that.
And in fact, it's potentially a little dangerous, as InferRepository
might potentially change in the future so it can infer more than one
charm site, and then we run the danger of giving our secret credentials
to that other site.
I'd be tempted to keep the auth credentials away from InferRepository
and let the client set them as appropriate.
Domas Monkus (tasdomas) wrote : | # |
Reviewers: mp+194130_
Message:
Please take a look.
Description:
Charm store auth token support.
Juju client (deploy and upgrade subcommands) should send an
authentication token
to the charm store with all requests, if the token was specified for the
environment
under the charm-store-auth key.
https:/
(do not edit description out of merge proposal)
Please review this at https:/
Affected files (+107, -8 lines):
A [revision details]
M charm/export_
M charm/repo.go
M cmd/juju/deploy.go
M cmd/juju/
M environs/
M environs/
M store/server.go
Roger Peppe (rogpeppe) wrote : | # |
Good direction, thanks. Time for another round :-)
https:/
File charm/repo.go (right):
https:/
charm/repo.go:70: func (s CharmStore) WithAuthToken(token string)
Repository {
This should be a method on *CharmStore like the other methods.
https:/
charm/repo.go:71: authCS := *Store
s/Store/*s/
https:/
charm/repo.go:83: /* To comply with RFC 2617, we send the auth token in
Please use the conventional // comments.
https:/
charm/repo.go:87:
base64.
A few things here:
"CSAuth" doesn't seem like a great name ("CS" can stand for many
things). How about just "charmstore" (this is always used in an
Authorization header context, so the "Auth" seems redundant)?
Also, can't we define the authorization token to be hex (for example)
thus avoiding the need to base64 encode it? In actual fact standard
base64 really isn't good here, as / is a separator character which is
forbidden in a token.
Also I don't think this actually conforms to RFC 2617 (AFAICS auth-param
is of the form [token "=" token] and the auth-scheme shouldn't be a
quoted string)
To summarise, how about this instead?
req.Header.
?
https:/
File cmd/juju/
https:/
cmd/juju/
pass it on to the charm store
Perhaps factor out this piece of code into a separate function
to be called from anywhere necessary.
e.g.
// authorizeCharmRepo returns a version of the given
// charm repository with authorization information
// added from the given environment as appropriate.
func authorizeCharmR
charm.Repository
https:/
File store/server.go (right):
https:/
store/server.go:86: tokenParts := strings.
This should check that the authorization scheme is as expected.
Also, I'm not sure if this is quite in the right place. I'd be more
inclined to add an authorization checker in NewServer.
There's an idiom that comes in useful here.
Something like this, perhaps:
// server is the internal version of Server, which
// is created when serving a request for a particular
// user.
type server struct {
*Server
authorized bool
}
func (s *Server) handle(path string, serverHandler func(server,
http.ResponseWr
handler := func(w http.ResponseWr
s := server{Server: s}
if token := r.Header.
if authorization matches ... {
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
Roger Peppe (rogpeppe) wrote : | # |
Almost there, thanks! Just a few more minor fixes required.
https:/
File charm/repo.go (right):
https:/
charm/repo.go:83: /* To comply with RFC 2617, we send the auth token in
On 2013/11/08 13:08:29, rog wrote:
> Please use the conventional // comments.
not done yet...
https:/
File charm/repo.go (right):
https:/
charm/repo.go:85: req.Header.
"+s.authToken)
I still think that we should follow the letter of rfc 2617 and use the
attribute=value form (probably with "token" for the attribute. Apart
from anything else, it means we're free to add a "realm=" specifier at
some point (also keeping within the RFC).
https:/
File cmd/juju/main.go (right):
https:/
cmd/juju/
authorization token (if any)
Please use // comments to fit in with the usual style.
Also, doc comments should be whole sentences.
Something like this, perhaps:
// AuthorizeCharmRepo returns a repository that adds
// authorization from the given configuration
// to the given repository.
One other thing: I don't think this is universal enough
to warrant inclusion in main.go. I'd put it just next to
one of the places that it's used.
https:/
File store/server.go (right):
https:/
store/server.go:67: log.Errorf("Invalid authentication scheme: %s",
scheme)
If you wrote some of this information to the response writer, it would
be easy to write a test for this functionality (something that's
probably worth doing, even if the specific test will change in time).
Kapil Thangavelu (hazmat) wrote : | # |
Looks like there's a conflict marker in the merge.
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
Roger Peppe (rogpeppe) wrote : | # |
Looking good. Just a few more minor things and we'll soon be there.
Thanks for bearing with me.
https:/
File charm/repo.go (right):
https:/
charm/repo.go:68: // Return a copy of the store with auth token set
// WithAuthToken returns a copy of the store with the
// authorization token set.
(doc comments should always be full sentences)
https:/
charm/repo.go:69: func (s *CharmStore) WithAuthToken(token string)
Repository {
Maybe we should call this WithAuthAttrs (and rename "token" to
"authAttrs" now that that's what it is, and document in this method that
it should be a list of attr=value pairs.
https:/
File cmd/juju/
https:/
cmd/juju/
specified.
How does this comment relate to the statement it's attached to?
https:/
File environs/
https:/
environs/
"development", of types string
We should mention charm-store-auth here.
https:/
environs/
key=value pairs.
Or perhaps (please verify the regexp though, and perhaps make it
stricter):
var validAuthToken = regexp.
then check with validAuthToken.
https:/
File environs/
https:/
environs/
I'd like to see a few more test cases here - there are quite a few ways
the code could get it wrong.
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
William Reade (fwereade) wrote : | # |
LGTM so long as the API-side work is the next branch (ie ServiceDeploy
and ServiceSetCharm, which may themselves hit the charm store).
I'd like it most if the two sides landed together, tbh, because this
work will break unexpectedly as soon as the CLI deploy work moves behind
the API -- but if the followup is in progress I can live with it.
https:/
File charm/repo.go (right):
https:/
charm/repo.go:70: func (s *CharmStore) WithAuthAttrs(
Repository {
I'm not totally wild about just passing in the header content -- what's
the ultimate source of this? Do people have to specify the exact header
contents in their config? Can we maybe provide a slightly cleaner
experience there?
That said, I keep forgetting... this is just a first cut for useful
demoability.
Roger Peppe (rogpeppe) wrote : | # |
LGTM modulo the below.
Thanks!
https:/
File environs/
https:/
environs/
authToken)
s/'%s'/%q/
https:/
File environs/
https:/
environs/
2.",
s/2./3./
how about some more test cases?
ok cases:
"x=y,"
"x=y, \tz=w"
""
"x=y,z=w, \t"
not ok cases:
"x==y"
" x=y"
"x=y "
"=y"
"x=y =z"
see the missingAttribut
of how to do the above without duplicating all the
struct initialisation code for each case.
e.g.
// authTokenConfigTest returns a config test that checks
// that a configuration with the given auth token
// will pass or fail, depending on the value of ok.
func authTokenConfig
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
Roger Peppe (rogpeppe) wrote : | # |
Still LGTM with one trivial remark.
Please feel free to go ahead and merge with that addressed (that's what
LGTM means in general)
https:/
File environs/
https:/
environs/
token test: %q", token)
s/Valid/Invalid/
I think. otherwise there's no need to construct the message twice.
John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 2013-11-27 17:03, Roger Peppe wrote:
> Still LGTM with one trivial remark.
>
> Please feel free to go ahead and merge with that addressed (that's
> what LGTM means in general)
>
>
> https:/
>
>
File environs/
>
> https:/
>
>
environs/
> token test: %q", token) s/Valid/Invalid/ I think. otherwise there's
> no need to construct the message twice.
>
> https:/
>
I don't think Domas is in the right group to land it, so we probably
need to merge it for him once it is finished.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (Cygwin)
Comment: Using GnuPG with Thunderbird - http://
iEYEARECAAYFAlK
cjwAoIPzAC0vYjz
=YiqQ
-----END PGP SIGNATURE-----
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
Domas Monkus (tasdomas) wrote : | # |
Please take a look.
Preview Diff
1 | === modified file 'charm/export_test.go' |
2 | --- charm/export_test.go 2013-09-04 13:24:57 +0000 |
3 | +++ charm/export_test.go 2013-12-02 15:48:16 +0000 |
4 | @@ -8,5 +8,5 @@ |
5 | var IfaceExpander = ifaceExpander |
6 | |
7 | func NewStore(url string) *CharmStore { |
8 | - return &CharmStore{url} |
9 | + return &CharmStore{BaseURL: url} |
10 | } |
11 | |
12 | === modified file 'charm/repo.go' |
13 | --- charm/repo.go 2013-10-10 20:58:54 +0000 |
14 | +++ charm/repo.go 2013-12-02 15:48:16 +0000 |
15 | @@ -59,15 +59,39 @@ |
16 | |
17 | // CharmStore is a Repository that provides access to the public juju charm store. |
18 | type CharmStore struct { |
19 | - BaseURL string |
20 | -} |
21 | - |
22 | -var Store = &CharmStore{"https://store.juju.ubuntu.com"} |
23 | + BaseURL string |
24 | + authAttrs string // a list of attr=value pairs, comma separated |
25 | +} |
26 | + |
27 | +var Store = &CharmStore{BaseURL: "https://store.juju.ubuntu.com"} |
28 | + |
29 | +// WithAuthAttrs return a Repository with the authentication token list set. |
30 | +// authAttrs is a list of attr=value pairs. |
31 | +func (s *CharmStore) WithAuthAttrs(authAttrs string) Repository { |
32 | + authCS := *s |
33 | + authCS.authAttrs = authAttrs |
34 | + return &authCS |
35 | +} |
36 | + |
37 | +// Perform an http get, adding custom auth header if necessary. |
38 | +func (s *CharmStore) get(url string) (resp *http.Response, err error) { |
39 | + req, err := http.NewRequest("GET", url, nil) |
40 | + if err != nil { |
41 | + return nil, err |
42 | + } |
43 | + if s.authAttrs != "" { |
44 | + // To comply with RFC 2617, we send the authentication data in |
45 | + // the Authorization header with a custom auth scheme |
46 | + // and the authentication attributes. |
47 | + req.Header.Add("Authorization", "charmstore "+s.authAttrs) |
48 | + } |
49 | + return http.DefaultClient.Do(req) |
50 | +} |
51 | |
52 | // Info returns details for a charm in the charm store. |
53 | func (s *CharmStore) Info(curl *URL) (*InfoResponse, error) { |
54 | key := curl.String() |
55 | - resp, err := http.Get(s.BaseURL + "/charm-info?charms=" + url.QueryEscape(key)) |
56 | + resp, err := s.get(s.BaseURL + "/charm-info?charms=" + url.QueryEscape(key)) |
57 | if err != nil { |
58 | return nil, err |
59 | } |
60 | @@ -99,7 +123,7 @@ |
61 | if digest != "" { |
62 | query += "@" + digest |
63 | } |
64 | - resp, err := http.Get(s.BaseURL + "/charm-event?charms=" + url.QueryEscape(query)) |
65 | + resp, err := s.get(s.BaseURL + "/charm-event?charms=" + url.QueryEscape(query)) |
66 | if err != nil { |
67 | return nil, err |
68 | } |
69 | @@ -240,7 +264,7 @@ |
70 | } |
71 | path := filepath.Join(CacheDir, Quote(curl.String())+".charm") |
72 | if verify(path, digest) != nil { |
73 | - resp, err := http.Get(s.BaseURL + "/charm/" + url.QueryEscape(curl.Path())) |
74 | + resp, err := s.get(s.BaseURL + "/charm/" + url.QueryEscape(curl.Path())) |
75 | if err != nil { |
76 | return nil, err |
77 | } |
78 | |
79 | === modified file 'cmd/juju/deploy.go' |
80 | --- cmd/juju/deploy.go 2013-10-08 14:53:20 +0000 |
81 | +++ cmd/juju/deploy.go 2013-12-02 15:48:16 +0000 |
82 | @@ -9,10 +9,10 @@ |
83 | "os" |
84 | |
85 | "launchpad.net/gnuflag" |
86 | - |
87 | "launchpad.net/juju-core/charm" |
88 | "launchpad.net/juju-core/cmd" |
89 | "launchpad.net/juju-core/constraints" |
90 | + "launchpad.net/juju-core/environs/config" |
91 | "launchpad.net/juju-core/juju" |
92 | "launchpad.net/juju-core/juju/osenv" |
93 | "launchpad.net/juju-core/names" |
94 | @@ -128,6 +128,9 @@ |
95 | if err != nil { |
96 | return err |
97 | } |
98 | + |
99 | + repo = AuthorizeCharmRepo(repo, conf) |
100 | + |
101 | // TODO(fwereade) it's annoying to roundtrip the bytes through the client |
102 | // here, but it's the original behaviour and not convenient to change. |
103 | // PutCharm will always be required in some form for local charms; and we |
104 | @@ -173,3 +176,15 @@ |
105 | }) |
106 | return err |
107 | } |
108 | + |
109 | +// AuthorizeCharmRepo returns a repository with authentication added |
110 | +// from the specified configuration. |
111 | +func AuthorizeCharmRepo(repo charm.Repository, cfg *config.Config) charm.Repository { |
112 | + // If a charm store auth token is set, pass it on to the charm store |
113 | + if auth := cfg.CharmStoreAuth(); auth != "" { |
114 | + if CS, isCS := repo.(*charm.CharmStore); isCS { |
115 | + repo = CS.WithAuthAttrs(auth) |
116 | + } |
117 | + } |
118 | + return repo |
119 | +} |
120 | |
121 | === modified file 'cmd/juju/upgradecharm.go' |
122 | --- cmd/juju/upgradecharm.go 2013-10-02 21:36:45 +0000 |
123 | +++ cmd/juju/upgradecharm.go 2013-12-02 15:48:16 +0000 |
124 | @@ -111,6 +111,12 @@ |
125 | if err != nil { |
126 | return err |
127 | } |
128 | + |
129 | + conf, err := conn.State.EnvironConfig() |
130 | + if err != nil { |
131 | + return err |
132 | + } |
133 | + |
134 | oldURL, _ := service.CharmURL() |
135 | var newURL *charm.URL |
136 | if c.SwitchURL != "" { |
137 | @@ -131,6 +137,9 @@ |
138 | if err != nil { |
139 | return err |
140 | } |
141 | + |
142 | + repo = AuthorizeCharmRepo(repo, conf) |
143 | + |
144 | // If no explicit revision was set with either SwitchURL |
145 | // or Revision flags, discover the latest. |
146 | explicitRevision := true |
147 | |
148 | === modified file 'environs/config/config.go' |
149 | --- environs/config/config.go 2013-11-28 09:09:30 +0000 |
150 | +++ environs/config/config.go 2013-12-02 15:48:16 +0000 |
151 | @@ -8,6 +8,7 @@ |
152 | "io/ioutil" |
153 | "os" |
154 | "path/filepath" |
155 | + "regexp" |
156 | "strings" |
157 | "time" |
158 | |
159 | @@ -82,8 +83,8 @@ |
160 | // |
161 | // The required keys (after any files have been read) are "name", |
162 | // "type" and "authorized-keys", all of type string. Additional keys |
163 | -// recognised are "agent-version" and "development", of types string |
164 | -// and bool respectively. |
165 | +// recognised are "agent-version" (string) and "development" (bool) as |
166 | +// well as charm-store-auth (string containing comma-separated key=value pairs). |
167 | func New(withDefaults Defaulting, attrs map[string]interface{}) (*Config, error) { |
168 | checker := noDefaultsChecker |
169 | if withDefaults { |
170 | @@ -235,6 +236,14 @@ |
171 | } |
172 | } |
173 | |
174 | + // Ensure that the auth token is a set of key=value pairs. |
175 | + authToken := cfg.CharmStoreAuth() |
176 | + validAuthToken := regexp.MustCompile(`^([^\s=]+=[^\s=]+(,\s*)?)*$`) |
177 | + if !validAuthToken.MatchString(authToken) { |
178 | + return fmt.Errorf("charm store auth token needs to be a set"+ |
179 | + " of key-value pairs, not %q", authToken) |
180 | + } |
181 | + |
182 | // Check the immutable config values. These can't change |
183 | if old != nil { |
184 | for _, attr := range immutableAttributes { |
185 | @@ -478,6 +487,11 @@ |
186 | return c.asString("logging-config") |
187 | } |
188 | |
189 | +// Auth token sent to charm store |
190 | +func (c *Config) CharmStoreAuth() string { |
191 | + return c.asString("charm-store-auth") |
192 | +} |
193 | + |
194 | // ProvisionerSafeMode reports whether the provisioner should not |
195 | // destroy machines it does not know about. |
196 | func (c *Config) ProvisionerSafeMode() bool { |
197 | @@ -536,6 +550,7 @@ |
198 | "api-port": schema.ForceInt(), |
199 | "syslog-port": schema.ForceInt(), |
200 | "logging-config": schema.String(), |
201 | + "charm-store-auth": schema.String(), |
202 | "provisioner-safe-mode": schema.Bool(), |
203 | |
204 | // Deprecated fields, retain for backwards compatibility. |
205 | @@ -578,6 +593,8 @@ |
206 | "state-port": DefaultStatePort, |
207 | "api-port": DefaultAPIPort, |
208 | "syslog-port": DefaultSyslogPort, |
209 | + // Authentication string sent with requests to the charm store |
210 | + "charm-store-auth": "", |
211 | } |
212 | |
213 | func allowEmpty(attr string) bool { |
214 | |
215 | === modified file 'environs/config/config_test.go' |
216 | --- environs/config/config_test.go 2013-11-28 09:11:12 +0000 |
217 | +++ environs/config/config_test.go 2013-12-02 15:48:16 +0000 |
218 | @@ -5,6 +5,7 @@ |
219 | |
220 | import ( |
221 | "fmt" |
222 | + "regexp" |
223 | "strings" |
224 | stdtesting "testing" |
225 | "time" |
226 | @@ -555,6 +556,20 @@ |
227 | "type": "ec2", |
228 | }, |
229 | }, |
230 | + authTokenConfigTest("token=value, tokensecret=value", true), |
231 | + authTokenConfigTest("token=value, ", true), |
232 | + authTokenConfigTest("token=value, \ttokensecret=value", true), |
233 | + authTokenConfigTest("", true), |
234 | + authTokenConfigTest("token=value, tokensecret=value, \t", true), |
235 | + authTokenConfigTest("=", false), |
236 | + authTokenConfigTest("tokenvalue", false), |
237 | + authTokenConfigTest("token=value, sometoken=", false), |
238 | + authTokenConfigTest("token==value", false), |
239 | + authTokenConfigTest(" token=value", false), |
240 | + authTokenConfigTest("=value", false), |
241 | + authTokenConfigTest("token=value, =z", false), |
242 | + authTokenConfigTest("token=value =z", false), |
243 | + authTokenConfigTest("\t", false), |
244 | missingAttributeNoDefault("default-series"), |
245 | missingAttributeNoDefault("firewall-mode"), |
246 | missingAttributeNoDefault("development"), |
247 | @@ -565,6 +580,28 @@ |
248 | // missingAttributeNoDefault("api-port"), |
249 | } |
250 | |
251 | +// authTokenConfigTest returns a config test that checks |
252 | +// that a configuration with the given auth token |
253 | +// will pass or fail, depending on the value of ok. |
254 | +func authTokenConfigTest(token string, ok bool) configTest { |
255 | + var testName string |
256 | + var err string |
257 | + |
258 | + if ok { |
259 | + testName = fmt.Sprintf("Valid auth token test: %q", token) |
260 | + } else { |
261 | + testName = fmt.Sprintf("Invalid auth token test: %q", token) |
262 | + err = fmt.Sprintf("charm store auth token needs to be a set of key-value pairs, not %q", token) |
263 | + } |
264 | + |
265 | + return configTest{ |
266 | + about: testName, |
267 | + useDefaults: config.UseDefaults, |
268 | + attrs: sampleConfig.Merge(testing.Attrs{"charm-store-auth": token}), |
269 | + err: regexp.QuoteMeta(err), |
270 | + } |
271 | +} |
272 | + |
273 | func missingAttributeNoDefault(attrName string) configTest { |
274 | return configTest{ |
275 | about: fmt.Sprintf("No default: missing %s", attrName), |
276 | @@ -853,6 +890,7 @@ |
277 | "api-port": 4321, |
278 | "syslog-port": 2345, |
279 | "default-series": "precise", |
280 | + "charm-store-auth": "token=auth", |
281 | } |
282 | cfg, err := config.New(config.NoDefaults, attrs) |
283 | c.Assert(err, gc.IsNil) |
state/apiserver /client/ client. go also uses the charm store, and should
use the auth token. That's not hard in itself, but it's maybe a sign
that we should factor it a little differently -- should we maybe be
passing a CharmStoreAuther into InferRepository? That'd get around the
what-if-it's-global problem I allude to below, at the cost of uglifying
an already ugly func... but I think it's the lesser sin. Thoughts?
https:/ /codereview. appspot. com/22390043/ diff/1/ cmd/juju/ deploy. go
File cmd/juju/deploy.go (right):
https:/ /codereview. appspot. com/22390043/ diff/1/ cmd/juju/ deploy. go#newcode135 deploy. go:135: CS.SetAuthToken (auth)
cmd/juju/
I'm a bit twitchy about setting the auth token on what is likely to be
the global charm store object.
https:/ /codereview. appspot. com/22390043/