Merge ~ubuntu-docker-images/ubuntu-docker-images/+git/utils:multi-arch-tagger into ~ubuntu-docker-images/ubuntu-docker-images/+git/utils:master
- Git
- lp:~ubuntu-docker-images/ubuntu-docker-images/+git/utils
- multi-arch-tagger
- Merge into master
Status: | Merged |
---|---|
Merged at revision: | 75a4c7e37ead57073f4020eb71df9c053724650f |
Proposed branch: | ~ubuntu-docker-images/ubuntu-docker-images/+git/utils:multi-arch-tagger |
Merge into: | ~ubuntu-docker-images/ubuntu-docker-images/+git/utils:master |
Diff against target: |
1196 lines (+1148/-0) 8 files modified
README.multi-arch-tagger.md (+67/-0) helpers/log.sh (+49/-0) helpers/registry-login.sh (+122/-0) helpers/validate-args.sh (+85/-0) list-all-images.sh (+129/-0) list-tags-for-image.sh (+141/-0) multi-arch-tagger.sh (+220/-0) tag-images.sh (+335/-0) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Paride Legovini | Approve | ||
Bryce Harrington | Approve | ||
Review via email: mp+399681@code.launchpad.net |
Commit message
Description of the change
Implement a multi-architecture tagger framework.
This MP consists of a series of scripts that work together to implement the multi-architecture tagging, which is one of the "missing pieces" for our OCI work.
The scripts are a bit complex, and they all work independently. The main script here is the "tag-images.sh", which encapsulates all the logic needed to properly create the "latest" and the "M.N-XX.YY_beta" tags (not to mention the special "ubuntu" base image).
I've tested this on some images and it's working OK, but most of my test was done using "echo" statements to make sure that the right commands would be invoked. We're still not ready to mass-tag everything, but I don't foresee any problems.
The aws-cli tool needs to be installed and configured locally in order to communicate with the AWS service. I've written some instructions on how to configure it in the README file.
Bryce Harrington (bryce) wrote : | # |
Sergio Durigan Junior (sergiodj) wrote : | # |
Thanks for the initial review, Bryce. Comments inline, and new version force-pushed.
Bryce Harrington (bryce) wrote : | # |
Ok, I've had a chance to play with the script in earnest for a bit (just enough to be usefully confused), and have some usage suggestions.
Before I get too far into it, I'll mention upfront that if this is going to be used entirely non-interactively, then a lot of the remarks in this comment are probably going to be irrelevant. However, if you anticipate anyone might be running this by hand, then they may be worthwhile.
And first, here's how I'm running things:
stirling:
Username: bryceharrington
Password:
ubuntu/nginx
ubuntu/redis
ubuntu/postgres
ubuntu/mysql
ubuntu/apache2
ubuntu/memcached
ubuntu/grafana
ubuntu/prometheus
ubuntu/cortex
ubuntu/telegraf
ubuntu/
$ ./list-
Username: bryceharrington
Password:
ubuntu/redis:latest
ubuntu/
ubuntu/redis:edge
ubuntu/
ubuntu/
ubuntu/
One thing that's a bit confusing is after running ./list-
stirling:
Username: bryceharrington
Password:
E: Invalid image name 'ubuntu/redis'.
I'd probably just chalk that up to user error, but wouldn't be surprised if other rookie users make the same mistake. Again, if this is mainly intended to be used in an automated workflow, it's probably better to not have too many smarts in the argument parsing logic. You've got the --no-prefix argument if someone really needs the bare image name.
stirling:
Username: bryceharrington
Password:
I: Tagging ubuntu/redis:20.04 as ubuntu/redis:foobar (on docker)
ERROR: Image tagging failed with status code 400
So first comment is I'm not entirely certain this is the right way to be calling this tool. Given the large number of required parameters it would help a lot to have an example or two in the help text (or in a man page). For this tool, maybe one cut-and-paste command line to test it, and then one real-world usage.
Second, I'd kind of like to be able to run this against a throwaway image in my own namespace, just to test it, but passing '-n bryceharrington' gives an error. If that's intended/desired then a --dry-run parameter might be handy to add.
Third, I was initially a bit confused why there are --username and --password arguments, yet it still prompts me for username and password. From the help text I got the impression they were two different sets of usernames and passwords, but still a bit confusing what they might be. Also, taking a password on the command line might have some security implications, although I gather this is just for testing convenience and the auth token is used more in practice?
And fourth, the error message with status code 400 is a bit cryptic. That said, I'll bet it's not really going to be possible to give a better error message since that c...
Sergio Durigan Junior (sergiodj) wrote : | # |
On Tuesday, March 16 2021, Bryce Harrington wrote:
> Review: Approve
>
> Ok, I've had a chance to play with the script in earnest for a bit (just enough to be usefully confused), and have some usage suggestions.
Thanks for the thorough review, Bryce.
I'm replying by email because it's easier to do inline comments.
> Before I get too far into it, I'll mention upfront that if this is
> going to be used entirely non-interactively, then a lot of the remarks
> in this comment are probably going to be irrelevant. However, if you
> anticipate anyone might be running this by hand, then they may be
> worthwhile.
Yeah, I think the only script that will be "heavily" used in an
interactive way is the "tag-images.sh", and *perhaps* the
"multi-
interactively much, although I wanted them to support that as well.
> And first, here's how I'm running things:
>
> stirling:
> Username: bryceharrington
> Password:
> ubuntu/nginx
> ubuntu/redis
> ubuntu/postgres
> ubuntu/mysql
> ubuntu/apache2
> ubuntu/memcached
> ubuntu/grafana
> ubuntu/prometheus
> ubuntu/cortex
> ubuntu/telegraf
> ubuntu/
Cool!
> $ ./list-
> Username: bryceharrington
> Password:
> ubuntu/redis:latest
> ubuntu/
> ubuntu/redis:edge
> ubuntu/
> ubuntu/
> ubuntu/
Cool again! :-)
> One thing that's a bit confusing is after running
> ./list-
> its input to pass to ./list-
>
> stirling:
> Username: bryceharrington
> Password:
> E: Invalid image name 'ubuntu/redis'.
Yeah, that doesn't work because you're already specifying the namespace
via the "-n" parameter, and that will prepended to the image name when
composing the URL used to retrieve the information. I mean, I can
understand the initial confusion, but once you have a clearer picture of
what an "image", a "namespace" and a "registry" are, I think it makes
more sense.
On a side note, I've been thinking about reversing the "--no-prefix"
option and making it the default, because that's what we usually want as
an output.
> I'd probably just chalk that up to user error, but wouldn't be
> surprised if other rookie users make the same mistake. Again, if this
> is mainly intended to be used in an automated workflow, it's probably
> better to not have too many smarts in the argument parsing logic.
> You've got the --no-prefix argument if someone really needs the bare
> image name.
I'm expecting some of these scripts to be used interactively, as I
mentioned above, so maybe it's a good idea to write a more comprehensive
documentation. I don't think nothing fancy is needed; maybe just
extending the README file will be enough.
> stirling:
> Username: bryceharrington
...
Sergio Durigan Junior (sergiodj) : | # |
Bryce Harrington (bryce) : | # |
Paride Legovini (paride) wrote : | # |
Great work with these scripts Sergio! I did a shellcheck run and they're almost clean, just with some nits like:
SC2086: Double quote to prevent globbing and word splitting.
here and there, mostly about "safe" variables known to have no spaces. There are also some false positives I think . Given the high quality of the scripts it may be worth to make shellcheck fully happy, and test future MPs (possibly in CI) to keep quality high.
Sergio Durigan Junior (sergiodj) wrote : | # |
Thanks for the review, Paride :-).
I have silenced everything that shellcheck flagged, and I have also addressed most (if not all) of the comments that Bryce has made.
I am merging the script now, because it is ready to be used. We can always come back and improve it if needed.
Preview Diff
1 | diff --git a/README.multi-arch-tagger.md b/README.multi-arch-tagger.md | |||
2 | 0 | new file mode 100644 | 0 | new file mode 100644 |
3 | index 0000000..dd33eee | |||
4 | --- /dev/null | |||
5 | +++ b/README.multi-arch-tagger.md | |||
6 | @@ -0,0 +1,67 @@ | |||
7 | 1 | Multi-arch tagger | ||
8 | 2 | ================= | ||
9 | 3 | |||
10 | 4 | The following scripts are part of the multi-architecture tagger | ||
11 | 5 | application. They are used to tag multiple architectures of an OCI | ||
12 | 6 | image without having to actually run `docker tag` in those | ||
13 | 7 | architectures. | ||
14 | 8 | |||
15 | 9 | - `list-all-images.sh`: This script can be used to list all images | ||
16 | 10 | from a certain registry/namespace combination. | ||
17 | 11 | |||
18 | 12 | - `list-tags-for-image.sh`: This script can be used to list all tags | ||
19 | 13 | pertaining to a certain image from a registry/namespace. | ||
20 | 14 | |||
21 | 15 | - `multi-arch-tagger.sh`: This script can be used to perform the | ||
22 | 16 | multi-architecture tagging of a certain image from a | ||
23 | 17 | registry/namespace. | ||
24 | 18 | |||
25 | 19 | - `tag-images.sh`: This script is a wrapper on top of | ||
26 | 20 | `multi-arch-tagger.sh`; it iterates over the available images from a | ||
27 | 21 | certain registry/namespace combination and tags them. | ||
28 | 22 | |||
29 | 23 | All of these scripts accept a `--help` argument and print their | ||
30 | 24 | usages, so it should not be hard to understand how to use them. | ||
31 | 25 | |||
32 | 26 | Most of the time, you will actually just use the `tag-images.sh` | ||
33 | 27 | script, which invokes all the other scripts in order to do its job. | ||
34 | 28 | |||
35 | 29 | Requirements to run these scripts | ||
36 | 30 | --------------------------------- | ||
37 | 31 | |||
38 | 32 | You have to have the following programs installed in your system in | ||
39 | 33 | order to run the scripts: | ||
40 | 34 | |||
41 | 35 | - jq | ||
42 | 36 | - curl | ||
43 | 37 | - awscli | ||
44 | 38 | |||
45 | 39 | You don't need `docker` installed, since we use API endpoints when | ||
46 | 40 | communicating with Dockerhub. AWS doesn't support most of the API | ||
47 | 41 | endpoints yet, so that is why you need `awscli`. | ||
48 | 42 | |||
49 | 43 | Configuring awscli for tag-images.sh | ||
50 | 44 | ------------------------------------- | ||
51 | 45 | |||
52 | 46 | The tag-images.sh script requires that you install and configure | ||
53 | 47 | `awscli` in order to tag images on AWS. | ||
54 | 48 | |||
55 | 49 | To install the package: | ||
56 | 50 | |||
57 | 51 | ``` | ||
58 | 52 | $ apt install awscli | ||
59 | 53 | ``` | ||
60 | 54 | |||
61 | 55 | To configure the software, you will need to create two profiles in | ||
62 | 56 | order to use the tagger. | ||
63 | 57 | |||
64 | 58 | ``` | ||
65 | 59 | $ aws configure --profile ubuntu | ||
66 | 60 | ... | ||
67 | 61 | |||
68 | 62 | $ aws configure --profile lts | ||
69 | 63 | ... | ||
70 | 64 | ``` | ||
71 | 65 | |||
72 | 66 | Each profile needs to be properly configured using the corresponding | ||
73 | 67 | access keys, of course. | ||
74 | diff --git a/helpers/log.sh b/helpers/log.sh | |||
75 | 0 | new file mode 100644 | 68 | new file mode 100644 |
76 | index 0000000..e3e6b9b | |||
77 | --- /dev/null | |||
78 | +++ b/helpers/log.sh | |||
79 | @@ -0,0 +1,49 @@ | |||
80 | 1 | #!/bin/bash | ||
81 | 2 | |||
82 | 3 | # Generic logging function. | ||
83 | 4 | _log () | ||
84 | 5 | { | ||
85 | 6 | local type="$1" | ||
86 | 7 | local msg="$2" | ||
87 | 8 | local redir=/dev/stdout | ||
88 | 9 | |||
89 | 10 | case "$type" in | ||
90 | 11 | "E"|"D") | ||
91 | 12 | redir=/dev/stderr | ||
92 | 13 | ;; | ||
93 | 14 | "I"|"W") | ||
94 | 15 | : | ||
95 | 16 | ;; | ||
96 | 17 | *) | ||
97 | 18 | # Can't call error here, can we? | ||
98 | 19 | echo "E: Internal error in the '_log' function!" \ | ||
99 | 20 | > /dev/stderr | ||
100 | 21 | ;; | ||
101 | 22 | esac | ||
102 | 23 | |||
103 | 24 | printf "%s: %s\n" "$type" "$msg" > $redir | ||
104 | 25 | } | ||
105 | 26 | |||
106 | 27 | # Error logging. | ||
107 | 28 | error () | ||
108 | 29 | { | ||
109 | 30 | _log "E" "$1" | ||
110 | 31 | } | ||
111 | 32 | |||
112 | 33 | # Warn logging. | ||
113 | 34 | warn () | ||
114 | 35 | { | ||
115 | 36 | _log "W" "$1" | ||
116 | 37 | } | ||
117 | 38 | |||
118 | 39 | # Info logging. | ||
119 | 40 | info () | ||
120 | 41 | { | ||
121 | 42 | _log "I" "$1" | ||
122 | 43 | } | ||
123 | 44 | |||
124 | 45 | # Debug logging. | ||
125 | 46 | debug () | ||
126 | 47 | { | ||
127 | 48 | _log "D" "$1" | ||
128 | 49 | } | ||
129 | diff --git a/helpers/registry-login.sh b/helpers/registry-login.sh | |||
130 | 0 | new file mode 100644 | 50 | new file mode 100644 |
131 | index 0000000..fa68362 | |||
132 | --- /dev/null | |||
133 | +++ b/helpers/registry-login.sh | |||
134 | @@ -0,0 +1,122 @@ | |||
135 | 1 | #!/bin/bash | ||
136 | 2 | |||
137 | 3 | # Log into a registry. | ||
138 | 4 | |||
139 | 5 | set -e | ||
140 | 6 | |||
141 | 7 | readonly AWS_URL="https://public.ecr.aws" | ||
142 | 8 | readonly AWS_REGISTRY_URL="https://public.ecr.aws" | ||
143 | 9 | export AWS_URL AWS_REGISTRY_URL | ||
144 | 10 | |||
145 | 11 | readonly DOCKERHUB_URL="https://hub.docker.com" | ||
146 | 12 | readonly DOCKERHUB_REGISTRY_URL="https://registry-1.docker.io" | ||
147 | 13 | export DOCKERHUB_URL DOCKERHUB_REGISTRY_URL | ||
148 | 14 | |||
149 | 15 | # Log into dockerhub, asking the user and the password. | ||
150 | 16 | # | ||
151 | 17 | # This function doesn't take arguments. It sets the AUTH_TOKEN | ||
152 | 18 | # environment variable to the Authentication Token that can be used on | ||
153 | 19 | # further requests. | ||
154 | 20 | _login_docker () | ||
155 | 21 | { | ||
156 | 22 | if [ -z "$USERNAME" ]; then | ||
157 | 23 | read -rp "Username: " USERNAME | ||
158 | 24 | fi | ||
159 | 25 | if [ -z "$PASSWORD" ]; then | ||
160 | 26 | read -rsp "Password: " PASSWORD | ||
161 | 27 | echo > /dev/stderr | ||
162 | 28 | fi | ||
163 | 29 | |||
164 | 30 | AUTH_TOKEN=$(curl -s \ | ||
165 | 31 | -H "Content-Type: application/json" \ | ||
166 | 32 | -X POST -d '{"username": "'"${USERNAME}"'", "password": "'"${PASSWORD}"'"}' \ | ||
167 | 33 | "${DOCKERHUB_URL}/v2/users/login/" | jq -r .token) | ||
168 | 34 | |||
169 | 35 | if [ -z "$AUTH_TOKEN" ]; then | ||
170 | 36 | error "There was an error while logging into the service." | ||
171 | 37 | exit 1 | ||
172 | 38 | fi | ||
173 | 39 | |||
174 | 40 | if [ -n "$DEBUG" ]; then | ||
175 | 41 | debug "This is the Dockerhub authentication token I got, in case you want to reuse it:" | ||
176 | 42 | printf 'AUTH_TOKEN=%s\n' "$AUTH_TOKEN" > /dev/stderr | ||
177 | 43 | fi | ||
178 | 44 | |||
179 | 45 | REGISTRY_USERNAME="$USERNAME" | ||
180 | 46 | REGISTRY_PASSWORD="$PASSWORD" | ||
181 | 47 | } | ||
182 | 48 | |||
183 | 49 | # Log into dockerhub's registry1, asking the user and the password. | ||
184 | 50 | # | ||
185 | 51 | # It takes the list of images to be used as the scope as argument. It | ||
186 | 52 | # assumes that the image names are not prefixed with "$NAMESPACE/" | ||
187 | 53 | _login_docker_registry1 () | ||
188 | 54 | { | ||
189 | 55 | local IMAGES_FOR_SCOPE="$1" | ||
190 | 56 | local REGISTRY_USERNAME="$2" | ||
191 | 57 | local REGISTRY_PASSWORD="$3" | ||
192 | 58 | local URL="https://auth.docker.io/token?service=registry.docker.io" | ||
193 | 59 | |||
194 | 60 | if [ -z "$REGISTRY_USERNAME" ] || [ -z "$REGISTRY_PASSWORD" ]; then | ||
195 | 61 | error "You must specify the registry username and password." | ||
196 | 62 | exit 1 | ||
197 | 63 | fi | ||
198 | 64 | |||
199 | 65 | for image in $IMAGES_FOR_SCOPE; do | ||
200 | 66 | URL+="&scope=repository:${NAMESPACE}/${image}:pull,push" | ||
201 | 67 | done | ||
202 | 68 | |||
203 | 69 | REGISTRY_AUTH_TOKEN=$(curl -s \ | ||
204 | 70 | -u "${REGISTRY_USERNAME}:${REGISTRY_PASSWORD}" \ | ||
205 | 71 | "$URL" | jq -r .token) | ||
206 | 72 | export REGISTRY_AUTH_TOKEN | ||
207 | 73 | |||
208 | 74 | if [ -z "$REGISTRY_AUTH_TOKEN" ]; then | ||
209 | 75 | error "There was an error while logging into the registry." | ||
210 | 76 | exit 1 | ||
211 | 77 | fi | ||
212 | 78 | |||
213 | 79 | export REGISTRY_URL="$DOCKERHUB_REGISTRY_URL" | ||
214 | 80 | } | ||
215 | 81 | |||
216 | 82 | _login_aws () | ||
217 | 83 | { | ||
218 | 84 | # No need to login with aws since we will be using aws-cli. | ||
219 | 85 | if ! command -v aws >& /dev/null; then | ||
220 | 86 | error "You must have the aws-cli program installed." | ||
221 | 87 | exit 1 | ||
222 | 88 | fi | ||
223 | 89 | |||
224 | 90 | if ! aws configure list --profile "$NAMESPACE" >& /dev/null; then | ||
225 | 91 | error "You must configure aws-cli with a profile named '$NAMESPACE'." | ||
226 | 92 | error "Please refer to the README file for instructions." | ||
227 | 93 | exit 1 | ||
228 | 94 | fi | ||
229 | 95 | |||
230 | 96 | export AWS_PROFILE="$NAMESPACE" | ||
231 | 97 | } | ||
232 | 98 | |||
233 | 99 | _login_aws_registry1 () | ||
234 | 100 | { | ||
235 | 101 | REGISTRY_AUTH_TOKEN=$(aws ecr-public get-authorization-token \ | ||
236 | 102 | --region us-east-1 \ | ||
237 | 103 | --output=text \ | ||
238 | 104 | --query 'authorizationData.authorizationToken') | ||
239 | 105 | |||
240 | 106 | |||
241 | 107 | if [ -z "$REGISTRY_AUTH_TOKEN" ]; then | ||
242 | 108 | error "There was an error while logging into the registry." | ||
243 | 109 | exit 1 | ||
244 | 110 | fi | ||
245 | 111 | |||
246 | 112 | REGISTRY_URL="$AWS_REGISTRY_URL" | ||
247 | 113 | } | ||
248 | 114 | |||
249 | 115 | do_login () | ||
250 | 116 | { | ||
251 | 117 | if [ -n "$AUTH_TOKEN" ]; then | ||
252 | 118 | return | ||
253 | 119 | fi | ||
254 | 120 | |||
255 | 121 | _login_"${REGISTRY}" | ||
256 | 122 | } | ||
257 | diff --git a/helpers/validate-args.sh b/helpers/validate-args.sh | |||
258 | 0 | new file mode 100644 | 123 | new file mode 100644 |
259 | index 0000000..551c6f7 | |||
260 | --- /dev/null | |||
261 | +++ b/helpers/validate-args.sh | |||
262 | @@ -0,0 +1,85 @@ | |||
263 | 1 | #!/bin/bash | ||
264 | 2 | |||
265 | 3 | progname=$(basename "${0}") | ||
266 | 4 | progdir=$(dirname "$(readlink -f "${0}")") | ||
267 | 5 | |||
268 | 6 | export progname progdir | ||
269 | 7 | |||
270 | 8 | # shellcheck disable=SC1090 | ||
271 | 9 | . "${progdir}/helpers/log.sh" | ||
272 | 10 | |||
273 | 11 | validate_var () | ||
274 | 12 | { | ||
275 | 13 | local varname="$1" | ||
276 | 14 | local varvalue | ||
277 | 15 | |||
278 | 16 | varvalue=$(eval "echo \$${varname}") | ||
279 | 17 | |||
280 | 18 | if [ -z "$varvalue" ]; then | ||
281 | 19 | error "You must specify the ${varname} parameter." | ||
282 | 20 | return 1 | ||
283 | 21 | fi | ||
284 | 22 | |||
285 | 23 | return 0 | ||
286 | 24 | } | ||
287 | 25 | |||
288 | 26 | validate_registry () | ||
289 | 27 | { | ||
290 | 28 | local registry="$1" | ||
291 | 29 | |||
292 | 30 | case "$registry" in | ||
293 | 31 | "docker"|"aws") | ||
294 | 32 | : | ||
295 | 33 | ;; | ||
296 | 34 | *) | ||
297 | 35 | error "Invalid registry: '$registry'." | ||
298 | 36 | return 1 | ||
299 | 37 | ;; | ||
300 | 38 | esac | ||
301 | 39 | |||
302 | 40 | return 0 | ||
303 | 41 | } | ||
304 | 42 | |||
305 | 43 | validate_namespace () | ||
306 | 44 | { | ||
307 | 45 | local namespace="$1" | ||
308 | 46 | |||
309 | 47 | case "$namespace" in | ||
310 | 48 | "ubuntu") | ||
311 | 49 | : | ||
312 | 50 | ;; | ||
313 | 51 | "lts") | ||
314 | 52 | # shellcheck disable=SC2153 | ||
315 | 53 | if [ "$REGISTRY" = "docker" ]; then | ||
316 | 54 | error "The lts namespace is not available on dockerhub yet." | ||
317 | 55 | return 1 | ||
318 | 56 | fi | ||
319 | 57 | ;; | ||
320 | 58 | *) | ||
321 | 59 | error "Invalid namespace: '$namespace:'." | ||
322 | 60 | return 1 | ||
323 | 61 | ;; | ||
324 | 62 | esac | ||
325 | 63 | |||
326 | 64 | return 0 | ||
327 | 65 | } | ||
328 | 66 | |||
329 | 67 | validate_image () | ||
330 | 68 | { | ||
331 | 69 | local image="$1" | ||
332 | 70 | local -a ALL_IMAGES | ||
333 | 71 | |||
334 | 72 | # shellcheck disable=SC2153 | ||
335 | 73 | # Fill the ALL_IMAGES array. | ||
336 | 74 | readarray -t ALL_IMAGES < <("${progdir}/list-all-images.sh" \ | ||
337 | 75 | -r "$REGISTRY" \ | ||
338 | 76 | -n "$NAMESPACE" \ | ||
339 | 77 | -a "$AUTH_TOKEN") | ||
340 | 78 | |||
341 | 79 | if ! grep -Fwq "$image" <<< "${ALL_IMAGES[@]}"; then | ||
342 | 80 | error "Invalid image name '$image'." | ||
343 | 81 | return 1 | ||
344 | 82 | fi | ||
345 | 83 | |||
346 | 84 | return 0 | ||
347 | 85 | } | ||
348 | diff --git a/list-all-images.sh b/list-all-images.sh | |||
349 | 0 | new file mode 100755 | 86 | new file mode 100755 |
350 | index 0000000..c82b50c | |||
351 | --- /dev/null | |||
352 | +++ b/list-all-images.sh | |||
353 | @@ -0,0 +1,129 @@ | |||
354 | 1 | #!/bin/bash | ||
355 | 2 | |||
356 | 3 | set -e | ||
357 | 4 | |||
358 | 5 | progname=$(basename "${0}") | ||
359 | 6 | progdir=$(dirname "$(readlink -f "${0}")") | ||
360 | 7 | |||
361 | 8 | # shellcheck disable=SC1090 | ||
362 | 9 | . "${progdir}/helpers/validate-args.sh" | ||
363 | 10 | # shellcheck disable=SC1090 | ||
364 | 11 | . "${progdir}/helpers/log.sh" | ||
365 | 12 | # shellcheck disable=SC1090 | ||
366 | 13 | . "${progdir}/helpers/registry-login.sh" | ||
367 | 14 | |||
368 | 15 | # Obtain the list of images for a certain repository. | ||
369 | 16 | |||
370 | 17 | # Usage. | ||
371 | 18 | usage () | ||
372 | 19 | { | ||
373 | 20 | cat > /dev/stderr <<EOF | ||
374 | 21 | $progname -- Obtain the list of images from a certain repository. | ||
375 | 22 | |||
376 | 23 | Usage: | ||
377 | 24 | |||
378 | 25 | $progname [-a <TOKEN>] -r <docker|aws> -n <ubuntu|lts> [-d] | ||
379 | 26 | |||
380 | 27 | Arguments: | ||
381 | 28 | |||
382 | 29 | -a TOKEN|--auth-token TOKEN Specify the authorization token that | ||
383 | 30 | should be used when connecting to the | ||
384 | 31 | registry's API. If not provided, the | ||
385 | 32 | script will ask for your user/password. | ||
386 | 33 | |||
387 | 34 | -r REGISTRY|--registry REGISTRY Registry to consult. | ||
388 | 35 | Possible values: docker, aws | ||
389 | 36 | |||
390 | 37 | -n NAMESPACE|--namespace NAMESPACE: Namespace to consult. | ||
391 | 38 | Possible values: ubuntu, lts | ||
392 | 39 | |||
393 | 40 | -d|--debug Print debug statements. This includes printing | ||
394 | 41 | the authentication token if using Dockerhub. | ||
395 | 42 | |||
396 | 43 | -h|--help Display this usage | ||
397 | 44 | EOF | ||
398 | 45 | } | ||
399 | 46 | |||
400 | 47 | validate_args () | ||
401 | 48 | { | ||
402 | 49 | local ret=0 | ||
403 | 50 | |||
404 | 51 | validate_var REGISTRY || ret=1 | ||
405 | 52 | validate_var NAMESPACE || ret=1 | ||
406 | 53 | |||
407 | 54 | validate_registry "$REGISTRY" || ret=1 | ||
408 | 55 | validate_namespace "$NAMESPACE" || ret=1 | ||
409 | 56 | |||
410 | 57 | if [ "$ret" -eq 1 ]; then | ||
411 | 58 | error "Failed to validate arguments." | ||
412 | 59 | exit 1 | ||
413 | 60 | fi | ||
414 | 61 | |||
415 | 62 | do_login | ||
416 | 63 | } | ||
417 | 64 | |||
418 | 65 | print_list_of_images_docker () | ||
419 | 66 | { | ||
420 | 67 | for image in $(curl -s \ | ||
421 | 68 | -H "Authorization: Bearer ${AUTH_TOKEN}" \ | ||
422 | 69 | "${DOCKERHUB_URL}/v2/repositories/${NAMESPACE}/?page_size=10000" \ | ||
423 | 70 | | jq -r '.results|.[]|.name'); do | ||
424 | 71 | echo "$image" | ||
425 | 72 | done | ||
426 | 73 | } | ||
427 | 74 | |||
428 | 75 | print_list_of_images_aws () | ||
429 | 76 | { | ||
430 | 77 | aws --region us-east-1 ecr-public describe-repositories \ | ||
431 | 78 | | jq -r '.repositories[].repositoryName' | ||
432 | 79 | } | ||
433 | 80 | |||
434 | 81 | print_list_of_images () | ||
435 | 82 | { | ||
436 | 83 | print_list_of_images_"${REGISTRY}" | ||
437 | 84 | } | ||
438 | 85 | |||
439 | 86 | if [ $# -lt 2 ]; then | ||
440 | 87 | error "You need to provide at least the registry (-r) and the namespace (-n) arguments." | ||
441 | 88 | usage | ||
442 | 89 | exit 1 | ||
443 | 90 | fi | ||
444 | 91 | |||
445 | 92 | while [ -n "$1" ]; do | ||
446 | 93 | case "$1" in | ||
447 | 94 | "-a"|"--auth-token") | ||
448 | 95 | AUTH_TOKEN="$2" | ||
449 | 96 | shift 2 | ||
450 | 97 | ;; | ||
451 | 98 | |||
452 | 99 | "-r"|"--registry") | ||
453 | 100 | REGISTRY="$2" | ||
454 | 101 | shift 2 | ||
455 | 102 | ;; | ||
456 | 103 | |||
457 | 104 | "-n"|"--namespace") | ||
458 | 105 | NAMESPACE="$2" | ||
459 | 106 | shift 2 | ||
460 | 107 | ;; | ||
461 | 108 | |||
462 | 109 | "-d"|"--debug") | ||
463 | 110 | export DEBUG=1 | ||
464 | 111 | shift | ||
465 | 112 | ;; | ||
466 | 113 | |||
467 | 114 | "-h"|"--help") | ||
468 | 115 | usage | ||
469 | 116 | exit 0 | ||
470 | 117 | ;; | ||
471 | 118 | |||
472 | 119 | *) | ||
473 | 120 | error "Invalid argument '$1'." | ||
474 | 121 | usage | ||
475 | 122 | exit 1 | ||
476 | 123 | ;; | ||
477 | 124 | esac | ||
478 | 125 | done | ||
479 | 126 | |||
480 | 127 | validate_args | ||
481 | 128 | |||
482 | 129 | print_list_of_images | ||
483 | diff --git a/list-tags-for-image.sh b/list-tags-for-image.sh | |||
484 | 0 | new file mode 100755 | 130 | new file mode 100755 |
485 | index 0000000..305c9be | |||
486 | --- /dev/null | |||
487 | +++ b/list-tags-for-image.sh | |||
488 | @@ -0,0 +1,141 @@ | |||
489 | 1 | #!/bin/bash | ||
490 | 2 | |||
491 | 3 | set -e | ||
492 | 4 | |||
493 | 5 | progname=$(basename "${0}") | ||
494 | 6 | progdir=$(dirname "$(readlink -f "${0}")") | ||
495 | 7 | |||
496 | 8 | # shellcheck disable=SC1090 | ||
497 | 9 | . "${progdir}/helpers/validate-args.sh" | ||
498 | 10 | # shellcheck disable=SC1090 | ||
499 | 11 | . "${progdir}/helpers/log.sh" | ||
500 | 12 | # shellcheck disable=SC1090 | ||
501 | 13 | . "${progdir}/helpers/registry-login.sh" | ||
502 | 14 | |||
503 | 15 | # Obtain the list of tags for a certain image. | ||
504 | 16 | |||
505 | 17 | # Usage. | ||
506 | 18 | usage () | ||
507 | 19 | { | ||
508 | 20 | cat > /dev/stderr <<EOF | ||
509 | 21 | $progname -- Obtain the list of tags from a certain image. | ||
510 | 22 | |||
511 | 23 | Usage: | ||
512 | 24 | |||
513 | 25 | $progname [-a <TOKEN>] -r <docker|aws> -n <ubuntu|lts> -i <IMAGE> [-d] | ||
514 | 26 | |||
515 | 27 | Arguments: | ||
516 | 28 | |||
517 | 29 | -a TOKEN|--auth-token TOKEN Specify the authorization token that | ||
518 | 30 | should be used when connecting to the | ||
519 | 31 | registry's API. If not provided, the | ||
520 | 32 | script will ask for your user/password. | ||
521 | 33 | |||
522 | 34 | -r REGISTRY|--registry REGISTRY Registry to consult. | ||
523 | 35 | Possible values: docker, aws | ||
524 | 36 | |||
525 | 37 | -n NAMESPACE|--namespace NAMESPACE Namespace to consult. | ||
526 | 38 | Possible values: ubuntu, lts | ||
527 | 39 | |||
528 | 40 | -i IMAGE|--image IMAGE Image to obtain tags from. | ||
529 | 41 | |||
530 | 42 | -d|--debug Print debug statements. This includes printing | ||
531 | 43 | the authentication token if using Dockerhub. | ||
532 | 44 | |||
533 | 45 | -h|--help Display this usage | ||
534 | 46 | EOF | ||
535 | 47 | } | ||
536 | 48 | |||
537 | 49 | validate_args () | ||
538 | 50 | { | ||
539 | 51 | local ret=0 | ||
540 | 52 | |||
541 | 53 | validate_var REGISTRY || ret=1 | ||
542 | 54 | validate_var NAMESPACE || ret=1 | ||
543 | 55 | validate_var IMAGE || ret=1 | ||
544 | 56 | |||
545 | 57 | validate_registry "$REGISTRY" || ret=1 | ||
546 | 58 | validate_namespace "$NAMESPACE" || ret=1 | ||
547 | 59 | |||
548 | 60 | if [ "$ret" -eq 1 ]; then | ||
549 | 61 | error "Failed to validate arguments." | ||
550 | 62 | exit 1 | ||
551 | 63 | fi | ||
552 | 64 | |||
553 | 65 | do_login | ||
554 | 66 | |||
555 | 67 | validate_image "$IMAGE" | ||
556 | 68 | } | ||
557 | 69 | |||
558 | 70 | print_tags_for_image_docker () | ||
559 | 71 | { | ||
560 | 72 | for tag in $(curl -s \ | ||
561 | 73 | -H "Authorization: JWT ${AUTH_TOKEN}" \ | ||
562 | 74 | "${DOCKERHUB_URL}/v2/repositories/${NAMESPACE}/${IMAGE}/tags/?page_size=10000" \ | ||
563 | 75 | | jq -r '.results|.[]|.name'); do | ||
564 | 76 | echo "${tag}" | ||
565 | 77 | done | ||
566 | 78 | } | ||
567 | 79 | |||
568 | 80 | print_tags_for_image_aws () | ||
569 | 81 | { | ||
570 | 82 | for tag in $(aws --region us-east-1 ecr-public describe-image-tags --repository-name "${IMAGE}" \ | ||
571 | 83 | | jq -r '.imageTagDetails[].imageTag'); do | ||
572 | 84 | echo "${tag}" | ||
573 | 85 | done | ||
574 | 86 | } | ||
575 | 87 | |||
576 | 88 | print_tags_for_image () | ||
577 | 89 | { | ||
578 | 90 | print_tags_for_image_"${REGISTRY}" | ||
579 | 91 | } | ||
580 | 92 | |||
581 | 93 | if [ $# -lt 2 ]; then | ||
582 | 94 | error "You need to provide at least the registry (-r), the namespace (-n) and the image (-i) arguments." | ||
583 | 95 | usage | ||
584 | 96 | exit 1 | ||
585 | 97 | fi | ||
586 | 98 | |||
587 | 99 | while [ -n "$1" ]; do | ||
588 | 100 | case "$1" in | ||
589 | 101 | "-a"|"--auth-token") | ||
590 | 102 | AUTH_TOKEN="$2" | ||
591 | 103 | shift 2 | ||
592 | 104 | ;; | ||
593 | 105 | |||
594 | 106 | "-r"|"--registry") | ||
595 | 107 | REGISTRY="$2" | ||
596 | 108 | shift 2 | ||
597 | 109 | ;; | ||
598 | 110 | |||
599 | 111 | "-n"|"--namespace") | ||
600 | 112 | NAMESPACE="$2" | ||
601 | 113 | shift 2 | ||
602 | 114 | ;; | ||
603 | 115 | |||
604 | 116 | "-i"|"--image") | ||
605 | 117 | IMAGE="$2" | ||
606 | 118 | shift 2 | ||
607 | 119 | ;; | ||
608 | 120 | |||
609 | 121 | "-d"|"--debug") | ||
610 | 122 | export DEBUG=1 | ||
611 | 123 | shift | ||
612 | 124 | ;; | ||
613 | 125 | |||
614 | 126 | "-h"|"--help") | ||
615 | 127 | usage | ||
616 | 128 | exit 0 | ||
617 | 129 | ;; | ||
618 | 130 | |||
619 | 131 | *) | ||
620 | 132 | error "Invalid argument '$1'." | ||
621 | 133 | usage | ||
622 | 134 | exit 1 | ||
623 | 135 | ;; | ||
624 | 136 | esac | ||
625 | 137 | done | ||
626 | 138 | |||
627 | 139 | validate_args | ||
628 | 140 | |||
629 | 141 | print_tags_for_image | ||
630 | diff --git a/multi-arch-tagger.sh b/multi-arch-tagger.sh | |||
631 | 0 | new file mode 100755 | 142 | new file mode 100755 |
632 | index 0000000..a493fa2 | |||
633 | --- /dev/null | |||
634 | +++ b/multi-arch-tagger.sh | |||
635 | @@ -0,0 +1,220 @@ | |||
636 | 1 | #!/bin/bash | ||
637 | 2 | |||
638 | 3 | set -e | ||
639 | 4 | |||
640 | 5 | progname=$(basename "${0}") | ||
641 | 6 | progdir=$(dirname "$(readlink -f "${0}")") | ||
642 | 7 | |||
643 | 8 | # shellcheck disable=SC1090 | ||
644 | 9 | . "${progdir}/helpers/validate-args.sh" | ||
645 | 10 | # shellcheck disable=SC1090 | ||
646 | 11 | . "${progdir}/helpers/log.sh" | ||
647 | 12 | # shellcheck disable=SC1090 | ||
648 | 13 | . "${progdir}/helpers/registry-login.sh" | ||
649 | 14 | |||
650 | 15 | # Tag all architectures of an image. | ||
651 | 16 | |||
652 | 17 | # Usage. | ||
653 | 18 | usage () | ||
654 | 19 | { | ||
655 | 20 | cat > /dev/stderr <<EOF | ||
656 | 21 | $progname -- Tag all architectures of an image. | ||
657 | 22 | |||
658 | 23 | Due to the way docker's API works, you can provide an authentication | ||
659 | 24 | token for the hub.docker.com service (if you don't, the script will | ||
660 | 25 | ask for you user and password), but you *must* provide your username | ||
661 | 26 | and password for the registry1.docker.io service. This is likely the | ||
662 | 27 | same username/password you use for docker. | ||
663 | 28 | |||
664 | 29 | Usage (when using Dockerhub): | ||
665 | 30 | |||
666 | 31 | $progname [-a <TOKEN>] -r docker -n <ubuntu|lts> -i <IMAGE> -s <SOURCE_TAG> -t <NEW_TAG> [-u <USERNAME>] [-p <PASSWORD>] [-d] | ||
667 | 32 | |||
668 | 33 | Usage (when using AWS): | ||
669 | 34 | |||
670 | 35 | $progname [-a <TOKEN>] -r aws -n <ubuntu|lts> -i <IMAGE> -s <SOURCE_TAG> -t <NEW_TAG> [-d] | ||
671 | 36 | |||
672 | 37 | Arguments: | ||
673 | 38 | |||
674 | 39 | -a TOKEN|--auth-token TOKEN Specify the authorization token that | ||
675 | 40 | should be used when connecting to the | ||
676 | 41 | registry's API. If not provided, the | ||
677 | 42 | script will ask for your user/password. | ||
678 | 43 | This is **NOT** the authentication token | ||
679 | 44 | for the registry (i.e., not for e.g. | ||
680 | 45 | registry1.docker.io, if you're using | ||
681 | 46 | docker). See the -u and -p options | ||
682 | 47 | below. | ||
683 | 48 | |||
684 | 49 | -r REGISTRY|--registry REGISTRY Registry to consult. | ||
685 | 50 | Possible values: docker, aws | ||
686 | 51 | |||
687 | 52 | -n NAMESPACE|--namespace NAMESPACE Namespace to consult. | ||
688 | 53 | Possible values: ubuntu, lts | ||
689 | 54 | |||
690 | 55 | -i IMAGE|--image IMAGE Name of an image from the repository (without | ||
691 | 56 | namespace), for example 'postgres' or 'redis'. | ||
692 | 57 | |||
693 | 58 | -s SOURCE_TAG|--source-tag SOURCE_TAG Use SOURCE_TAG as the source tag when | ||
694 | 59 | tagging the image. | ||
695 | 60 | |||
696 | 61 | -t NEW_TAG|--tag NEW_TAG Tag image using NEW_TAG. | ||
697 | 62 | |||
698 | 63 | -u USERNAME|--username USERNAME This is your Dockerhub registry username. Only | ||
699 | 64 | required when using Dockerhub as the registry. | ||
700 | 65 | This will be the same as your regular Dockerhub | ||
701 | 66 | username. | ||
702 | 67 | |||
703 | 68 | -p PASSWORD|--password PASSWORD This is your Dockerhub registry password. Only | ||
704 | 69 | required when using Dockerhub as the registry. | ||
705 | 70 | This will be the same as your Dockerhub username. | ||
706 | 71 | |||
707 | 72 | -d|--debug Print debug statements. This includes printing | ||
708 | 73 | the authentication token if using Dockerhub. | ||
709 | 74 | |||
710 | 75 | -h|--help Display this usage | ||
711 | 76 | EOF | ||
712 | 77 | } | ||
713 | 78 | |||
714 | 79 | do_tag_image () | ||
715 | 80 | { | ||
716 | 81 | info "Tagging ${NAMESPACE}/${IMAGE}:${SOURCE_TAG} as ${NAMESPACE}/${IMAGE}:${TAG} (on ${REGISTRY})" | ||
717 | 82 | |||
718 | 83 | local tmpfile | ||
719 | 84 | local ret | ||
720 | 85 | |||
721 | 86 | tmpfile=$(mktemp) | ||
722 | 87 | trap 'rm -f ${tmpfile}' 0 INT QUIT ABRT PIPE TERM | ||
723 | 88 | |||
724 | 89 | curl -s -H "Authorization: Bearer ${REGISTRY_AUTH_TOKEN}" \ | ||
725 | 90 | -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" \ | ||
726 | 91 | "${REGISTRY_URL}/v2/${NAMESPACE}/${IMAGE}/manifests/${SOURCE_TAG}" \ | ||
727 | 92 | > "$tmpfile" | ||
728 | 93 | |||
729 | 94 | # We use "|| true" here because we want the command to complete | ||
730 | 95 | # even if it fails. We check for the HTTP response code below. | ||
731 | 96 | ret=$(curl -X PUT -H "Authorization: Bearer ${REGISTRY_AUTH_TOKEN}" \ | ||
732 | 97 | -H "Content-Type: application/vnd.docker.distribution.manifest.list.v2+json" \ | ||
733 | 98 | "${REGISTRY_URL}/v2/${NAMESPACE}/${IMAGE}/manifests/${TAG}" \ | ||
734 | 99 | -d "@${tmpfile}" \ | ||
735 | 100 | -s -o /dev/null -w "%{http_code}" || true) | ||
736 | 101 | |||
737 | 102 | rm "$tmpfile" | ||
738 | 103 | |||
739 | 104 | if [ "$ret" -ne 201 ]; then | ||
740 | 105 | echo "ERROR: Image tagging failed with status code $ret" | ||
741 | 106 | exit 1 | ||
742 | 107 | fi | ||
743 | 108 | } | ||
744 | 109 | |||
745 | 110 | do_registry_login () | ||
746 | 111 | { | ||
747 | 112 | _login_"${REGISTRY}"_registry1 "$IMAGE" "$REGISTRY_USERNAME" "$REGISTRY_PASSWORD" | ||
748 | 113 | } | ||
749 | 114 | |||
750 | 115 | validate_args () | ||
751 | 116 | { | ||
752 | 117 | local ret=0 | ||
753 | 118 | |||
754 | 119 | validate_var REGISTRY || ret=1 | ||
755 | 120 | validate_var NAMESPACE || ret=1 | ||
756 | 121 | validate_var IMAGE || ret=1 | ||
757 | 122 | validate_var SOURCE_TAG || ret=1 | ||
758 | 123 | validate_var TAG || ret=1 | ||
759 | 124 | |||
760 | 125 | validate_registry "$REGISTRY" || ret=1 | ||
761 | 126 | validate_namespace "$NAMESPACE" || ret=1 | ||
762 | 127 | |||
763 | 128 | if [ "$ret" -eq 1 ]; then | ||
764 | 129 | error "Failed to validate arguments." | ||
765 | 130 | exit 1 | ||
766 | 131 | fi | ||
767 | 132 | |||
768 | 133 | if [ -n "$REGISTRY_USERNAME" ]; then | ||
769 | 134 | export USERNAME="$REGISTRY_USERNAME" | ||
770 | 135 | fi | ||
771 | 136 | if [ -n "$REGISTRY_PASSWORD" ]; then | ||
772 | 137 | export PASSWORD="$REGISTRY_PASSWORD" | ||
773 | 138 | fi | ||
774 | 139 | |||
775 | 140 | do_login | ||
776 | 141 | |||
777 | 142 | validate_image "$IMAGE" || exit 1 | ||
778 | 143 | |||
779 | 144 | if [ "$REGISTRY" = "docker" ]; then | ||
780 | 145 | validate_var REGISTRY_USERNAME || exit 1 | ||
781 | 146 | validate_var REGISTRY_PASSWORD || exit 1 | ||
782 | 147 | fi | ||
783 | 148 | |||
784 | 149 | do_registry_login | ||
785 | 150 | } | ||
786 | 151 | |||
787 | 152 | if [ $# -lt 2 ]; then | ||
788 | 153 | error "You need to provide arguments to the script." | ||
789 | 154 | usage | ||
790 | 155 | exit 1 | ||
791 | 156 | fi | ||
792 | 157 | |||
793 | 158 | while [ -n "$1" ]; do | ||
794 | 159 | case "$1" in | ||
795 | 160 | "-a"|"--auth-token") | ||
796 | 161 | export AUTH_TOKEN="$2" | ||
797 | 162 | shift 2 | ||
798 | 163 | ;; | ||
799 | 164 | |||
800 | 165 | "-r"|"--registry") | ||
801 | 166 | REGISTRY="$2" | ||
802 | 167 | shift 2 | ||
803 | 168 | ;; | ||
804 | 169 | |||
805 | 170 | "-n"|"--namespace") | ||
806 | 171 | NAMESPACE="$2" | ||
807 | 172 | shift 2 | ||
808 | 173 | ;; | ||
809 | 174 | |||
810 | 175 | "-i"|"--image") | ||
811 | 176 | IMAGE="$2" | ||
812 | 177 | shift 2 | ||
813 | 178 | ;; | ||
814 | 179 | |||
815 | 180 | "-s"|"--source-tag") | ||
816 | 181 | SOURCE_TAG="$2" | ||
817 | 182 | shift 2 | ||
818 | 183 | ;; | ||
819 | 184 | |||
820 | 185 | "-t"|"--tag") | ||
821 | 186 | TAG="$2" | ||
822 | 187 | shift 2 | ||
823 | 188 | ;; | ||
824 | 189 | |||
825 | 190 | "-u"|"--username") | ||
826 | 191 | REGISTRY_USERNAME="$2" | ||
827 | 192 | shift 2 | ||
828 | 193 | ;; | ||
829 | 194 | |||
830 | 195 | "-p"|"--password") | ||
831 | 196 | REGISTRY_PASSWORD="$2" | ||
832 | 197 | shift 2 | ||
833 | 198 | ;; | ||
834 | 199 | |||
835 | 200 | "-d"|"--debug") | ||
836 | 201 | export DEBUG=1 | ||
837 | 202 | shift | ||
838 | 203 | ;; | ||
839 | 204 | |||
840 | 205 | "-h"|"--help") | ||
841 | 206 | usage | ||
842 | 207 | exit 0 | ||
843 | 208 | ;; | ||
844 | 209 | |||
845 | 210 | *) | ||
846 | 211 | error "Invalid argument '$1'." | ||
847 | 212 | usage | ||
848 | 213 | exit 1 | ||
849 | 214 | ;; | ||
850 | 215 | esac | ||
851 | 216 | done | ||
852 | 217 | |||
853 | 218 | validate_args | ||
854 | 219 | |||
855 | 220 | do_tag_image | ||
856 | diff --git a/tag-images.sh b/tag-images.sh | |||
857 | 0 | new file mode 100755 | 221 | new file mode 100755 |
858 | index 0000000..874673a | |||
859 | --- /dev/null | |||
860 | +++ b/tag-images.sh | |||
861 | @@ -0,0 +1,335 @@ | |||
862 | 1 | #!/bin/bash | ||
863 | 2 | |||
864 | 3 | set -e | ||
865 | 4 | |||
866 | 5 | progname=$(basename "${0}") | ||
867 | 6 | progdir=$(dirname "$(readlink -f "${0}")") | ||
868 | 7 | |||
869 | 8 | # shellcheck disable=SC1090 | ||
870 | 9 | . "${progdir}/helpers/validate-args.sh" | ||
871 | 10 | # shellcheck disable=SC1090 | ||
872 | 11 | . "${progdir}/helpers/log.sh" | ||
873 | 12 | # shellcheck disable=SC1090 | ||
874 | 13 | . "${progdir}helpers/registry-login.sh" | ||
875 | 14 | |||
876 | 15 | # Tag images from a repository. | ||
877 | 16 | |||
878 | 17 | # The list of images to tag. | ||
879 | 18 | IMAGES=() | ||
880 | 19 | |||
881 | 20 | # The list of all available images. | ||
882 | 21 | ALL_IMAGES=() | ||
883 | 22 | |||
884 | 23 | # Default source tag is "edge". | ||
885 | 24 | SOURCE_TAG=edge | ||
886 | 25 | |||
887 | 26 | # Usage. | ||
888 | 27 | usage () | ||
889 | 28 | { | ||
890 | 29 | cat > /dev/stderr <<EOF | ||
891 | 30 | $progname -- Tag images from a repository (multi-architecture). | ||
892 | 31 | |||
893 | 32 | This script will tag images from a repository using a certain tag the | ||
894 | 33 | source tag. | ||
895 | 34 | |||
896 | 35 | ******************************* DISCLAIMER ************************************** | ||
897 | 36 | ** This script will only work with the 'latest' and the 'M.N-XX.YY_beta' tags! ** | ||
898 | 37 | ********************************************************************************* | ||
899 | 38 | |||
900 | 39 | This script is needed because of a few particularities in the tagging | ||
901 | 40 | policy for our OCI images. | ||
902 | 41 | |||
903 | 42 | The first is the fact that Launchpad will not tag anything using the | ||
904 | 43 | "latest" tag. When it finishes a build, the only tags it will | ||
905 | 44 | generate are the "edge" and the "M.N-XX.YY_edge" tags. This means | ||
906 | 45 | that we need to manually tag everything using "latest" after the | ||
907 | 46 | images have been uploaded to the registries. We also need to create | ||
908 | 47 | the "M.N-XX.YY_beta" tag, since that is not automatically created by | ||
909 | 48 | Launchpad either. | ||
910 | 49 | |||
911 | 50 | The other problem we have to consider is that, when tagging an image, | ||
912 | 51 | we must tag all of the architectures that were uploaded. If we were | ||
913 | 52 | to do that using "docker", we would need to run it on several | ||
914 | 53 | machines, one for each supported architectures, which is not always | ||
915 | 54 | possible/feasible. By using API calls directly, we are able to tag | ||
916 | 55 | all architectures using a single machine. | ||
917 | 56 | |||
918 | 57 | Usage: | ||
919 | 58 | |||
920 | 59 | $progname [-a <TOKEN>] -r <docker|aws> -n <ubuntu|lts> -s <SOURCE_TAG> [-d] [-- IMAGE_1 IMAGE_2 ... IMAGE_N] | ||
921 | 60 | |||
922 | 61 | Arguments: | ||
923 | 62 | |||
924 | 63 | -r REGISTRY|--registry REGISTRY Registry to consult. | ||
925 | 64 | Possible values: docker, aws | ||
926 | 65 | |||
927 | 66 | -n NAMESPACE|--namespace NAMESPACE Namespace to consult. | ||
928 | 67 | Possible values: ubuntu, lts | ||
929 | 68 | |||
930 | 69 | -s TAG|--source-tag SOURCE_TAG Use SOURCE_TAG as the source tag when tagging | ||
931 | 70 | the images. If not specified, the default | ||
932 | 71 | is 'edge'. | ||
933 | 72 | |||
934 | 73 | -f|--force Force tagging, even if the script would think | ||
935 | 74 | that it's not needed. This mostly applies for | ||
936 | 75 | the 'M.N-XX.YY_beta' tag, which is only created | ||
937 | 76 | if it doesn't exist. | ||
938 | 77 | |||
939 | 78 | -d|--debug Print debug statements. This includes printing | ||
940 | 79 | the authentication token if using Dockerhub. | ||
941 | 80 | |||
942 | 81 | -- IMAGE_1 IMAGE_2 ... IMAGE_N Images to tag. If not specified, | ||
943 | 82 | the script will obtain the list of | ||
944 | 83 | all images from REGISTRY/NAMESPACE | ||
945 | 84 | and tag them. | ||
946 | 85 | |||
947 | 86 | -h|--help Display this usage | ||
948 | 87 | EOF | ||
949 | 88 | } | ||
950 | 89 | |||
951 | 90 | # Tag the 'ubuntu' base image. | ||
952 | 91 | # | ||
953 | 92 | # We perform the tagging of '_stable', '_beta', '_candidate', 'XX.YY' | ||
954 | 93 | # and 'RELEASE_NAME'. | ||
955 | 94 | do_tag_base_ubuntu_image () | ||
956 | 95 | { | ||
957 | 96 | local image="ubuntu" | ||
958 | 97 | local -a TAG_SUFFIXES=( "stable" "beta" "candidate" ) | ||
959 | 98 | |||
960 | 99 | if [ "$REGISTRY" != "aws" ]; then | ||
961 | 100 | error "The 'ubuntu' base image is only present in the 'aws' registry." | ||
962 | 101 | exit 1 | ||
963 | 102 | fi | ||
964 | 103 | |||
965 | 104 | local -a IMAGE_TAGS EDGE_TAGS | ||
966 | 105 | |||
967 | 106 | readarray -t IMAGE_TAGS < <("${progdir}./list-tags-for-image.sh" \ | ||
968 | 107 | -a "$AUTH_TOKEN" \ | ||
969 | 108 | -r "$REGISTRY" \ | ||
970 | 109 | -n "$NAMESPACE" \ | ||
971 | 110 | -i "$image") | ||
972 | 111 | |||
973 | 112 | readarray -t EDGE_TAGS < <(printf '%s\n' "${IMAGE_TAGS[@]}" \ | ||
974 | 113 | | grep '_edge$') | ||
975 | 114 | |||
976 | 115 | for edgetag in "${EDGE_TAGS[@]}"; do | ||
977 | 116 | # We support retagging images tagged as DISTRONAME-XX.YY_edge. | ||
978 | 117 | local EDGE_TAG_PREFIX | ||
979 | 118 | EDGE_TAG_PREFIX=$(sed -ne 's@\([[:alpha:]]\+-[[:digit:]]\+\.[[:digit:]]\+\)_edge@\1@p' <<< "$edgetag") | ||
980 | 119 | if [ -z "$EDGE_TAG_PREFIX" ]; then | ||
981 | 120 | warn "Unable to obtain prefix for tag '$edgetag'." | ||
982 | 121 | continue | ||
983 | 122 | fi | ||
984 | 123 | |||
985 | 124 | local IMG_DISTRONAME | ||
986 | 125 | local IMG_RELEASEVER | ||
987 | 126 | |||
988 | 127 | IMG_DISTRONAME=$(cut -d'-' -f1 <<< "$EDGE_TAG_PREFIX") | ||
989 | 128 | IMG_RELEASEVER=$(cut -d'-' -f2 <<< "$EDGE_TAG_PREFIX") | ||
990 | 129 | for tagsuffix in "${TAG_SUFFIXES[@]}"; do | ||
991 | 130 | # Tag each suffix. | ||
992 | 131 | if [ -n "$FORCE" ] || ! grep -Fwq "${IMG_RELEASEVER}_${tagsuffix}" <<< "${IMAGE_TAGS[@]}"; then | ||
993 | 132 | info "Invoking multi-arch tagger for ${NAMESPACE}/${image}:${IMG_RELEASEVER}_${tagsuffix} (source tag: ${EDGE_TAG_PREFIX}_edge) (on ${REGISTRY})" | ||
994 | 133 | "${progdir}./multi-arch-tagger.sh" \ | ||
995 | 134 | -a "$AUTH_TOKEN" \ | ||
996 | 135 | -r "$REGISTRY" \ | ||
997 | 136 | -n "$NAMESPACE" \ | ||
998 | 137 | -i "$image" \ | ||
999 | 138 | -s "${EDGE_TAG_PREFIX}_edge" \ | ||
1000 | 139 | -t "${IMG_RELEASEVER}_${tagsuffix}" \ | ||
1001 | 140 | -u "$USERNAME" \ | ||
1002 | 141 | -p "$PASSWORD" | ||
1003 | 142 | fi | ||
1004 | 143 | done | ||
1005 | 144 | |||
1006 | 145 | # Tag the DISTRONAME. | ||
1007 | 146 | if [ -n "$FORCE" ] || ! grep -Fwq "${IMG_DISTRONAME}" <<< "${IMAGE_TAGS[@]}"; then | ||
1008 | 147 | info "Invoking multi-arch tagger for ${NAMESPACE}/${image}:${IMG_DISTRONAME} (source tag: ${EDGE_TAG_PREFIX}_edge) (on ${REGISTRY})" | ||
1009 | 148 | "${progdir}./multi-arch-tagger.sh" \ | ||
1010 | 149 | -a "$AUTH_TOKEN" \ | ||
1011 | 150 | -r "$REGISTRY" \ | ||
1012 | 151 | -n "$NAMESPACE" \ | ||
1013 | 152 | -i "$image" \ | ||
1014 | 153 | -s "${EDGE_TAG_PREFIX}_edge" \ | ||
1015 | 154 | -t "${IMG_DISTRONAME}" \ | ||
1016 | 155 | -u "$USERNAME" \ | ||
1017 | 156 | -p "$PASSWORD" | ||
1018 | 157 | fi | ||
1019 | 158 | |||
1020 | 159 | # Tag the RELEASEVER | ||
1021 | 160 | if [ -n "$FORCE" ] || ! grep -Fwq "${IMG_RELEASEVER}" <<< "${IMAGE_TAGS[@]}"; then | ||
1022 | 161 | info "Invoking multi-arch tagger for ${NAMESPACE}/${image}:${IMG_RELEASEVER} (source tag: ${EDGE_TAG_PREFIX}_edge) (on ${REGISTRY})" | ||
1023 | 162 | "${progdir}./multi-arch-tagger.sh" \ | ||
1024 | 163 | -a "$AUTH_TOKEN" \ | ||
1025 | 164 | -r "$REGISTRY" \ | ||
1026 | 165 | -n "$NAMESPACE" \ | ||
1027 | 166 | -i "$image" \ | ||
1028 | 167 | -s "${EDGE_TAG_PREFIX}_edge" \ | ||
1029 | 168 | -t "${IMG_RELEASEVER}" \ | ||
1030 | 169 | -u "$USERNAME" \ | ||
1031 | 170 | -p "$PASSWORD" | ||
1032 | 171 | fi | ||
1033 | 172 | done | ||
1034 | 173 | } | ||
1035 | 174 | |||
1036 | 175 | do_tag_images () | ||
1037 | 176 | { | ||
1038 | 177 | # Tag images using the 'latest' tag. | ||
1039 | 178 | for image in "${IMAGES[@]}"; do | ||
1040 | 179 | info "Invoking multi-arch tagger for ${NAMESPACE}/${image}:latest (source tag: ${SOURCE_TAG}) (on ${REGISTRY})" | ||
1041 | 180 | "${progdir}./multi-arch-tagger.sh" \ | ||
1042 | 181 | -a "$AUTH_TOKEN" \ | ||
1043 | 182 | -r "$REGISTRY" \ | ||
1044 | 183 | -n "$NAMESPACE" \ | ||
1045 | 184 | -i "$image" \ | ||
1046 | 185 | -s "$SOURCE_TAG" \ | ||
1047 | 186 | -t "latest" \ | ||
1048 | 187 | -u "$USERNAME" \ | ||
1049 | 188 | -p "$PASSWORD" | ||
1050 | 189 | done | ||
1051 | 190 | |||
1052 | 191 | # Now comes the fun part. We have to determine whether the image | ||
1053 | 192 | # needs a 'M.N-XX.YY_beta' tag, and which number should N, M, XX | ||
1054 | 193 | # and YY be. | ||
1055 | 194 | # | ||
1056 | 195 | # Basically, if there is a tag named 'M.N-XX.YY_edge' without a | ||
1057 | 196 | # corresponding '_beta' tag, then we assume that the image will | ||
1058 | 197 | # need to receive the '_beta' tag. This is the usual case | ||
1059 | 198 | # (currently) with images built using Launchpad. | ||
1060 | 199 | for image in "${IMAGES[@]}"; do | ||
1061 | 200 | if [ "$image" = "ubuntu" ]; then | ||
1062 | 201 | # The ubuntu image is special. | ||
1063 | 202 | do_tag_base_ubuntu_image | ||
1064 | 203 | continue | ||
1065 | 204 | fi | ||
1066 | 205 | |||
1067 | 206 | local -a IMAGE_TAGS EDGE_TAGS | ||
1068 | 207 | |||
1069 | 208 | readarray -t IMAGE_TAGS < <("${progdir}./list-tags-for-image.sh" \ | ||
1070 | 209 | -a "$AUTH_TOKEN" \ | ||
1071 | 210 | -r "$REGISTRY" \ | ||
1072 | 211 | -n "$NAMESPACE" \ | ||
1073 | 212 | -i "$image") | ||
1074 | 213 | |||
1075 | 214 | readarray -t EDGE_TAGS < <(printf '%s\n' "${IMAGE_TAGS[@]}" \ | ||
1076 | 215 | | grep '_edge$' | sort -r) | ||
1077 | 216 | |||
1078 | 217 | for edgetag in "${EDGE_TAGS[@]}"; do | ||
1079 | 218 | local EDGE_TAG_PREFIX | ||
1080 | 219 | EDGE_TAG_PREFIX=$(sed -ne 's@\([[:digit:]]\+\(\.[[:digit:]]\+\)\?-[[:digit:]]\+\.[[:digit:]]\+\)_edge@\1@p' <<< "$edgetag") | ||
1081 | 220 | if [ -z "$EDGE_TAG_PREFIX" ]; then | ||
1082 | 221 | warn "Unable to obtain prefix for tag '$edgetag'." | ||
1083 | 222 | continue | ||
1084 | 223 | fi | ||
1085 | 224 | if [ -n "$FORCE" ] || ! grep -Fwq "${EDGE_TAG_PREFIX}_beta" <<< "${IMAGE_TAGS[@]}"; then | ||
1086 | 225 | # If we're here, it means that there isn't a | ||
1087 | 226 | # corresponding 'M.N-XX.YY_beta' tag associated with | ||
1088 | 227 | # the '_edge' tag, so we need to create it. | ||
1089 | 228 | info "Invoking multi-arch tagger for ${NAMESPACE}/${image}:${EDGE_TAG_PREFIX}_beta (source tag: ${EDGE_TAG_PREFIX}_edge) (on ${REGISTRY})" | ||
1090 | 229 | "${progdir}./multi-arch-tagger.sh" \ | ||
1091 | 230 | -a "$AUTH_TOKEN" \ | ||
1092 | 231 | -r "$REGISTRY" \ | ||
1093 | 232 | -n "$NAMESPACE" \ | ||
1094 | 233 | -i "$image" \ | ||
1095 | 234 | -s "${EDGE_TAG_PREFIX}_edge" \ | ||
1096 | 235 | -t "${EDGE_TAG_PREFIX}_beta" \ | ||
1097 | 236 | -u "$USERNAME" \ | ||
1098 | 237 | -p "$PASSWORD" | ||
1099 | 238 | fi | ||
1100 | 239 | done | ||
1101 | 240 | done | ||
1102 | 241 | } | ||
1103 | 242 | |||
1104 | 243 | validate_args () | ||
1105 | 244 | { | ||
1106 | 245 | local ret=0 | ||
1107 | 246 | |||
1108 | 247 | validate_var REGISTRY || ret=1 | ||
1109 | 248 | validate_var NAMESPACE || ret=1 | ||
1110 | 249 | validate_var SOURCE_TAG || ret=1 | ||
1111 | 250 | |||
1112 | 251 | validate_registry "$REGISTRY" || ret=1 | ||
1113 | 252 | validate_namespace "$NAMESPACE" || ret=1 | ||
1114 | 253 | |||
1115 | 254 | if [ "$ret" -eq 1 ]; then | ||
1116 | 255 | error "Failed to validate arguments." | ||
1117 | 256 | exit 1 | ||
1118 | 257 | fi | ||
1119 | 258 | |||
1120 | 259 | do_login | ||
1121 | 260 | |||
1122 | 261 | # Fill the ALL_IMAGES array. | ||
1123 | 262 | readarray -t ALL_IMAGES < <("${progdir}./list-all-images.sh" \ | ||
1124 | 263 | -r "$REGISTRY" \ | ||
1125 | 264 | -n "$NAMESPACE" \ | ||
1126 | 265 | -a "$AUTH_TOKEN") | ||
1127 | 266 | |||
1128 | 267 | if [ "${#IMAGES[@]}" -eq 0 ]; then | ||
1129 | 268 | # The user hasn't provided any image names, so just use all of | ||
1130 | 269 | # them. | ||
1131 | 270 | IMAGES=( "${ALL_IMAGES[@]}" ) | ||
1132 | 271 | else | ||
1133 | 272 | # Validate the image names the user has provided. | ||
1134 | 273 | for img in "${IMAGES[@]}"; do | ||
1135 | 274 | validate_image "$img" || exit 1 | ||
1136 | 275 | done | ||
1137 | 276 | fi | ||
1138 | 277 | } | ||
1139 | 278 | |||
1140 | 279 | if [ $# -lt 2 ]; then | ||
1141 | 280 | error "You need to provide at least the registry (-r), the namespace (-n), and the source tag (-s)." | ||
1142 | 281 | usage | ||
1143 | 282 | exit 1 | ||
1144 | 283 | fi | ||
1145 | 284 | |||
1146 | 285 | while [ -n "$1" ]; do | ||
1147 | 286 | case "$1" in | ||
1148 | 287 | "-r"|"--registry") | ||
1149 | 288 | REGISTRY="$2" | ||
1150 | 289 | shift 2 | ||
1151 | 290 | ;; | ||
1152 | 291 | |||
1153 | 292 | "-n"|"--namespace") | ||
1154 | 293 | NAMESPACE="$2" | ||
1155 | 294 | shift 2 | ||
1156 | 295 | ;; | ||
1157 | 296 | |||
1158 | 297 | "-s"|"--source-tag") | ||
1159 | 298 | SOURCE_TAG="$2" | ||
1160 | 299 | shift 2 | ||
1161 | 300 | ;; | ||
1162 | 301 | |||
1163 | 302 | "-f"|"--force") | ||
1164 | 303 | FORCE=1 | ||
1165 | 304 | shift | ||
1166 | 305 | ;; | ||
1167 | 306 | |||
1168 | 307 | "-d"|"--debug") | ||
1169 | 308 | export DEBUG=1 | ||
1170 | 309 | shift | ||
1171 | 310 | ;; | ||
1172 | 311 | |||
1173 | 312 | "-h"|"--help") | ||
1174 | 313 | usage | ||
1175 | 314 | exit 0 | ||
1176 | 315 | ;; | ||
1177 | 316 | |||
1178 | 317 | "--") | ||
1179 | 318 | shift | ||
1180 | 319 | while [ -n "$1" ]; do | ||
1181 | 320 | IMAGES+=( "$1" ) | ||
1182 | 321 | shift | ||
1183 | 322 | done | ||
1184 | 323 | ;; | ||
1185 | 324 | |||
1186 | 325 | *) | ||
1187 | 326 | error "Invalid argument '$1'." | ||
1188 | 327 | usage | ||
1189 | 328 | exit 1 | ||
1190 | 329 | ;; | ||
1191 | 330 | esac | ||
1192 | 331 | done | ||
1193 | 332 | |||
1194 | 333 | validate_args | ||
1195 | 334 | |||
1196 | 335 | do_tag_images |
Initial pass of mostly just stylistic comments. I also looked for typos or coding errors but didn't find any! So nothing that actually "needs" fixed, but more just suggestions.
I haven't tried actually running the code and studying its execution, but will give that a shot for the next pass, but that'll have be for tomorrow.
I like that you broke out the log.sh to be a sourced file. I had already been thinking of breaking them out so I could use them, so that's cool.