Merge ~kajiya/+git/google-osconfig-agent:update-master-to-20240524.03 into ~ubuntu-core-dev/+git/google-osconfig-agent:ubuntu/master
- Git
- lp:~kajiya/+git/google-osconfig-agent
- update-master-to-20240524.03
- Merge into ubuntu/master
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | ffe4765a0f0679d706a0535bb2142b87cf180151 | ||||
Proposed branch: | ~kajiya/+git/google-osconfig-agent:update-master-to-20240524.03 | ||||
Merge into: | ~ubuntu-core-dev/+git/google-osconfig-agent:ubuntu/master | ||||
Diff against target: |
2679 lines (+1679/-190) 43 files modified
.github/dependabot.yml (+10/-0) .github/workflows/codeql.yml (+92/-0) OWNERS (+1/-0) agentconfig/agentconfig.go (+28/-6) agentendpoint/inventory.go (+4/-1) config/exec_resource.go (+1/-1) debian/changelog (+7/-0) debian/patches/0002-Edit-TestAptRepositories-for-signed-repos.patch (+29/-0) debian/patches/series (+1/-0) e2e_tests/test_suites/guestpolicies/guest_policies_utils.go (+9/-4) e2e_tests/test_suites/inventory/test_setup.go (+24/-8) e2e_tests/test_suites/inventoryreporting/test_setup.go (+47/-33) e2e_tests/test_suites/ospolicies/ospolicies_test_data.go (+6/-6) e2e_tests/test_suites/ospolicies/ospolicies_utils.go (+6/-5) e2e_tests/test_suites/patch/test_setup.go (+40/-22) e2e_tests/utils/utils.go (+100/-56) examples/OSPolicyAssignments/console/CIS/cis-exclude-check-once-a-day.yaml (+3/-4) examples/OSPolicyAssignments/console/CIS/cis-level1-once-an-hour-policy.yaml (+3/-4) examples/OSPolicyAssignments/console/CIS/cis-level2-once-a-day-policy.yaml (+3/-4) go.mod (+8/-9) go.sum (+16/-18) main_windows.go (+2/-1) packagebuild/README.md (+75/-0) packagebuild/common.sh (+84/-0) packagebuild/daisy_startupscript_deb.sh (+108/-0) packagebuild/daisy_startupscript_goo.sh (+61/-0) packagebuild/daisy_startupscript_rpm.sh (+147/-0) packagebuild/test_build.sh (+69/-0) packagebuild/workflows/build_deb10.wf.json (+42/-0) packagebuild/workflows/build_deb11.wf.json (+42/-0) packagebuild/workflows/build_deb11_arm64.wf.json (+44/-0) packagebuild/workflows/build_deb12.wf.json (+42/-0) packagebuild/workflows/build_deb12_arm64.wf.json (+44/-0) packagebuild/workflows/build_el6.wf.json (+42/-0) packagebuild/workflows/build_el7.wf.json (+42/-0) packagebuild/workflows/build_el8.wf.json (+42/-0) packagebuild/workflows/build_el8_arm64.wf.json (+44/-0) packagebuild/workflows/build_el9.wf.json (+42/-0) packagebuild/workflows/build_el9_arm64.wf.json (+44/-0) packagebuild/workflows/build_goo.wf.json (+40/-0) packagebuild/workflows/build_package.wf.json (+101/-0) policies/apt.go (+56/-8) policies/apt_test.go (+28/-0) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Utkarsh Gupta | Approve | ||
Review via email:
|
Commit message
New upstream version 20240524.03
Description of the change
Related MPs...
upstream: https:/
pristine-tar: https:/
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Utkarsh Gupta (utkarsh) wrote : | # |
Thank you Chlo. I just noticed a wrong versioning there in the changelog. :(
Whilst at it - can you also add the same description in the changelog for posterity? Many thanks!
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Utkarsh Gupta (utkarsh) wrote : | # |
Ok, I can fix this on the go -
```
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,7 +1,10 @@
-google-
+google-
* New upstream version for upstream tag 20240524.03. (LP: #2073161)
- * d/patches - Edit TestAptRepositories to avail of GPG signing
+ * d/p/0002-
+ fix the failing and the blocking `TestAptReposit
+ account for the repo files being signed. This patch adds the expected
+ snippet "[signed-
-- Chloé 'kajiya' Smith <email address hidden> Mon, 15 Jul 2024 21:28:39 +0100
```
Uploading with this fix. :)
Preview Diff
1 | diff --git a/.github/dependabot.yml b/.github/dependabot.yml |
2 | new file mode 100644 |
3 | index 0000000..07449d1 |
4 | --- /dev/null |
5 | +++ b/.github/dependabot.yml |
6 | @@ -0,0 +1,10 @@ |
7 | +version: 2 |
8 | +updates: |
9 | + - package-ecosystem: "gomod" |
10 | + directory: "/" |
11 | + labels: |
12 | + - "dependencies" |
13 | + schedule: |
14 | + interval: "daily" |
15 | + allow: |
16 | + - dependency-type: "all" |
17 | diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml |
18 | new file mode 100644 |
19 | index 0000000..99c27d7 |
20 | --- /dev/null |
21 | +++ b/.github/workflows/codeql.yml |
22 | @@ -0,0 +1,92 @@ |
23 | +# For most projects, this workflow file will not need changing; you simply need |
24 | +# to commit it to your repository. |
25 | +# |
26 | +# You may wish to alter this file to override the set of languages analyzed, |
27 | +# or to provide custom queries or build logic. |
28 | +# |
29 | +# ******** NOTE ******** |
30 | +# We have attempted to detect the languages in your repository. Please check |
31 | +# the `language` matrix defined below to confirm you have the correct set of |
32 | +# supported CodeQL languages. |
33 | +# |
34 | +name: "CodeQL" |
35 | + |
36 | +on: |
37 | + push: |
38 | + branches: [ "master" ] |
39 | + pull_request: |
40 | + branches: [ "master" ] |
41 | + schedule: |
42 | + - cron: '45 4 * * 1' |
43 | + |
44 | +jobs: |
45 | + analyze: |
46 | + name: Analyze (${{ matrix.language }}) |
47 | + # Runner size impacts CodeQL analysis time. To learn more, please see: |
48 | + # - https://gh.io/recommended-hardware-resources-for-running-codeql |
49 | + # - https://gh.io/supported-runners-and-hardware-resources |
50 | + # - https://gh.io/using-larger-runners (GitHub.com only) |
51 | + # Consider using larger runners or machines with greater resources for possible analysis time improvements. |
52 | + runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }} |
53 | + timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }} |
54 | + permissions: |
55 | + # required for all workflows |
56 | + security-events: write |
57 | + |
58 | + # required to fetch internal or private CodeQL packs |
59 | + packages: read |
60 | + |
61 | + # only required for workflows in private repositories |
62 | + actions: read |
63 | + contents: read |
64 | + |
65 | + strategy: |
66 | + fail-fast: false |
67 | + matrix: |
68 | + include: |
69 | + - language: go |
70 | + build-mode: autobuild |
71 | + # CodeQL supports the following values keywords for 'language': 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift' |
72 | + # Use `c-cpp` to analyze code written in C, C++ or both |
73 | + # Use 'java-kotlin' to analyze code written in Java, Kotlin or both |
74 | + # Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both |
75 | + # To learn more about changing the languages that are analyzed or customizing the build mode for your analysis, |
76 | + # see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning. |
77 | + # If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how |
78 | + # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages |
79 | + steps: |
80 | + - name: Checkout repository |
81 | + uses: actions/checkout@v4 |
82 | + |
83 | + # Initializes the CodeQL tools for scanning. |
84 | + - name: Initialize CodeQL |
85 | + uses: github/codeql-action/init@v3 |
86 | + with: |
87 | + languages: ${{ matrix.language }} |
88 | + build-mode: ${{ matrix.build-mode }} |
89 | + # If you wish to specify custom queries, you can do so here or in a config file. |
90 | + # By default, queries listed here will override any specified in a config file. |
91 | + # Prefix the list here with "+" to use these queries and those in the config file. |
92 | + |
93 | + # For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs |
94 | + # queries: security-extended,security-and-quality |
95 | + |
96 | + # If the analyze step fails for one of the languages you are analyzing with |
97 | + # "We were unable to automatically build your code", modify the matrix above |
98 | + # to set the build mode to "manual" for that language. Then modify this step |
99 | + # to build your code. |
100 | + # ℹ️ Command-line programs to run using the OS shell. |
101 | + # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun |
102 | + - if: matrix.build-mode == 'manual' |
103 | + run: | |
104 | + echo 'If you are using a "manual" build mode for one or more of the' \ |
105 | + 'languages you are analyzing, replace this with the commands to build' \ |
106 | + 'your code, for example:' |
107 | + echo ' make bootstrap' |
108 | + echo ' make release' |
109 | + exit 1 |
110 | + |
111 | + - name: Perform CodeQL Analysis |
112 | + uses: github/codeql-action/analyze@v3 |
113 | + with: |
114 | + category: "/language:${{matrix.language}}" |
115 | diff --git a/OWNERS b/OWNERS |
116 | index 20cdb27..4b8646c 100644 |
117 | --- a/OWNERS |
118 | +++ b/OWNERS |
119 | @@ -10,3 +10,4 @@ approvers: |
120 | - ekremenetskii |
121 | - MahmoudOuka |
122 | - savijatv |
123 | + - MarcMarc |
124 | diff --git a/agentconfig/agentconfig.go b/agentconfig/agentconfig.go |
125 | index 05da6fc..7e1fa21 100644 |
126 | --- a/agentconfig/agentconfig.go |
127 | +++ b/agentconfig/agentconfig.go |
128 | @@ -68,14 +68,12 @@ const ( |
129 | debugEnabledDefault = false |
130 | |
131 | oldConfigDirLinux = "/etc/osconfig" |
132 | - cacheDirWindows = `C:\Program Files\Google\OSConfig` |
133 | cacheDirLinux = "/var/lib/google_osconfig_agent" |
134 | + windowsCacheDir = `Google\OSConfig` |
135 | |
136 | - taskStateFileWindows = cacheDirWindows + `\osconfig_task.state` |
137 | taskStateFileLinux = cacheDirLinux + "/osconfig_task.state" |
138 | oldTaskStateFileLinux = oldConfigDirLinux + "/osconfig_task.state" |
139 | |
140 | - restartFileWindows = cacheDirWindows + `\osconfig_agent_restart_required` |
141 | restartFileLinux = cacheDirLinux + "/osconfig_agent_restart_required" |
142 | oldRestartFileLinux = oldConfigDirLinux + "/osconfig_agent_restart_required" |
143 | |
144 | @@ -130,6 +128,7 @@ type config struct { |
145 | taskNotificationEnabled bool |
146 | guestPoliciesEnabled bool |
147 | osInventoryEnabled bool |
148 | + guestAttributesEnabled bool |
149 | } |
150 | |
151 | func (c *config) parseFeatures(features string, enabled bool) { |
152 | @@ -216,6 +215,7 @@ type attributesJSON struct { |
153 | OSConfigEndpoint string `json:"osconfig-endpoint"` |
154 | OSConfigEnabled string `json:"enable-osconfig"` |
155 | DisabledFeatures string `json:"osconfig-disabled-features"` |
156 | + EnableGuestAttributes string `json:"enable-guest-attributes"` |
157 | } |
158 | |
159 | func createConfigFromMetadata(md metadataJSON) *config { |
160 | @@ -334,6 +334,13 @@ func createConfigFromMetadata(md metadataJSON) *config { |
161 | c.debugEnabled = false |
162 | } |
163 | |
164 | + if md.Project.Attributes.EnableGuestAttributes != "" { |
165 | + c.guestAttributesEnabled = parseBool(md.Project.Attributes.EnableGuestAttributes) |
166 | + } |
167 | + if md.Instance.Attributes.EnableGuestAttributes != "" { |
168 | + c.guestAttributesEnabled = parseBool(md.Instance.Attributes.EnableGuestAttributes) |
169 | + } |
170 | + |
171 | // Flags take precedence over metadata. |
172 | if *debug { |
173 | c.debugEnabled = true |
174 | @@ -412,6 +419,15 @@ func getMetadata(suffix string) ([]byte, string, error) { |
175 | return all, resp.Header.Get("Etag"), nil |
176 | } |
177 | |
178 | +// GetCacheDirWindows returns the folder for the temp files location on Windows. |
179 | +func GetCacheDirWindows() string { |
180 | + cacheDir, dirErr := os.UserCacheDir() |
181 | + if dirErr != nil { |
182 | + cacheDir = os.TempDir() |
183 | + } |
184 | + return filepath.Join(cacheDir, windowsCacheDir) |
185 | +} |
186 | + |
187 | // WatchConfig looks for changes in metadata keys. Upon receiving successful response, |
188 | // it create a new agent config. |
189 | func WatchConfig(ctx context.Context) error { |
190 | @@ -627,6 +643,11 @@ func ID() string { |
191 | return getAgentConfig().instanceID |
192 | } |
193 | |
194 | +// GuestAttributesEnabled is a boolean flag that signal that guest attributes feature is enabled. |
195 | +func GuestAttributesEnabled() bool { |
196 | + return getAgentConfig().guestAttributesEnabled |
197 | +} |
198 | + |
199 | type idToken struct { |
200 | exp *time.Time |
201 | raw string |
202 | @@ -686,7 +707,7 @@ func Capabilities() []string { |
203 | // TaskStateFile is the location of the task state file. |
204 | func TaskStateFile() string { |
205 | if runtime.GOOS == "windows" { |
206 | - return taskStateFileWindows |
207 | + return filepath.Join(GetCacheDirWindows(), "osconfig_task.state") |
208 | } |
209 | |
210 | return taskStateFileLinux |
211 | @@ -700,7 +721,8 @@ func OldTaskStateFile() string { |
212 | // RestartFile is the location of the restart required file. |
213 | func RestartFile() string { |
214 | if runtime.GOOS == "windows" { |
215 | - return restartFileWindows |
216 | + return filepath.Join( |
217 | + GetCacheDirWindows(), "osconfig_agent_restart_required") |
218 | } |
219 | |
220 | return restartFileLinux |
221 | @@ -714,7 +736,7 @@ func OldRestartFile() string { |
222 | // CacheDir is the location of the cache directory. |
223 | func CacheDir() string { |
224 | if runtime.GOOS == "windows" { |
225 | - return cacheDirWindows |
226 | + return GetCacheDirWindows() |
227 | } |
228 | |
229 | return cacheDirLinux |
230 | diff --git a/agentendpoint/inventory.go b/agentendpoint/inventory.go |
231 | index 9239fbc..28ae6b3 100644 |
232 | --- a/agentendpoint/inventory.go |
233 | +++ b/agentendpoint/inventory.go |
234 | @@ -26,9 +26,12 @@ const ( |
235 | // ReportInventory writes inventory to guest attributes and reports it to agent endpoint. |
236 | func (c *Client) ReportInventory(ctx context.Context) { |
237 | state := inventory.Get(ctx) |
238 | - if !agentconfig.DisableInventoryWrite() { |
239 | + |
240 | + if agentconfig.GuestAttributesEnabled() && !agentconfig.DisableInventoryWrite() { |
241 | + clog.Infof(ctx, "Writing inventory to guest attributes") |
242 | write(ctx, state, inventoryURL) |
243 | } |
244 | + |
245 | c.report(ctx, state) |
246 | } |
247 | |
248 | diff --git a/config/exec_resource.go b/config/exec_resource.go |
249 | index 18eaac9..a51416d 100644 |
250 | --- a/config/exec_resource.go |
251 | +++ b/config/exec_resource.go |
252 | @@ -31,7 +31,7 @@ import ( |
253 | agentendpointpb "google.golang.org/genproto/googleapis/cloud/osconfig/agentendpoint/v1" |
254 | ) |
255 | |
256 | -const maxExecOutputSize = 100 * 1024 |
257 | +const maxExecOutputSize = 500 * 1024 |
258 | |
259 | var runner = util.CommandRunner(&util.DefaultRunner{}) |
260 | |
261 | diff --git a/debian/changelog b/debian/changelog |
262 | index 1aae5cb..2a86e29 100644 |
263 | --- a/debian/changelog |
264 | +++ b/debian/changelog |
265 | @@ -1,3 +1,10 @@ |
266 | +google-osconfig-agent (20240320.00-0ubuntu2) oracular; urgency=medium |
267 | + |
268 | + * New upstream version for upstream tag 20240524.03. (LP: #2073161) |
269 | + * d/patches - Edit TestAptRepositories to avail of GPG signing |
270 | + |
271 | + -- Chloé 'kajiya' Smith <chloe.smith@canonical.com> Mon, 15 Jul 2024 21:28:39 +0100 |
272 | + |
273 | google-osconfig-agent (20240320.00-0ubuntu1) oracular; urgency=medium |
274 | |
275 | [ Chloé 'kajiya' Smith ] |
276 | diff --git a/debian/patches/0002-Edit-TestAptRepositories-for-signed-repos.patch b/debian/patches/0002-Edit-TestAptRepositories-for-signed-repos.patch |
277 | new file mode 100644 |
278 | index 0000000..f0f7def |
279 | --- /dev/null |
280 | +++ b/debian/patches/0002-Edit-TestAptRepositories-for-signed-repos.patch |
281 | @@ -0,0 +1,29 @@ |
282 | +Subject: Edit TestAptRepositories to avail of "[signed-by=<managed-GPG-file>]" |
283 | +Description: The test `TestAptRepositories` was not taking into account that |
284 | +the managed repo files are signed by a (managed) .gpg file. This patch |
285 | +adds that snippet into the expected output so the tests pass and the |
286 | +package can build. |
287 | +Author: Chloé 'kajiya' Smith <chloe.smith@canonical.com> |
288 | +Forwarded: not-needed |
289 | +Last-Update: 2024-07-15 |
290 | +--- |
291 | +--- a/policies/apt_test.go |
292 | ++++ b/policies/apt_test.go |
293 | +@@ -58,7 +58,7 @@ |
294 | + []*agentendpointpb.AptRepository{ |
295 | + {Uri: "http://repo1-url/", Distribution: "distribution", Components: []string{"component1"}}, |
296 | + }, |
297 | +- "# Repo file managed by Google OSConfig agent\n\ndeb http://repo1-url/ distribution component1\n", |
298 | ++ "# Repo file managed by Google OSConfig agent\n\ndeb [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo1-url/ distribution component1\n", |
299 | + }, |
300 | + { |
301 | + "2 repos", |
302 | +@@ -66,7 +66,7 @@ |
303 | + {Uri: "http://repo1-url/", Distribution: "distribution", Components: []string{"component1"}, ArchiveType: agentendpointpb.AptRepository_DEB_SRC}, |
304 | + {Uri: "http://repo2-url/", Distribution: "distribution", Components: []string{"component1", "component2"}, ArchiveType: agentendpointpb.AptRepository_DEB}, |
305 | + }, |
306 | +- "# Repo file managed by Google OSConfig agent\n\ndeb-src http://repo1-url/ distribution component1\n\ndeb http://repo2-url/ distribution component1 component2\n", |
307 | ++ "# Repo file managed by Google OSConfig agent\n\ndeb-src [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo1-url/ distribution component1\n\ndeb [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo2-url/ distribution component1 component2\n", |
308 | + }, |
309 | + } |
310 | + |
311 | diff --git a/debian/patches/series b/debian/patches/series |
312 | index 8ec831e..e5f81ee 100644 |
313 | --- a/debian/patches/series |
314 | +++ b/debian/patches/series |
315 | @@ -1 +1,2 @@ |
316 | 0001-Disable-TestGetAptGPGKey-for-LP-build-environment.patch |
317 | +0002-Edit-TestAptRepositories-for-signed-repos.patch |
318 | diff --git a/e2e_tests/test_suites/guestpolicies/guest_policies_utils.go b/e2e_tests/test_suites/guestpolicies/guest_policies_utils.go |
319 | index cc5eb98..0fcd991 100644 |
320 | --- a/e2e_tests/test_suites/guestpolicies/guest_policies_utils.go |
321 | +++ b/e2e_tests/test_suites/guestpolicies/guest_policies_utils.go |
322 | @@ -67,7 +67,7 @@ while true; do |
323 | sleep 5 |
324 | done` |
325 | |
326 | - ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(), waitForRestartLinux, packageName, packageInstalled, packageNotInstalled) |
327 | + ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(image), waitForRestartLinux, packageName, packageInstalled, packageNotInstalled) |
328 | key = "startup-script" |
329 | |
330 | case "yum": |
331 | @@ -150,6 +150,11 @@ func getUpdateStartupScript(image, pkgManager string) *computeApi.MetadataItems |
332 | case "apt": |
333 | ss = ` |
334 | echo 'Adding test repo' |
335 | + |
336 | +# install gnupg2 if not exist |
337 | +apt-get update |
338 | +apt-get install -y gnupg2 |
339 | + |
340 | echo 'deb http://packages.cloud.google.com/apt osconfig-agent-test-repository main' >> /etc/apt/sources.list |
341 | curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - |
342 | while fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1; do |
343 | @@ -171,7 +176,7 @@ while true; do |
344 | sleep 5; |
345 | done` |
346 | |
347 | - ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(), waitForRestartLinux, packageInstalled, packageNotInstalled) |
348 | + ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(image), waitForRestartLinux, packageInstalled, packageNotInstalled) |
349 | key = "startup-script" |
350 | |
351 | case "yum": |
352 | @@ -304,7 +309,7 @@ while ($true) { |
353 | case "cos": |
354 | script = fmt.Sprintf("%s\n%s\n%s", utils.CurlPost, waitForRestartLinux, scriptLinux) |
355 | case "apt": |
356 | - script = fmt.Sprintf("%s\n%s\n%s", utils.InstallOSConfigDeb(), waitForRestartLinux, scriptLinux) |
357 | + script = fmt.Sprintf("%s\n%s\n%s", utils.InstallOSConfigDeb(image), waitForRestartLinux, scriptLinux) |
358 | case "yum": |
359 | script = fmt.Sprintf("%s\n%s\n%s", utils.InstallOSConfigEL(image), waitForRestartLinux, scriptLinux) |
360 | case "zypper": |
361 | @@ -397,7 +402,7 @@ while ($true) { |
362 | case "cos": |
363 | script = fmt.Sprintf("%s\n%s\n%s", utils.CurlPost, waitForRestartLinux, scriptLinux) |
364 | case "apt": |
365 | - script = fmt.Sprintf("%s\n%s\n%s", utils.InstallOSConfigDeb(), waitForRestartLinux, scriptLinux) |
366 | + script = fmt.Sprintf("%s\n%s\n%s", utils.InstallOSConfigDeb(image), waitForRestartLinux, scriptLinux) |
367 | case "yum": |
368 | // A dependancy package for ed-1.1-3.3.el6.x86_64.rpm which is used in Enterprise-Linux Recipe steps tests |
369 | yumInstallInfoPackage := fmt.Sprintf("\n%s\n", "yum install -y info") |
370 | diff --git a/e2e_tests/test_suites/inventory/test_setup.go b/e2e_tests/test_suites/inventory/test_setup.go |
371 | index 05580c2..d75e8ad 100644 |
372 | --- a/e2e_tests/test_suites/inventory/test_setup.go |
373 | +++ b/e2e_tests/test_suites/inventory/test_setup.go |
374 | @@ -45,9 +45,23 @@ var ( |
375 | timeout: 30 * time.Minute, |
376 | } |
377 | |
378 | - aptSetup = &inventoryTestSetup{ |
379 | + busterAptSetup = &inventoryTestSetup{ |
380 | packageType: []string{"deb"}, |
381 | - startup: compute.BuildInstanceMetadataItem("startup-script", utils.InstallOSConfigDeb()), |
382 | + startup: compute.BuildInstanceMetadataItem("startup-script", utils.InstallOSConfigDeb("debian-10")), |
383 | + machineType: "e2-standard-2", |
384 | + timeout: 25 * time.Minute, |
385 | + } |
386 | + |
387 | + bullseyeAptSetup = &inventoryTestSetup{ |
388 | + packageType: []string{"deb"}, |
389 | + startup: compute.BuildInstanceMetadataItem("startup-script", utils.InstallOSConfigDeb("debian-11")), |
390 | + machineType: "e2-standard-2", |
391 | + timeout: 25 * time.Minute, |
392 | + } |
393 | + |
394 | + bookwormAptSetup = &inventoryTestSetup{ |
395 | + packageType: []string{"deb"}, |
396 | + startup: compute.BuildInstanceMetadataItem("startup-script", utils.InstallOSConfigDeb("debian-12")), |
397 | machineType: "e2-standard-2", |
398 | timeout: 25 * time.Minute, |
399 | } |
400 | @@ -90,12 +104,14 @@ var ( |
401 | func headImageTestSetup() (setup []*inventoryTestSetup) { |
402 | // This maps a specific inventoryTestSetup to test setup names and associated images. |
403 | headTestSetupMapping := map[*inventoryTestSetup]map[string]string{ |
404 | - windowsSetup: utils.HeadWindowsImages, |
405 | - el7Setup: utils.HeadEL7Images, |
406 | - el8Setup: utils.HeadEL8Images, |
407 | - el9Setup: utils.HeadEL9Images, |
408 | - aptSetup: utils.HeadAptImages, |
409 | - suseSetup: utils.HeadSUSEImages, |
410 | + windowsSetup: utils.HeadWindowsImages, |
411 | + el7Setup: utils.HeadEL7Images, |
412 | + el8Setup: utils.HeadEL8Images, |
413 | + el9Setup: utils.HeadEL9Images, |
414 | + busterAptSetup: utils.HeadBusterAptImages, |
415 | + bullseyeAptSetup: utils.HeadBullseyeAptImages, |
416 | + bookwormAptSetup: utils.HeadBookwormAptImages, |
417 | + suseSetup: utils.HeadSUSEImages, |
418 | } |
419 | |
420 | // TODO: remove this hack and setup specific test suites for each test type. |
421 | diff --git a/e2e_tests/test_suites/inventoryreporting/test_setup.go b/e2e_tests/test_suites/inventoryreporting/test_setup.go |
422 | index 319ebf4..21f6a6e 100644 |
423 | --- a/e2e_tests/test_suites/inventoryreporting/test_setup.go |
424 | +++ b/e2e_tests/test_suites/inventoryreporting/test_setup.go |
425 | @@ -110,31 +110,10 @@ var ( |
426 | }, |
427 | } |
428 | |
429 | - aptSetup = &inventoryTestSetup{ |
430 | - packageType: []string{"deb"}, |
431 | - startup: compute.BuildInstanceMetadataItem("startup-script", getStartupScriptDeb()), |
432 | - machineType: "e2-medium", |
433 | - timeout: 25 * time.Minute, |
434 | - itemCheck: func(items map[string]*osconfigpb.Inventory_Item) error { |
435 | - var bashFound bool |
436 | - var cowsayFound bool |
437 | - for _, item := range items { |
438 | - if item.GetInstalledPackage().GetAptPackage().GetPackageName() == "bash" { |
439 | - bashFound = true |
440 | - } |
441 | - if item.GetAvailablePackage().GetAptPackage().GetPackageName() == "cowsay" { |
442 | - cowsayFound = true |
443 | - } |
444 | - } |
445 | - if !bashFound { |
446 | - return errors.New("did not find 'bash' in installed packages") |
447 | - } |
448 | - if !cowsayFound { |
449 | - return errors.New("did not find 'cowsay' in available packages") |
450 | - } |
451 | - return nil |
452 | - }, |
453 | - } |
454 | + // apt setup |
455 | + busterAptSetup = createAptSetup("debian-10") |
456 | + bullseyeAptSetup = createAptSetup("debian-11") |
457 | + bookwormAptSetup = createAptSetup("debian-12") |
458 | |
459 | yumBashInstalledCheck = func(items map[string]*osconfigpb.Inventory_Item) error { |
460 | var bashFound bool |
461 | @@ -221,6 +200,34 @@ var ( |
462 | } |
463 | ) |
464 | |
465 | +func createAptSetup(image string) *inventoryTestSetup { |
466 | + return &inventoryTestSetup{ |
467 | + packageType: []string{"deb"}, |
468 | + startup: compute.BuildInstanceMetadataItem("startup-script", getStartupScriptDeb(image)), |
469 | + machineType: "e2-medium", |
470 | + timeout: 25 * time.Minute, |
471 | + itemCheck: func(items map[string]*osconfigpb.Inventory_Item) error { |
472 | + var bashFound bool |
473 | + var cowsayFound bool |
474 | + for _, item := range items { |
475 | + if item.GetInstalledPackage().GetAptPackage().GetPackageName() == "bash" { |
476 | + bashFound = true |
477 | + } |
478 | + if item.GetAvailablePackage().GetAptPackage().GetPackageName() == "cowsay" { |
479 | + cowsayFound = true |
480 | + } |
481 | + } |
482 | + if !bashFound { |
483 | + return errors.New("did not find 'bash' in installed packages") |
484 | + } |
485 | + if !cowsayFound { |
486 | + return errors.New("did not find 'cowsay' in available packages") |
487 | + } |
488 | + return nil |
489 | + }, |
490 | + } |
491 | +} |
492 | + |
493 | func getStartupScriptEL(image string) string { |
494 | ss := ` |
495 | echo 'Adding test repo' |
496 | @@ -243,9 +250,14 @@ done |
497 | return fmt.Sprintf(ss, utils.InstallOSConfigEL(image)) |
498 | } |
499 | |
500 | -func getStartupScriptDeb() string { |
501 | +func getStartupScriptDeb(image string) string { |
502 | ss := ` |
503 | echo 'Adding test repo' |
504 | + |
505 | +# install gnupg2 if not exist |
506 | +apt-get update |
507 | +apt-get install -y gnupg2 |
508 | + |
509 | echo 'deb http://packages.cloud.google.com/apt osconfig-agent-test-repository main' >> /etc/apt/sources.list |
510 | curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - |
511 | while fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1; do |
512 | @@ -254,7 +266,7 @@ done |
513 | apt-get update |
514 | apt-get -y install cowsay=3.03+dfsg1-10 || exit 1 |
515 | %s` |
516 | - return fmt.Sprintf(ss, utils.InstallOSConfigDeb()) |
517 | + return fmt.Sprintf(ss, utils.InstallOSConfigDeb(image)) |
518 | } |
519 | |
520 | func getStartupScriptGoo() string { |
521 | @@ -284,12 +296,14 @@ zypper -n --no-gpg-checks install cowsay-3.03-20.el7 |
522 | func headImageTestSetup() (setup []*inventoryTestSetup) { |
523 | // This maps a specific inventoryTestSetup to test setup names and associated images. |
524 | headTestSetupMapping := map[*inventoryTestSetup]map[string]string{ |
525 | - windowsSetup: utils.HeadWindowsImages, |
526 | - el7Setup: utils.HeadEL7Images, |
527 | - el8Setup: utils.HeadEL8Images, |
528 | - el9Setup: utils.HeadEL9Images, |
529 | - aptSetup: utils.HeadAptImages, |
530 | - suseSetup: utils.HeadSUSEImages, |
531 | + windowsSetup: utils.HeadWindowsImages, |
532 | + el7Setup: utils.HeadEL7Images, |
533 | + el8Setup: utils.HeadEL8Images, |
534 | + el9Setup: utils.HeadEL9Images, |
535 | + busterAptSetup: utils.HeadBusterAptImages, |
536 | + bullseyeAptSetup: utils.HeadBullseyeAptImages, |
537 | + bookwormAptSetup: utils.HeadBookwormAptImages, |
538 | + suseSetup: utils.HeadSUSEImages, |
539 | } |
540 | |
541 | // TODO: remove this hack and setup specific test suites for each test type. |
542 | diff --git a/e2e_tests/test_suites/ospolicies/ospolicies_test_data.go b/e2e_tests/test_suites/ospolicies/ospolicies_test_data.go |
543 | index 59bf937..da99bde 100644 |
544 | --- a/e2e_tests/test_suites/ospolicies/ospolicies_test_data.go |
545 | +++ b/e2e_tests/test_suites/ospolicies/ospolicies_test_data.go |
546 | @@ -1383,7 +1383,7 @@ func buildLinuxExecResourceTests(name, image, pkgManager, key string) *osPolicyT |
547 | }, |
548 | Enforce: &osconfigpb.OSPolicy_Resource_ExecResource_Exec{ |
549 | Source: &osconfigpb.OSPolicy_Resource_ExecResource_Exec_Script{ |
550 | - Script: fmt.Sprintf("head -c 200KB /dev/zero > %s\nexit 100", checkPaths[5]), |
551 | + Script: fmt.Sprintf("head -c 1000KB /dev/zero > %s\nexit 100", checkPaths[5]), |
552 | }, |
553 | Interpreter: osconfigpb.OSPolicy_Resource_ExecResource_Exec_SHELL, |
554 | OutputFilePath: checkPaths[5], |
555 | @@ -1457,7 +1457,7 @@ func buildLinuxExecResourceTests(name, image, pkgManager, key string) *osPolicyT |
556 | }, |
557 | { |
558 | Type: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_OSPolicyResourceConfigStep_DESIRED_STATE_ENFORCEMENT, |
559 | - ErrorMessage: `Enforce state: resource "exec-output-too-large" error: contents of OutputFilePath greater than 100K`, |
560 | + ErrorMessage: `Enforce state: resource "exec-output-too-large" error: contents of OutputFilePath greater than 500K`, |
561 | }, |
562 | { |
563 | Type: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_OSPolicyResourceConfigStep_DESIRED_STATE_CHECK_POST_ENFORCEMENT, |
564 | @@ -1466,7 +1466,7 @@ func buildLinuxExecResourceTests(name, image, pkgManager, key string) *osPolicyT |
565 | ComplianceState: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_COMPLIANT, |
566 | Output: &osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_ExecResourceOutput_{ |
567 | ExecResourceOutput: &osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_ExecResourceOutput{ |
568 | - EnforcementOutput: make([]byte, 100*1024), |
569 | + EnforcementOutput: make([]byte, 500*1024), |
570 | }, |
571 | }, |
572 | }, |
573 | @@ -1725,7 +1725,7 @@ func buildWindowsExecResourceTests(name, image, pkgManager, key string) *osPolic |
574 | }, |
575 | Enforce: &osconfigpb.OSPolicy_Resource_ExecResource_Exec{ |
576 | Source: &osconfigpb.OSPolicy_Resource_ExecResource_Exec_Script{ |
577 | - Script: fmt.Sprintf("fsutil file createnew %s 200000\nexit 100", checkPaths[7]), |
578 | + Script: fmt.Sprintf("fsutil file createnew %s 1000000\nexit 100", checkPaths[7]), |
579 | }, |
580 | Interpreter: osconfigpb.OSPolicy_Resource_ExecResource_Exec_SHELL, |
581 | OutputFilePath: checkPaths[7], |
582 | @@ -1809,7 +1809,7 @@ func buildWindowsExecResourceTests(name, image, pkgManager, key string) *osPolic |
583 | }, |
584 | { |
585 | Type: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_OSPolicyResourceConfigStep_DESIRED_STATE_ENFORCEMENT, |
586 | - ErrorMessage: `Enforce state: resource "exec-output-too-large" error: contents of OutputFilePath greater than 100K`, |
587 | + ErrorMessage: `Enforce state: resource "exec-output-too-large" error: contents of OutputFilePath greater than 500K`, |
588 | }, |
589 | { |
590 | Type: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_OSPolicyResourceConfigStep_DESIRED_STATE_CHECK_POST_ENFORCEMENT, |
591 | @@ -1818,7 +1818,7 @@ func buildWindowsExecResourceTests(name, image, pkgManager, key string) *osPolic |
592 | ComplianceState: osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_COMPLIANT, |
593 | Output: &osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_ExecResourceOutput_{ |
594 | ExecResourceOutput: &osconfigpb.OSPolicyAssignmentReport_OSPolicyCompliance_OSPolicyResourceCompliance_ExecResourceOutput{ |
595 | - EnforcementOutput: make([]byte, 100*1024), |
596 | + EnforcementOutput: make([]byte, 500*1024), |
597 | }, |
598 | }, |
599 | }, |
600 | diff --git a/e2e_tests/test_suites/ospolicies/ospolicies_utils.go b/e2e_tests/test_suites/ospolicies/ospolicies_utils.go |
601 | index 21c7cd4..af91080 100644 |
602 | --- a/e2e_tests/test_suites/ospolicies/ospolicies_utils.go |
603 | +++ b/e2e_tests/test_suites/ospolicies/ospolicies_utils.go |
604 | @@ -31,6 +31,7 @@ func getStartupScriptPackage(image, pkgManager string) *computeApi.MetadataItems |
605 | wantRemove := "vim" |
606 | ss = `set -x |
607 | # install the package we want removed |
608 | +apt-get update |
609 | apt-get -y install %[2]s |
610 | # remove the package we want installed |
611 | apt-get -y remove %[3]s |
612 | @@ -57,7 +58,7 @@ while true; do |
613 | sleep 10 |
614 | done` |
615 | |
616 | - ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(), wantRemove, wantInstall, packageInstalled, packageNotInstalled) |
617 | + ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(image), wantRemove, wantInstall, packageInstalled, packageNotInstalled) |
618 | key = "startup-script" |
619 | |
620 | case "deb": |
621 | @@ -81,7 +82,7 @@ while true; do |
622 | fi |
623 | sleep 10 |
624 | done` |
625 | - ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(), wantInstall[0], wantInstall[1], packageInstalled) |
626 | + ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(image), wantInstall[0], wantInstall[1], packageInstalled) |
627 | key = "startup-script" |
628 | |
629 | case "yum": |
630 | @@ -241,7 +242,7 @@ while true; do |
631 | sleep 10 |
632 | done` |
633 | |
634 | - ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(), packageName, packageInstalled) |
635 | + ss = fmt.Sprintf(ss, utils.InstallOSConfigDeb(image), packageName, packageInstalled) |
636 | key = "startup-script" |
637 | |
638 | case "yum": |
639 | @@ -331,7 +332,7 @@ curl -X PUT --data "1" $uri -H "Metadata-Flavor: Google"`, fileExists) |
640 | |
641 | switch pkgManager { |
642 | case "apt": |
643 | - ss = fmt.Sprintf(linux, dnePath, utils.InstallOSConfigDeb(), fileDNE) |
644 | + ss = fmt.Sprintf(linux, dnePath, utils.InstallOSConfigDeb(image), fileDNE) |
645 | for _, p := range wantPaths { |
646 | ss += fmt.Sprintf(linuxCheck, p) |
647 | } |
648 | @@ -424,7 +425,7 @@ curl -X PUT --data "1" $uri -H "Metadata-Flavor: Google"`, fileExists) |
649 | |
650 | switch pkgManager { |
651 | case "apt": |
652 | - ss = linux + utils.InstallOSConfigDeb() + linuxChecks + linuxEnd |
653 | + ss = linux + utils.InstallOSConfigDeb(image) + linuxChecks + linuxEnd |
654 | key = "startup-script" |
655 | |
656 | case "yum": |
657 | diff --git a/e2e_tests/test_suites/patch/test_setup.go b/e2e_tests/test_suites/patch/test_setup.go |
658 | index f2b701c..3728501 100644 |
659 | --- a/e2e_tests/test_suites/patch/test_setup.go |
660 | +++ b/e2e_tests/test_suites/patch/test_setup.go |
661 | @@ -100,61 +100,76 @@ echo 'Pin-priority: 9999' >> /etc/apt/preferences |
662 | }, |
663 | machineType: "e2-standard-4", |
664 | } |
665 | - aptSetup = &patchTestSetup{ |
666 | + |
667 | + // apt setup |
668 | + busterAptSetup = createAptPatchTestSetup("debian-10") |
669 | + bullseyeAptSetup = createAptPatchTestSetup("debian-11") |
670 | + bookwormAptSetup = createAptPatchTestSetup("debian-12") |
671 | + |
672 | + // apt Downgrade setup |
673 | + bullseyeAptDowngradeSetup = createAptDowngradeSetup("debian-11") |
674 | + |
675 | + el7Setup = &patchTestSetup{ |
676 | assertTimeout: 30 * time.Minute, |
677 | metadata: []*computeApi.MetadataItems{ |
678 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigDeb()+linuxLocalPrePatchScript), |
679 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL7()+linuxLocalPrePatchScript), |
680 | enableOsconfig, |
681 | disableFeatures, |
682 | }, |
683 | machineType: "e2-medium", |
684 | } |
685 | - aptDowngradeSetup = &patchTestSetup{ |
686 | + el8Setup = &patchTestSetup{ |
687 | assertTimeout: 30 * time.Minute, |
688 | metadata: []*computeApi.MetadataItems{ |
689 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigDeb()+linuxLocalPrePatchScript+setUpDowngradeState), |
690 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL8()+linuxLocalPrePatchScript), |
691 | enableOsconfig, |
692 | disableFeatures, |
693 | }, |
694 | machineType: "e2-medium", |
695 | } |
696 | - el7Setup = &patchTestSetup{ |
697 | + el9Setup = &patchTestSetup{ |
698 | assertTimeout: 30 * time.Minute, |
699 | metadata: []*computeApi.MetadataItems{ |
700 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL7()+linuxLocalPrePatchScript), |
701 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL9()+linuxLocalPrePatchScript), |
702 | enableOsconfig, |
703 | disableFeatures, |
704 | }, |
705 | machineType: "e2-medium", |
706 | } |
707 | - el8Setup = &patchTestSetup{ |
708 | + suseSetup = &patchTestSetup{ |
709 | assertTimeout: 30 * time.Minute, |
710 | metadata: []*computeApi.MetadataItems{ |
711 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL8()+linuxLocalPrePatchScript), |
712 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigSUSE()+linuxLocalPrePatchScript), |
713 | enableOsconfig, |
714 | disableFeatures, |
715 | }, |
716 | machineType: "e2-medium", |
717 | } |
718 | - el9Setup = &patchTestSetup{ |
719 | +) |
720 | + |
721 | +func createAptPatchTestSetup(image string) *patchTestSetup { |
722 | + return &patchTestSetup{ |
723 | assertTimeout: 30 * time.Minute, |
724 | metadata: []*computeApi.MetadataItems{ |
725 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigEL9()+linuxLocalPrePatchScript), |
726 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigDeb(image)+linuxLocalPrePatchScript), |
727 | enableOsconfig, |
728 | disableFeatures, |
729 | }, |
730 | machineType: "e2-medium", |
731 | } |
732 | - suseSetup = &patchTestSetup{ |
733 | +} |
734 | + |
735 | +func createAptDowngradeSetup(image string) *patchTestSetup { |
736 | + return &patchTestSetup{ |
737 | assertTimeout: 30 * time.Minute, |
738 | metadata: []*computeApi.MetadataItems{ |
739 | - compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigSUSE()+linuxLocalPrePatchScript), |
740 | + compute.BuildInstanceMetadataItem("startup-script", linuxRecordBoot+utils.InstallOSConfigDeb(image)+linuxLocalPrePatchScript+setUpDowngradeState), |
741 | enableOsconfig, |
742 | disableFeatures, |
743 | }, |
744 | machineType: "e2-medium", |
745 | } |
746 | -) |
747 | +} |
748 | |
749 | func imageTestSetup(mapping map[*patchTestSetup]map[string]string) (setup []*patchTestSetup) { |
750 | for s, m := range mapping { |
751 | @@ -171,12 +186,14 @@ func imageTestSetup(mapping map[*patchTestSetup]map[string]string) (setup []*pat |
752 | func headImageTestSetup() []*patchTestSetup { |
753 | // This maps a specific patchTestSetup to test setup names and associated images. |
754 | mapping := map[*patchTestSetup]map[string]string{ |
755 | - windowsSetup: utils.HeadWindowsImages, |
756 | - el7Setup: utils.HeadEL7Images, |
757 | - el8Setup: utils.HeadEL8Images, |
758 | - el9Setup: utils.HeadEL9Images, |
759 | - aptSetup: utils.HeadAptImages, |
760 | - suseSetup: utils.HeadSUSEImages, |
761 | + windowsSetup: utils.HeadWindowsImages, |
762 | + el7Setup: utils.HeadEL7Images, |
763 | + el8Setup: utils.HeadEL8Images, |
764 | + el9Setup: utils.HeadEL9Images, |
765 | + busterAptSetup: utils.HeadBusterAptImages, |
766 | + bullseyeAptSetup: utils.HeadBullseyeAptImages, |
767 | + bookwormAptSetup: utils.HeadBookwormAptImages, |
768 | + suseSetup: utils.HeadSUSEImages, |
769 | } |
770 | |
771 | return imageTestSetup(mapping) |
772 | @@ -189,7 +206,6 @@ func oldImageTestSetup() []*patchTestSetup { |
773 | el7Setup: utils.OldEL7Images, |
774 | el8Setup: utils.OldEL8Images, |
775 | el9Setup: utils.OldEL9Images, |
776 | - aptSetup: utils.OldAptImages, |
777 | suseSetup: utils.OldSUSEImages, |
778 | } |
779 | |
780 | @@ -199,7 +215,9 @@ func oldImageTestSetup() []*patchTestSetup { |
781 | func aptHeadImageTestSetup() []*patchTestSetup { |
782 | // This maps a specific patchTestSetup to test setup names and associated images. |
783 | mapping := map[*patchTestSetup]map[string]string{ |
784 | - aptSetup: utils.HeadAptImages, |
785 | + busterAptSetup: utils.HeadBusterAptImages, |
786 | + bullseyeAptSetup: utils.HeadBullseyeAptImages, |
787 | + busterAptSetup: utils.HeadBookwormAptImages, |
788 | } |
789 | |
790 | return imageTestSetup(mapping) |
791 | @@ -208,7 +226,7 @@ func aptHeadImageTestSetup() []*patchTestSetup { |
792 | func aptDowngradeImageTestSetup() []*patchTestSetup { |
793 | // This maps a specific patchTestSetup to test setup names and associated images. |
794 | mapping := map[*patchTestSetup]map[string]string{ |
795 | - aptDowngradeSetup: utils.DowngradeAptImages, |
796 | + bullseyeAptDowngradeSetup: utils.DowngradeBullseyeAptImages, |
797 | } |
798 | |
799 | return imageTestSetup(mapping) |
800 | diff --git a/e2e_tests/utils/utils.go b/e2e_tests/utils/utils.go |
801 | index c37619d..d288a3f 100644 |
802 | --- a/e2e_tests/utils/utils.go |
803 | +++ b/e2e_tests/utils/utils.go |
804 | @@ -108,18 +108,24 @@ EOM` |
805 | ) |
806 | |
807 | // InstallOSConfigDeb installs the osconfig agent on deb based systems. |
808 | -func InstallOSConfigDeb() string { |
809 | +func InstallOSConfigDeb(image string) string { |
810 | if config.AgentRepo() == "" { |
811 | return CurlPost |
812 | } |
813 | + osName := getDebOsName(image) |
814 | return fmt.Sprintf(` |
815 | sleep 10 |
816 | systemctl stop google-osconfig-agent |
817 | -echo 'deb http://packages.cloud.google.com/apt google-osconfig-agent-%s main' >> /etc/apt/sources.list |
818 | + |
819 | +# install gnupg2 if not exist |
820 | +apt-get update |
821 | +apt-get install -y gnupg2 |
822 | + |
823 | +echo 'deb http://packages.cloud.google.com/apt google-osconfig-agent-%s-%s main' >> /etc/apt/sources.list |
824 | curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - |
825 | apt-get update |
826 | apt-get install -y google-osconfig-agent |
827 | -systemctl start google-osconfig-agent`+CurlPost, config.AgentRepo()) |
828 | +systemctl start google-osconfig-agent`+CurlPost, osName, config.AgentRepo()) |
829 | } |
830 | |
831 | // InstallOSConfigGooGet installs the osconfig agent on Windows systems. |
832 | @@ -191,58 +197,78 @@ func InstallOSConfigEL7() string { |
833 | return fmt.Sprintf(yumRepoSetup+yumInstallAgent, "el7", config.AgentRepo(), 0) |
834 | } |
835 | |
836 | +// containsAnyOf checks if a string contains any substring from a given list. |
837 | +func containsAnyOf(str string, substrings []string) bool { |
838 | + for _, substring := range substrings { |
839 | + if strings.Contains(str, substring) { |
840 | + return true |
841 | + } |
842 | + } |
843 | + return false |
844 | +} |
845 | + |
846 | // InstallOSConfigEL installs the osconfig agent on el based systems. |
847 | func InstallOSConfigEL(image string) string { |
848 | + imageName := path.Base(image) |
849 | switch { |
850 | - case strings.Contains(path.Base(image), "9"): |
851 | + case image == "9" || containsAnyOf(imageName, []string{"rhel-9", "rhel-sap-9", "centos-stream-9", "rocky-linux-9"}): |
852 | return InstallOSConfigEL9() |
853 | - case strings.Contains(path.Base(image), "8"): |
854 | + case image == "8" || containsAnyOf(imageName, []string{"rhel-8", "rhel-sap-8", "centos-stream-8", "rocky-linux-8"}): |
855 | return InstallOSConfigEL8() |
856 | - case strings.Contains(path.Base(image), "7"): |
857 | + case image == "7" || containsAnyOf(imageName, []string{"rhel-7", "rhel-sap-7", "centos-7"}): |
858 | return InstallOSConfigEL7() |
859 | |
860 | } |
861 | return "" |
862 | } |
863 | |
864 | -// DowngradeAptImages is a single image that are used for testing downgrade case with apt-get |
865 | -var DowngradeAptImages = map[string]string{ |
866 | - "debian-cloud/debian-10": "projects/debian-cloud/global/images/debian-10-buster-v20231010", |
867 | +// getDebOsType returns the equivalent os_name for deb version (e.g. debian-11 --> bullseye) |
868 | +func getDebOsName(image string) string { |
869 | + imageName := path.Base(image) |
870 | + switch { |
871 | + case image == "10" || containsAnyOf(imageName, []string{"debian-10", "buster"}): |
872 | + return "buster" |
873 | + case image == "11" || containsAnyOf(imageName, []string{"debian-11", "bullseye"}): |
874 | + return "bullseye" |
875 | + case image == "12" || containsAnyOf(imageName, []string{"debian-12", "bookworm"}): |
876 | + return "bookworm" |
877 | + } |
878 | + return "" |
879 | } |
880 | |
881 | -// HeadAptImages is a map of names to image paths for public image families that use APT. |
882 | -var HeadAptImages = map[string]string{ |
883 | - // Debian images. |
884 | +// DowngradeBullseyeAptImages is a single image that are used for testing downgrade case with apt-get |
885 | +var DowngradeBullseyeAptImages = map[string]string{ |
886 | "debian-cloud/debian-11": "projects/debian-cloud/global/images/debian-11-bullseye-v20231010", |
887 | - "debian-cloud/debian-12": "projects/debian-cloud/global/images/debian-12-bookworm-v20231010", |
888 | +} |
889 | |
890 | - // Ubuntu images. |
891 | - "ubuntu-os-cloud/ubuntu-2304": "projects/ubuntu-os-cloud/global/images/ubuntu-2304-lunar-amd64-v20231020", |
892 | - "ubuntu-os-cloud/ubuntu-2314": "projects/ubuntu-os-cloud/global/images/ubuntu-2310-mantic-amd64-v20231011", |
893 | +// HeadBusterAptImages empty for now as debian-10 wil reach EOL and some of its repos are not reachable anymore. |
894 | +var HeadBusterAptImages = map[string]string{ |
895 | + // Debian images. |
896 | } |
897 | |
898 | -// OldAptImages is a map of names to image paths for old (deprecated) images that use APT. |
899 | -var OldAptImages = map[string]string{ |
900 | +// HeadBullseyeAptImages is a map of names to image paths for public debian-11 images |
901 | +var HeadBullseyeAptImages = map[string]string{ |
902 | // Debian images. |
903 | - "old/debian-10": "projects/debian-cloud/global/images/debian-10-buster-v20231010", |
904 | + "debian-cloud/debian-11": "projects/debian-cloud/global/images/family/debian-11", |
905 | +} |
906 | |
907 | - // Ubuntu images. |
908 | - "old/ubuntu-2004": "projects/ubuntu-os-cloud/global/images/ubuntu-2004-focal-v20230918", |
909 | +// HeadBookwormAptImages is a map of names to image paths for public debian-12 images |
910 | +var HeadBookwormAptImages = map[string]string{ |
911 | + // Debian images. |
912 | + "debian-cloud/debian-12": "projects/debian-cloud/global/images/family/debian-12", |
913 | } |
914 | |
915 | // HeadSUSEImages is a map of names to image paths for public SUSE images. |
916 | var HeadSUSEImages = map[string]string{ |
917 | - "suse-cloud/sles-12-sp5": "projects/suse-cloud/global/images/sles-12-sp5-v20230807-x86-64", |
918 | - "suse-cloud/sles-15-sp5": "projects/suse-cloud/global/images/sles-15-sp5-v20230921-x86-64", |
919 | + "suse-cloud/sles-12-sp5": "projects/suse-cloud/global/images/family/sles-12", |
920 | + "suse-cloud/sles-15-sp5": "projects/suse-cloud/global/images/family/sles-15", |
921 | |
922 | - "suse-sap-cloud/sles-12-sp5-sap": "projects/suse-sap-cloud/global/images/sles-12-sp5-sap-v20231019-x86-64", |
923 | - "suse-sap-cloud/sles-15-sp5-sap": "projects/suse-sap-cloud/global/images/sles-15-sp5-sap-v20230921-x86-64", |
924 | + "suse-sap-cloud/sles-12-sp5-sap": "projects/suse-sap-cloud/global/images/family/sles-12-sp5-sap", |
925 | + "suse-sap-cloud/sles-15-sp5-sap": "projects/suse-sap-cloud/global/images/family/sles-15-sp5-sap", |
926 | |
927 | - "suse-sap-cloud/sles-15-sp4-hardened-sap": "projects/suse-sap-cloud/global/images/sles-sap-15-sp4-hardened-v20230828-x86-64", |
928 | - "suse-sap-cloud/sles-15-sp5-hardened-sap": "projects/suse-sap-cloud/global/images/sles-sap-15-sp5-hardened-v20230921-x86-64", |
929 | + "suse-sap-cloud/sles-15-sp5-hardened-sap": "projects/suse-sap-cloud/global/images/family/sles-sap-15-sp5-hardened", |
930 | |
931 | - "opensuse-cloud/opensuse-leap-15-4": "projects/opensuse-cloud/global/images/opensuse-leap-15-4-v20230907-x86-64", |
932 | - "opensuse-cloud/opensuse-leap-15-5": "projects/opensuse-cloud/global/images/opensuse-leap-15-5-v20230908-x86-64", |
933 | + "opensuse-cloud/opensuse-leap-15": "projects/opensuse-cloud/global/images/family/opensuse-leap", |
934 | } |
935 | |
936 | // OldSUSEImages is a map of names to image paths for old SUSE images. |
937 | @@ -255,11 +281,11 @@ var OldSUSEImages = map[string]string{ |
938 | |
939 | // HeadEL7Images is a map of names to image paths for public EL7 image families. (RHEL, CentOS) |
940 | var HeadEL7Images = map[string]string{ |
941 | - "centos-cloud/centos-7": "projects/centos-cloud/global/images/centos-7-v20231010", |
942 | + "centos-cloud/centos-7": "projects/centos-cloud/global/images/family/centos-7", |
943 | |
944 | - "rhel-cloud/rhel-7": "projects/rhel-cloud/global/images/rhel-7-v20231010", |
945 | + "rhel-cloud/rhel-7": "projects/rhel-cloud/global/images/family/rhel-7", |
946 | |
947 | - "rhel-sap-cloud/rhel-7-sap": "projects/rhel-sap-cloud/global/images/rhel-7-9-sap-v20231011", |
948 | + "rhel-sap-cloud/rhel-7-9-sap": "projects/rhel-sap-cloud/global/images/family/rhel-7-9-sap-ha", |
949 | } |
950 | |
951 | // OldEL7Images is a map of names to image paths for old EL7 images. |
952 | @@ -269,18 +295,16 @@ var OldEL7Images = map[string]string{ |
953 | |
954 | // HeadEL8Images is a map of names to image paths for public EL8 image families. (RHEL, CentOS, Rocky) |
955 | var HeadEL8Images = map[string]string{ |
956 | - "centos-cloud/centos-stream-8": "projects/centos-cloud/global/images/centos-stream-8-v20231010", |
957 | + "centos-cloud/centos-stream-8": "projects/centos-cloud/global/images/family/centos-stream-8", |
958 | |
959 | - "rhel-cloud/rhel-8": "projects/rhel-cloud/global/images/rhel-8-v20231010", |
960 | + "rhel-cloud/rhel-8": "projects/rhel-cloud/global/images/family/rhel-8", |
961 | |
962 | - "rhel-sap-cloud/rhel-8-1-sap": "projects/rhel-sap-cloud/global/images/rhel-8-1-sap-v20231010", |
963 | - "rhel-sap-cloud/rhel-8-2-sap": "projects/rhel-sap-cloud/global/images/rhel-8-2-sap-v20231010", |
964 | - "rhel-sap-cloud/rhel-8-4-sap": "projects/rhel-sap-cloud/global/images/rhel-8-4-sap-v20231010", |
965 | - "rhel-sap-cloud/rhel-8-6-sap": "projects/rhel-sap-cloud/global/images/rhel-8-6-sap-v20231010", |
966 | - "rhel-sap-cloud/rhel-8-8-sap": "projects/rhel-sap-cloud/global/images/rhel-8-8-sap-v20231010", |
967 | + "rhel-sap-cloud/rhel-8-4-sap": "projects/rhel-sap-cloud/global/images/family/rhel-8-4-sap-ha", |
968 | + "rhel-sap-cloud/rhel-8-6-sap": "projects/rhel-sap-cloud/global/images/family/rhel-8-6-sap-ha", |
969 | + "rhel-sap-cloud/rhel-8-8-sap": "projects/rhel-sap-cloud/global/images/family/rhel-8-8-sap-ha", |
970 | |
971 | - "rocky-linux-cloud/rocky-linux-8": "projects/rocky-linux-cloud/global/images/rocky-linux-8-v20231010", |
972 | - "rocky-linux-cloud/rocky-linux-8-opt-gcp": "projects/rocky-linux-cloud/global/images/rocky-linux-8-optimized-gcp-v20231010", |
973 | + "rocky-linux-cloud/rocky-linux-8": "projects/rocky-linux-cloud/global/images/family/rocky-linux-8", |
974 | + "rocky-linux-cloud/rocky-linux-8-optimized-gcp": "projects/rocky-linux-cloud/global/images/family/rocky-linux-8-optimized-gcp", |
975 | } |
976 | |
977 | // OldEL8Images is a map of names to image paths for old EL8 images. (RHEL, CentOS, Rocky) |
978 | @@ -290,15 +314,15 @@ var OldEL8Images = map[string]string{ |
979 | |
980 | // HeadEL9Images is a map of names to image paths for public EL9 image families. (RHEL, CentOS, Rocky) |
981 | var HeadEL9Images = map[string]string{ |
982 | - "centos-cloud/centos-stream-9": "projects/centos-cloud/global/images/centos-stream-9-v20231010", |
983 | + "centos-cloud/centos-stream-9": "projects/centos-cloud/global/images/family/centos-stream-9", |
984 | |
985 | - "rhel-cloud/rhel-9": "projects/rhel-cloud/global/images/rhel-9-v20231010", |
986 | + "rhel-cloud/rhel-9": "projects/rhel-cloud/global/images/family/rhel-9", |
987 | |
988 | - "rhel-sap-cloud/rhel-9-0-sap": "projects/rhel-sap-cloud/global/images/rhel-9-0-sap-v20231010", |
989 | - "rhel-sap-cloud/rhel-9-2-sap": "projects/rhel-sap-cloud/global/images/rhel-9-2-sap-v20231010", |
990 | + "rhel-sap-cloud/rhel-9-0-sap": "projects/rhel-sap-cloud/global/images/family/rhel-9-0-sap-ha", |
991 | + "rhel-sap-cloud/rhel-9-2-sap": "projects/rhel-sap-cloud/global/images/family/rhel-9-2-sap-ha", |
992 | |
993 | - "rocky-linux-cloud/rocky-linux-9": "projects/rocky-linux-cloud/global/images/rocky-linux-9-v20231010", |
994 | - "rocky-linux-cloud/rocky-linux-9-opt-gcp": "projects/rocky-linux-cloud/global/images/rocky-linux-9-optimized-gcp-v20231010", |
995 | + "rocky-linux-cloud/rocky-linux-9": "projects/rocky-linux-cloud/global/images/family/rocky-linux-9", |
996 | + "rocky-linux-cloud/rocky-linux-9-optimized-gcp": "projects/rocky-linux-cloud/global/images/family/rocky-linux-9-optimized-gcp", |
997 | } |
998 | |
999 | // OldEL9Images is a map of names to image paths for old EL9 images. (RHEL, CentOS, Rocky) |
1000 | @@ -321,14 +345,34 @@ var HeadELImages = func() (newMap map[string]string) { |
1001 | return |
1002 | }() |
1003 | |
1004 | +// HeadAptImages is a map of names to image paths for public EL image families. (RHEL, CentOS, Rocky) |
1005 | +var HeadAptImages = func() (newMap map[string]string) { |
1006 | + newMap = make(map[string]string) |
1007 | + for k, v := range HeadBusterAptImages { |
1008 | + newMap[k] = v |
1009 | + } |
1010 | + for k, v := range HeadBullseyeAptImages { |
1011 | + newMap[k] = v |
1012 | + } |
1013 | + for k, v := range HeadBookwormAptImages { |
1014 | + newMap[k] = v |
1015 | + } |
1016 | + return |
1017 | +}() |
1018 | + |
1019 | // HeadWindowsImages is a map of names to image paths for public Windows image families. |
1020 | var HeadWindowsImages = map[string]string{ |
1021 | - "windows-cloud/win-2016-dc-core": "projects/windows-cloud/global/images/windows-server-2016-dc-core-v20231011", |
1022 | - "windows-cloud/win-2016-dc": "projects/windows-cloud/global/images/windows-server-2016-dc-v20231011", |
1023 | - "windows-cloud/win-2019-dc-core": "projects/windows-cloud/global/images/windows-server-2019-dc-core-v20231011", |
1024 | - "windows-cloud/win-2019-dc": "projects/windows-cloud/global/images/windows-server-2019-dc-v20231011", |
1025 | - "windows-cloud/win-2022-dc-core": "projects/windows-cloud/global/images/windows-server-2022-dc-core-v20231011", |
1026 | - "windows-cloud/win-2022-dc": "projects/windows-cloud/global/images/windows-server-2022-dc-v20231011", |
1027 | + "windows-cloud/windows-2016": "projects/windows-cloud/global/images/family/windows-2016", |
1028 | + "windows-cloud/windows-2016-core": "projects/windows-cloud/global/images/family/windows-2016-core", |
1029 | + "windows-cloud/windows-2019": "projects/windows-cloud/global/images/family/windows-2019", |
1030 | + "windows-cloud/windows-2019-core": "projects/windows-cloud/global/images/family/windows-2019-core", |
1031 | + |
1032 | + // Testing of win-2022-dc disabled because of https://techcommunity.microsoft.com/t5/windows-server-for-it-pro/faulty-patches-on-server-2022/m-p/4028125 |
1033 | + |
1034 | + /* |
1035 | + "windows-cloud/windows-2022": "projects/windows-cloud/global/images/family/windows-2022", |
1036 | + "windows-cloud/windows-2022-core": "projects/windows-cloud/global/images/family/windows-2022-core", |
1037 | + */ |
1038 | } |
1039 | |
1040 | // OldWindowsImages is a map of names to image paths for old Windows images. |
1041 | @@ -338,9 +382,9 @@ var OldWindowsImages = map[string]string{ |
1042 | |
1043 | // HeadCOSImages is a map of names to image paths for public COS image families. |
1044 | var HeadCOSImages = map[string]string{ |
1045 | - "cos-cloud/cos-stable": "projects/cos-cloud/global/images/cos-stable-109-17800-0-51", |
1046 | - "cos-cloud/cos-beta": "projects/cos-cloud/global/images/cos-beta-109-17800-0-51", |
1047 | - "cos-cloud/cos-dev": "projects/cos-cloud/global/images/cos-dev-113-17965-0-0", |
1048 | + "cos-cloud/cos-stable": "projects/cos-cloud/global/images/family/cos-stable", |
1049 | + "cos-cloud/cos-beta": "projects/cos-cloud/global/images/family/cos-beta", |
1050 | + "cos-cloud/cos-dev": "projects/cos-cloud/global/images/family/cos-dev", |
1051 | } |
1052 | |
1053 | // RandString generates a random string of n length. |
1054 | diff --git a/examples/OSPolicyAssignments/console/CIS/cis-exclude-check-once-a-day.yaml b/examples/OSPolicyAssignments/console/CIS/cis-exclude-check-once-a-day.yaml |
1055 | index ed675d8..d1c74b7 100644 |
1056 | --- a/examples/OSPolicyAssignments/console/CIS/cis-exclude-check-once-a-day.yaml |
1057 | +++ b/examples/OSPolicyAssignments/console/CIS/cis-exclude-check-once-a-day.yaml |
1058 | @@ -1,8 +1,7 @@ |
1059 | # An OS policy to opt-out of CIS check and check compliance status once a day. |
1060 | -osPolicies: |
1061 | -- id: exclude-cis-check-and-check-compliance-once-a-day-policy |
1062 | - mode: ENFORCEMENT |
1063 | - resourceGroups: |
1064 | +id: exclude-cis-check-and-check-compliance-once-a-day-policy |
1065 | +mode: ENFORCEMENT |
1066 | +resourceGroups: |
1067 | - resources: |
1068 | id: exclude-cis-check-and-check-compliance-once-a-day |
1069 | exec: |
1070 | diff --git a/examples/OSPolicyAssignments/console/CIS/cis-level1-once-an-hour-policy.yaml b/examples/OSPolicyAssignments/console/CIS/cis-level1-once-an-hour-policy.yaml |
1071 | index 2b393b6..9b56f07 100644 |
1072 | --- a/examples/OSPolicyAssignments/console/CIS/cis-level1-once-an-hour-policy.yaml |
1073 | +++ b/examples/OSPolicyAssignments/console/CIS/cis-level1-once-an-hour-policy.yaml |
1074 | @@ -1,8 +1,7 @@ |
1075 | # An OS policy to check CIS level 1 compliance once an hour. |
1076 | -osPolicies: |
1077 | -- id: ensure-cis-level1-compliance-once-an-hour-policy |
1078 | - mode: ENFORCEMENT |
1079 | - resourceGroups: |
1080 | +id: ensure-cis-level1-compliance-once-an-hour-policy |
1081 | +mode: ENFORCEMENT |
1082 | +resourceGroups: |
1083 | - resources: |
1084 | id: ensure-cis-level1-compliance-once-an-hour |
1085 | exec: |
1086 | diff --git a/examples/OSPolicyAssignments/console/CIS/cis-level2-once-a-day-policy.yaml b/examples/OSPolicyAssignments/console/CIS/cis-level2-once-a-day-policy.yaml |
1087 | index 81314be..c77f451 100644 |
1088 | --- a/examples/OSPolicyAssignments/console/CIS/cis-level2-once-a-day-policy.yaml |
1089 | +++ b/examples/OSPolicyAssignments/console/CIS/cis-level2-once-a-day-policy.yaml |
1090 | @@ -1,8 +1,7 @@ |
1091 | # An OS policy to check CIS level 2 compliance once a day. |
1092 | -osPolicies: |
1093 | -- id: ensure-cis-level2-compliance-once-a-day-policy |
1094 | - mode: ENFORCEMENT |
1095 | - resourceGroups: |
1096 | +id: ensure-cis-level2-compliance-once-a-day-policy |
1097 | +mode: ENFORCEMENT |
1098 | +resourceGroups: |
1099 | - resources: |
1100 | id: ensure-cis-level2-compliance-once-a-day |
1101 | exec: |
1102 | diff --git a/go.mod b/go.mod |
1103 | index 7e6c171..88e1d72 100644 |
1104 | --- a/go.mod |
1105 | +++ b/go.mod |
1106 | @@ -1,6 +1,6 @@ |
1107 | module github.com/GoogleCloudPlatform/osconfig |
1108 | |
1109 | -go 1.21 |
1110 | +go 1.21.9 |
1111 | |
1112 | require ( |
1113 | cloud.google.com/go/compute/metadata v0.2.3 |
1114 | @@ -11,16 +11,16 @@ require ( |
1115 | github.com/StackExchange/wmi v1.2.1 |
1116 | github.com/go-ole/go-ole v1.2.6 |
1117 | github.com/golang/mock v1.6.0 |
1118 | - github.com/google/go-cmp v0.5.9 |
1119 | + github.com/google/go-cmp v0.6.0 |
1120 | github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07 |
1121 | github.com/ulikunitz/xz v0.5.11 |
1122 | - golang.org/x/crypto v0.17.0 |
1123 | + golang.org/x/crypto v0.22.0 |
1124 | golang.org/x/oauth2 v0.7.0 |
1125 | - golang.org/x/sys v0.15.0 |
1126 | + golang.org/x/sys v0.19.0 |
1127 | google.golang.org/api v0.114.0 |
1128 | google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 |
1129 | google.golang.org/grpc v1.56.3 |
1130 | - google.golang.org/protobuf v1.30.0 |
1131 | + google.golang.org/protobuf v1.33.0 |
1132 | ) |
1133 | |
1134 | require ( |
1135 | @@ -31,17 +31,16 @@ require ( |
1136 | cloud.google.com/go/longrunning v0.4.1 // indirect |
1137 | github.com/golang/glog v1.1.0 // indirect |
1138 | github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect |
1139 | - github.com/golang/protobuf v1.5.3 // indirect |
1140 | + github.com/golang/protobuf v1.5.4 // indirect |
1141 | github.com/google/uuid v1.3.0 // indirect |
1142 | github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect |
1143 | github.com/googleapis/gax-go/v2 v2.7.1 // indirect |
1144 | github.com/julienschmidt/httprouter v1.3.0 // indirect |
1145 | - github.com/konsorten/go-windows-terminal-sequences v1.0.3 // indirect |
1146 | github.com/pkg/errors v0.9.1 // indirect |
1147 | - github.com/sirupsen/logrus v1.6.0 // indirect |
1148 | + github.com/sirupsen/logrus v1.9.3 // indirect |
1149 | go.chromium.org/luci v0.0.0-20201204084249-3e81ee3e83fe // indirect |
1150 | go.opencensus.io v0.24.0 // indirect |
1151 | - golang.org/x/net v0.17.0 // indirect |
1152 | + golang.org/x/net v0.23.0 // indirect |
1153 | golang.org/x/sync v0.1.0 // indirect |
1154 | golang.org/x/text v0.14.0 // indirect |
1155 | golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect |
1156 | diff --git a/go.sum b/go.sum |
1157 | index ac7a2ba..e8b587e 100644 |
1158 | --- a/go.sum |
1159 | +++ b/go.sum |
1160 | @@ -111,9 +111,8 @@ github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvq |
1161 | github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= |
1162 | github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= |
1163 | github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= |
1164 | -github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= |
1165 | -github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= |
1166 | -github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= |
1167 | +github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= |
1168 | +github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= |
1169 | github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= |
1170 | github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= |
1171 | github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= |
1172 | @@ -124,9 +123,8 @@ github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ |
1173 | github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= |
1174 | github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= |
1175 | github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= |
1176 | -github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= |
1177 | -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= |
1178 | -github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= |
1179 | +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= |
1180 | +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= |
1181 | github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= |
1182 | github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no= |
1183 | github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= |
1184 | @@ -164,7 +162,6 @@ github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfV |
1185 | github.com/julienschmidt/httprouter v1.3.0 h1:U0609e9tgbseu3rBINet9P48AI/D3oJs4dN7jwJOQ1U= |
1186 | github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= |
1187 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= |
1188 | -github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8= |
1189 | github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= |
1190 | github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= |
1191 | github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= |
1192 | @@ -177,8 +174,9 @@ github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1: |
1193 | github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= |
1194 | github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= |
1195 | github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= |
1196 | -github.com/sirupsen/logrus v1.6.0 h1:UBcNElsrwanuuMsnGSlYmtmgbb23qDR5dG+6X6Oo89I= |
1197 | github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= |
1198 | +github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ= |
1199 | +github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ= |
1200 | github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= |
1201 | github.com/smartystreets/assertions v1.1.1 h1:T/YLemO5Yp7KPzS+lVtu+WsHn8yoSwTfItdAd1r3cck= |
1202 | github.com/smartystreets/assertions v1.1.1/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo= |
1203 | @@ -189,6 +187,7 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS |
1204 | github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= |
1205 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= |
1206 | github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= |
1207 | +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= |
1208 | github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= |
1209 | github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= |
1210 | github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= |
1211 | @@ -216,8 +215,8 @@ golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8U |
1212 | golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= |
1213 | golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= |
1214 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= |
1215 | -golang.org/x/crypto v0.17.0 h1:r8bRNjWL3GshPW3gkd+RpvzWrZAwPS49OmTGZ/uhM4k= |
1216 | -golang.org/x/crypto v0.17.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4= |
1217 | +golang.org/x/crypto v0.22.0 h1:g1v0xeRhjcugydODzvb3mEM9SQ0HGp9s/nh3COQ/C30= |
1218 | +golang.org/x/crypto v0.22.0/go.mod h1:vr6Su+7cTlO45qkww3VDJlzDn0ctJvRgYbC2NvXHt+M= |
1219 | golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= |
1220 | golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= |
1221 | golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= |
1222 | @@ -277,8 +276,8 @@ golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/ |
1223 | golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= |
1224 | golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= |
1225 | golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= |
1226 | -golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= |
1227 | -golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= |
1228 | +golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs= |
1229 | +golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg= |
1230 | golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= |
1231 | golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= |
1232 | golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= |
1233 | @@ -327,8 +326,9 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w |
1234 | golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= |
1235 | golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= |
1236 | golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= |
1237 | -golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= |
1238 | -golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= |
1239 | +golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= |
1240 | +golang.org/x/sys v0.19.0 h1:q5f1RH2jigJ1MoAWp2KTp3gm5zAGFUTarQZ5U386+4o= |
1241 | +golang.org/x/sys v0.19.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= |
1242 | golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= |
1243 | golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= |
1244 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= |
1245 | @@ -469,10 +469,8 @@ google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2 |
1246 | google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= |
1247 | google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= |
1248 | google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= |
1249 | -google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= |
1250 | -google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= |
1251 | -google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng= |
1252 | -google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= |
1253 | +google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI= |
1254 | +google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= |
1255 | gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= |
1256 | gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= |
1257 | gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= |
1258 | diff --git a/main_windows.go b/main_windows.go |
1259 | index 831069e..c65b4dc 100644 |
1260 | --- a/main_windows.go |
1261 | +++ b/main_windows.go |
1262 | @@ -25,6 +25,7 @@ import ( |
1263 | "unsafe" |
1264 | |
1265 | "github.com/GoogleCloudPlatform/guest-logging-go/logger" |
1266 | + "github.com/GoogleCloudPlatform/osconfig/agentconfig" |
1267 | "github.com/GoogleCloudPlatform/osconfig/packages" |
1268 | "golang.org/x/sys/windows" |
1269 | "golang.org/x/sys/windows/svc" |
1270 | @@ -76,7 +77,7 @@ func unlockFileEx(hFile uintptr, nNumberOfBytesToLockLow, nNumberOfBytesToLockHi |
1271 | } |
1272 | |
1273 | func obtainLock() { |
1274 | - lockFile := `C:\Program Files\Google\OSConfig\lock` |
1275 | + lockFile := filepath.Join(agentconfig.GetCacheDirWindows(), "lock") |
1276 | |
1277 | err := os.MkdirAll(filepath.Dir(lockFile), 0755) |
1278 | if err != nil && !os.IsExist(err) { |
1279 | diff --git a/packagebuild/README.md b/packagebuild/README.md |
1280 | new file mode 100644 |
1281 | index 0000000..7999450 |
1282 | --- /dev/null |
1283 | +++ b/packagebuild/README.md |
1284 | @@ -0,0 +1,75 @@ |
1285 | +# Package build workflows |
1286 | + |
1287 | +This directory contains the 'package builder', which is a set of [Daisy] |
1288 | +workflows and startup scripts. A package build workflow will accept a git |
1289 | +repository set up for package building, build it and produce a system package. |
1290 | + |
1291 | +We use these packagebuild workflows from our Concourse pipelines and Prow jobs. |
1292 | + |
1293 | +[Daisy]: https://github.com/GoogleCloudPlatform/compute-daisy |
1294 | + |
1295 | +## Repo layout for package building |
1296 | + |
1297 | +The package builder expects to check out a git repository that contains a |
1298 | +`packaging/` directory. This directory may contain one or more of: |
1299 | + |
1300 | +* 1 or more RPM spec files. |
1301 | +* A `debian/` directory, following Debian packaging standards |
1302 | +* A `googet/` directory, containing [GooGet] specs and scripts |
1303 | + |
1304 | +[GooGet]: https://github.com/google/googet |
1305 | + |
1306 | +## Workflow invocation |
1307 | + |
1308 | +Build workflows accept the following parameters: |
1309 | + |
1310 | +* gcs\_path - where to uploaded the resulting package, e.g. gs://my-bucket/ |
1311 | +* repo\_owner - the repo owner, for example 'GoogleCloudPlatform' |
1312 | +* repo\_name - the repo name, for example 'guest-agent' |
1313 | +* git\_ref - the git ref to check out, for example 'main' |
1314 | +* version - the version for the resulting package, for example '20221025.00' |
1315 | +* build\_dir - (optional) the subdirectory in the repo to `cd` into before |
1316 | + starting the build |
1317 | + |
1318 | +## Supported types |
1319 | + |
1320 | +Each packagebuild workflow corresponds to a single package type. All builds |
1321 | +default to the x86\_64 architecture unless otherwise specified. |
1322 | + |
1323 | +### Debian |
1324 | + |
1325 | +Debian workflows launch an instance of the specified Debian release and produce |
1326 | +.deb packages. |
1327 | + |
1328 | +Available workflows: |
1329 | + |
1330 | +* build\_deb9.wf.json |
1331 | +* build\_deb10.wf.json |
1332 | +* build\_deb11.wf.json |
1333 | + |
1334 | +An ARM64 workflow is also included: |
1335 | + |
1336 | +* build\_deb11\_arm64.wf.json |
1337 | + |
1338 | +### Enterprise Linux |
1339 | + |
1340 | +EL (Enterprise Linux) workflows launch an instance of a predefined EL type and |
1341 | +produce .rpm packages. The workflows use the OS that was best suited at the time |
1342 | +of development. |
1343 | + |
1344 | +Available workflows: |
1345 | + |
1346 | +* build\_el6.wf.json - uses CentOS 6 |
1347 | +* build\_el7.wf.json - uses CentOS 7 |
1348 | +* build\_el8.wf.json - uses RHEL 8 |
1349 | +* build\_el9.wf.json - uses CentOS Stream 9 |
1350 | + |
1351 | +ARM64 workflows are also included: |
1352 | + |
1353 | +* build\_el8\_arm64.wf.json - uses the GCP optimized version of Rocky Linux 8 |
1354 | +* build\_el9\_arm64.wf.json - uses RHEL 9 |
1355 | + |
1356 | +### GooGet |
1357 | + |
1358 | +The `build_goo.wf.json` (GooGet) workflow launches a Debian 10 instance and |
1359 | +produces .goo packages. |
1360 | diff --git a/packagebuild/common.sh b/packagebuild/common.sh |
1361 | new file mode 100644 |
1362 | index 0000000..c16e504 |
1363 | --- /dev/null |
1364 | +++ b/packagebuild/common.sh |
1365 | @@ -0,0 +1,84 @@ |
1366 | +#!/bin/bash |
1367 | +# Copyright 2019 Google Inc. All Rights Reserved. |
1368 | +# |
1369 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1370 | +# you may not use this file except in compliance with the License. |
1371 | +# You may obtain a copy of the License at |
1372 | +# |
1373 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1374 | +# |
1375 | +# Unless required by applicable law or agreed to in writing, software |
1376 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1377 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1378 | +# See the License for the specific language governing permissions and |
1379 | +# limitations under the License. |
1380 | + |
1381 | +function build_success() { |
1382 | + echo "Build succeeded: $@" |
1383 | + exit 0 |
1384 | +} |
1385 | + |
1386 | +function build_fail() { |
1387 | + echo "Build failed: $@" |
1388 | + exit 1 |
1389 | +} |
1390 | + |
1391 | +function exit_error() { |
1392 | + build_fail "$0:$1 \"$BASH_COMMAND\" returned $?" |
1393 | +} |
1394 | + |
1395 | +trap 'exit_error $LINENO' ERR |
1396 | + |
1397 | +function install_go() { |
1398 | + # Installs a specific version of go for compilation, since availability varies |
1399 | + # across linux distributions. Needs curl and tar to be installed. |
1400 | + local arch="amd64" |
1401 | + if [[ `uname -m` == "aarch64" ]]; then |
1402 | + arch="arm64" |
1403 | + fi |
1404 | + |
1405 | + local GOLANG="go1.22.2.linux-${arch}.tar.gz" |
1406 | + export GOPATH=/usr/share/gocode |
1407 | + export GOCACHE=/tmp/.cache |
1408 | + |
1409 | + # Golang setup |
1410 | + [[ -d /tmp/go ]] && rm -rf /tmp/go |
1411 | + mkdir -p /tmp/go/ |
1412 | + curl -s "https://dl.google.com/go/${GOLANG}" -o /tmp/go/go.tar.gz |
1413 | + tar -C /tmp/go/ --strip-components=1 -xf /tmp/go/go.tar.gz |
1414 | + |
1415 | + export PATH="/tmp/go/bin:${GOPATH}/bin:${PATH}" # set path for whoever invokes this function. |
1416 | + export GO=/tmp/go/bin/go # reference this go explicitly. |
1417 | +} |
1418 | + |
1419 | +function git_checkout() { |
1420 | + # Checks out a repo at a specified commit or ref into a specified directory. |
1421 | + |
1422 | + BASE_REPO="$1" |
1423 | + REPO="$2" |
1424 | + PULL_REF="$3" |
1425 | + |
1426 | + # pull the repository from github - start |
1427 | + mkdir -p $REPO |
1428 | + cd $REPO |
1429 | + git init |
1430 | + |
1431 | + # fetch only the branch that we want to build |
1432 | + git_command="git fetch https://github.com/${BASE_REPO}/${REPO}.git ${PULL_REF:-"master"}" |
1433 | + echo "Running ${git_command}" |
1434 | + $git_command |
1435 | + |
1436 | + git checkout FETCH_HEAD |
1437 | +} |
1438 | + |
1439 | +function try_command() { |
1440 | + n=0 |
1441 | + while ! "$@"; do |
1442 | + echo "try $n to run $@" |
1443 | + if [[ n -gt 3 ]]; then |
1444 | + return 1 |
1445 | + fi |
1446 | + ((n++)) |
1447 | + sleep 5 |
1448 | + done |
1449 | +} |
1450 | diff --git a/packagebuild/daisy_startupscript_deb.sh b/packagebuild/daisy_startupscript_deb.sh |
1451 | new file mode 100644 |
1452 | index 0000000..449c665 |
1453 | --- /dev/null |
1454 | +++ b/packagebuild/daisy_startupscript_deb.sh |
1455 | @@ -0,0 +1,108 @@ |
1456 | +#!/bin/bash |
1457 | +# Copyright 2019 Google Inc. All Rights Reserved. |
1458 | +# |
1459 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1460 | +# you may not use this file except in compliance with the License. |
1461 | +# You may obtain a copy of the License at |
1462 | +# |
1463 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1464 | +# |
1465 | +# Unless required by applicable law or agreed to in writing, software |
1466 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1467 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1468 | +# See the License for the specific language governing permissions and |
1469 | +# limitations under the License. |
1470 | + |
1471 | +URL="http://169.254.169.254/computeMetadata/v1/instance/attributes" |
1472 | +GCS_PATH=$(curl -f -H Metadata-Flavor:Google ${URL}/daisy-outs-path) |
1473 | +SRC_PATH=$(curl -f -H Metadata-Flavor:Google ${URL}/daisy-sources-path) |
1474 | +REPO_OWNER=$(curl -f -H Metadata-Flavor:Google ${URL}/repo-owner) |
1475 | +REPO_NAME=$(curl -f -H Metadata-Flavor:Google ${URL}/repo-name) |
1476 | +GIT_REF=$(curl -f -H Metadata-Flavor:Google ${URL}/git-ref) |
1477 | +BUILD_DIR=$(curl -f -H Metadata-Flavor:Google ${URL}/build-dir) |
1478 | +VERSION=$(curl -f -H Metadata-Flavor:Google ${URL}/version) |
1479 | +VERSION=${VERSION:="1dummy"} |
1480 | + |
1481 | +DEBIAN_FRONTEND=noninteractive |
1482 | + |
1483 | +echo "Started build..." |
1484 | + |
1485 | +gsutil cp "${SRC_PATH}/common.sh" ./ |
1486 | +. common.sh |
1487 | + |
1488 | +# disable the backports repo for debian-10 |
1489 | +sed -i 's/^.*debian buster-backports main.*$//g' /etc/apt/sources.list |
1490 | + |
1491 | +try_command apt-get -y update |
1492 | +try_command apt-get install -y --no-install-{suggests,recommends} git-core \ |
1493 | + debhelper devscripts build-essential equivs libdistro-info-perl |
1494 | + |
1495 | +git_checkout "$REPO_OWNER" "$REPO_NAME" "$GIT_REF" |
1496 | + |
1497 | +if [[ -n "$BUILD_DIR" ]]; then |
1498 | + cd "$BUILD_DIR" |
1499 | +fi |
1500 | + |
1501 | +PKGNAME="$(grep "^Package:" ./packaging/debian/control|cut -d' ' -f2-)" |
1502 | + |
1503 | +# Install build deps |
1504 | +mk-build-deps -t "apt-get -o Debug::pkgProblemResolver=yes \ |
1505 | + --no-install-recommends --yes" --install packaging/debian/control |
1506 | +dpkg-checkbuilddeps packaging/debian/control |
1507 | + |
1508 | +if grep -q '+deb' packaging/debian/changelog; then |
1509 | + DEB=$(</etc/debian_version) |
1510 | + # Currently Debian 11 doesn't use a numerical version number in its release. |
1511 | + if [[ "${DEB}" =~ "bullseye" ]]; then |
1512 | + DEB="11" |
1513 | + elif [[ "${DEB}" =~ "bookworm" ]]; then |
1514 | + DEB="12" |
1515 | + fi |
1516 | + DEB="+deb${DEB/.*}" |
1517 | +fi |
1518 | + |
1519 | +if grep -q 'golang' packaging/debian/control; then |
1520 | + echo "Installing go" |
1521 | + install_go |
1522 | + |
1523 | + echo "Installing go dependencies" |
1524 | + $GO mod download |
1525 | +fi |
1526 | + |
1527 | +COMMITURL="https://github.com/$REPO_OWNER/$REPO_NAME/tree/$(git rev-parse HEAD)" |
1528 | + |
1529 | +echo "Building package(s)" |
1530 | + |
1531 | +# Create build dir. |
1532 | +BUILD_DIR="/tmp/debpackage" |
1533 | +[[ -d $BUILD_DIR ]] || mkdir $BUILD_DIR |
1534 | + |
1535 | +# Create 'upstream' tarball. |
1536 | +TAR="${PKGNAME}_${VERSION}.orig.tar.gz" |
1537 | +tar czvf "${BUILD_DIR}/${TAR}" --exclude .git --exclude packaging \ |
1538 | + --transform "s/^\./${PKGNAME}-${VERSION}/" . |
1539 | + |
1540 | +# Extract tarball and build. |
1541 | +tar -C "$BUILD_DIR" -xzvf "${BUILD_DIR}/${TAR}" |
1542 | +PKGDIR="${BUILD_DIR}/${PKGNAME}-${VERSION}" |
1543 | +cp -r packaging/debian "${BUILD_DIR}/${PKGNAME}-${VERSION}/" |
1544 | + |
1545 | +cd "${BUILD_DIR}/${PKGNAME}-${VERSION}" |
1546 | + |
1547 | +sed -i"" "/^Source:/aXB-Git: ${COMMITURL}" debian/control |
1548 | + |
1549 | +# We generate this to enable auto-versioning. |
1550 | +[[ -f debian/changelog ]] && rm debian/changelog |
1551 | +RELEASE="g1${DEB}" |
1552 | +dch --create -M -v 1:${VERSION}-${RELEASE} --package $PKGNAME -D stable \ |
1553 | + "Debian packaging for ${PKGNAME}" |
1554 | +DEB_BUILD_OPTIONS="noautodbgsym nocheck" debuild -e "VERSION=${VERSION}" -e "RELEASE=${RELEASE}" -us -uc |
1555 | + |
1556 | +for deb in $BUILD_DIR/*.deb; do |
1557 | + dpkg-deb -I $deb |
1558 | + dpkg-deb -c $deb |
1559 | +done |
1560 | + |
1561 | +echo "copying $BUILD_DIR/*.deb to $GCS_PATH/" |
1562 | +gsutil cp -n $BUILD_DIR/*.deb "$GCS_PATH/" |
1563 | +build_success Built $BUILD_DIR/*.deb |
1564 | diff --git a/packagebuild/daisy_startupscript_goo.sh b/packagebuild/daisy_startupscript_goo.sh |
1565 | new file mode 100644 |
1566 | index 0000000..1fe2583 |
1567 | --- /dev/null |
1568 | +++ b/packagebuild/daisy_startupscript_goo.sh |
1569 | @@ -0,0 +1,61 @@ |
1570 | +#!/bin/bash |
1571 | +# Copyright 2019 Google Inc. All Rights Reserved. |
1572 | +# |
1573 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1574 | +# you may not use this file except in compliance with the License. |
1575 | +# You may obtain a copy of the License at |
1576 | +# |
1577 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1578 | +# |
1579 | +# Unless required by applicable law or agreed to in writing, software |
1580 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1581 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1582 | +# See the License for the specific language governing permissions and |
1583 | +# limitations under the License. |
1584 | + |
1585 | +URL="http://metadata/computeMetadata/v1/instance/attributes" |
1586 | +GCS_PATH=$(curl -f -H Metadata-Flavor:Google ${URL}/daisy-outs-path) |
1587 | +SRC_PATH=$(curl -f -H Metadata-Flavor:Google ${URL}/daisy-sources-path) |
1588 | +REPO_OWNER=$(curl -f -H Metadata-Flavor:Google ${URL}/repo-owner) |
1589 | +REPO_NAME=$(curl -f -H Metadata-Flavor:Google ${URL}/repo-name) |
1590 | +GIT_REF=$(curl -f -H Metadata-Flavor:Google ${URL}/git-ref) |
1591 | +BUILD_DIR=$(curl -f -H Metadata-Flavor:Google ${URL}/build-dir) |
1592 | +VERSION=$(curl -f -H Metadata-Flavor:Google ${URL}/version) |
1593 | + |
1594 | +echo "Started build..." |
1595 | + |
1596 | +gsutil cp "${SRC_PATH}/common.sh" ./ |
1597 | +. common.sh |
1598 | + |
1599 | +# disable the backports repo for debian-10 |
1600 | +sed -i 's/^.*debian buster-backports main.*$//g' /etc/apt/sources.list |
1601 | + |
1602 | +try_command apt-get -y update |
1603 | +try_command apt-get install -y --no-install-{suggests,recommends} git-core |
1604 | + |
1605 | +# We always install go, needed for goopack. |
1606 | +echo "Installing go" |
1607 | +install_go |
1608 | + |
1609 | +# Install goopack. |
1610 | +GO111MODULE=on $GO install -v github.com/google/googet/v2/goopack@latest |
1611 | + |
1612 | +git_checkout "$REPO_OWNER" "$REPO_NAME" "$GIT_REF" |
1613 | +if [[ -n "$BUILD_DIR" ]]; then |
1614 | + cd "$BUILD_DIR" |
1615 | +fi |
1616 | + |
1617 | +if find . -type f -iname '*.go' >/dev/null; then |
1618 | + echo "Installing go dependencies" |
1619 | + $GO mod download |
1620 | +fi |
1621 | + |
1622 | +echo "Building package(s)" |
1623 | +for spec in packaging/googet/*.goospec; do |
1624 | + goopack -var:version="$VERSION" "$spec" |
1625 | + name=$(basename "${spec}") |
1626 | + pref=${name%.*} |
1627 | +done |
1628 | + |
1629 | +gsutil cp -n *.goo "$GCS_PATH/" |
1630 | +build_success "Built `ls *.goo|xargs`" |
1631 | diff --git a/packagebuild/daisy_startupscript_rpm.sh b/packagebuild/daisy_startupscript_rpm.sh |
1632 | new file mode 100644 |
1633 | index 0000000..804fab8 |
1634 | --- /dev/null |
1635 | +++ b/packagebuild/daisy_startupscript_rpm.sh |
1636 | @@ -0,0 +1,147 @@ |
1637 | +#!/bin/bash |
1638 | +# Copyright 2019 Google Inc. All Rights Reserved. |
1639 | +# |
1640 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
1641 | +# you may not use this file except in compliance with the License. |
1642 | +# You may obtain a copy of the License at |
1643 | +# |
1644 | +# http://www.apache.org/licenses/LICENSE-2.0 |
1645 | +# |
1646 | +# Unless required by applicable law or agreed to in writing, software |
1647 | +# distributed under the License is distributed on an "AS IS" BASIS, |
1648 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
1649 | +# See the License for the specific language governing permissions and |
1650 | +# limitations under the License. |
1651 | + |
1652 | +# This script is expected to be run on an Enterprise Linux system (RHEL, CentOS) |
1653 | +# on GCE with various flags set in the instance metadata including the git |
1654 | +# repository to clone. The script produces an RPM defined by an RPM spec in the |
1655 | +# packaging/ directory from the cloned repo. |
1656 | + |
1657 | +URL="http://metadata/computeMetadata/v1/instance/attributes" |
1658 | +function get_md() { |
1659 | + curl -f -H Metadata-Flavor:Google "${URL}/${1}" |
1660 | +} |
1661 | +GCS_PATH=$(get_md daisy-outs-path) |
1662 | +SRC_PATH=$(get_md daisy-sources-path) |
1663 | +REPO_OWNER=$(get_md repo-owner) |
1664 | +REPO_NAME=$(get_md repo-name) |
1665 | +GIT_REF=$(get_md git-ref) |
1666 | +BUILD_DIR=$(get_md build-dir) |
1667 | +VERSION=$(get_md version) |
1668 | +VERSION=${VERSION:-"dummy"} |
1669 | + |
1670 | +echo "Started build..." |
1671 | + |
1672 | +# common.sh contains functions common to all builds. |
1673 | +gsutil cp "${SRC_PATH}/common.sh" ./ |
1674 | +. common.sh |
1675 | + |
1676 | +# Install git2 as this is not available in centos 6/7 |
1677 | +VERSION_ID=6 |
1678 | +if [[ -f /etc/os-release ]]; then |
1679 | + eval $(grep VERSION_ID /etc/os-release) |
1680 | + VERSION_ID=${VERSION_ID:0:1} |
1681 | +fi |
1682 | + |
1683 | +GIT="git" |
1684 | +if [[ ${VERSION_ID} =~ 6|7 ]]; then |
1685 | + try_command yum install -y "https://repo.ius.io/ius-release-el${VERSION_ID}.rpm" |
1686 | + rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-IUS-${VERSION_ID} |
1687 | + GIT="git236" |
1688 | +fi |
1689 | + |
1690 | +# Install DevToolSet with gcc 10 for EL7. |
1691 | +# Centos 7 has only gcc 4.8.5 available. |
1692 | +if (( ${VERSION_ID} == 7 )); then |
1693 | + try_command yum install -y centos-release-scl |
1694 | + try_command yum install -y devtoolset-10-gcc-c++.x86_64 |
1695 | +fi |
1696 | + |
1697 | +# Enable CRB repo on EL9. |
1698 | +if [[ ${VERSION_ID} = 9 ]]; then |
1699 | + eval $(grep ID /etc/os-release) |
1700 | + # RHEL has a different CRB repo than Rocky/CentOS. |
1701 | + if [[ ${ID} == "rhel" ]]; then |
1702 | + dnf config-manager --set-enabled rhui-codeready-builder-for-rhel-9-$(uname -m)-rhui-rpms |
1703 | + else |
1704 | + dnf config-manager --set-enabled crb |
1705 | + fi |
1706 | +fi |
1707 | + |
1708 | +try_command yum install -y $GIT rpmdevtools yum-utils python3-devel |
1709 | + |
1710 | +git_checkout "$REPO_OWNER" "$REPO_NAME" "$GIT_REF" |
1711 | + |
1712 | +if [[ -n "$BUILD_DIR" ]]; then |
1713 | + cd "$BUILD_DIR" |
1714 | +fi |
1715 | + |
1716 | +if grep -q '%{_go}' ./packaging/*.spec; then |
1717 | + echo "Installing go" |
1718 | + install_go |
1719 | + |
1720 | + echo "Installing go dependencies" |
1721 | + $GO mod download |
1722 | +fi |
1723 | + |
1724 | +# Make build dirs as needed. |
1725 | +RPMDIR=/usr/src/redhat |
1726 | +for dir in ${RPMDIR}/{SOURCES,SPECS}; do |
1727 | + [[ -d "$dir" ]] || mkdir -p "$dir" |
1728 | +done |
1729 | + |
1730 | +# Find the RPM specs to build for this version. |
1731 | +TOBUILD="" |
1732 | +SPECS=$(ls ./packaging/*.spec | sed -re 's/(\.el.)?.spec//' | sort -ru) |
1733 | +echo "all specs $SPECS" |
1734 | +for spec in $SPECS; do |
1735 | + spec=$(basename "$spec") |
1736 | + distspec="${spec}.el${VERSION_ID}.spec" |
1737 | + echo checking $spec and $distspec |
1738 | + if [[ -f "./packaging/$distspec" ]]; then |
1739 | + TOBUILD="${TOBUILD} ${distspec}" |
1740 | + else |
1741 | + TOBUILD="${TOBUILD} ${spec}.spec" |
1742 | + fi |
1743 | +done |
1744 | + |
1745 | +[[ -z "$TOBUILD" ]] && build_fail "No RPM specs found" |
1746 | + |
1747 | +COMMITURL="https://github.com/$REPO_OWNER/$REPO_NAME/tree/$(git rev-parse HEAD)" |
1748 | + |
1749 | +echo "Building package(s)" |
1750 | + |
1751 | +# Enable gcc 10 for EL7 only and before rpmbuild |
1752 | +if (( ${VERSION_ID} == 7 )); then |
1753 | + source /opt/rh/devtoolset-10/enable |
1754 | +fi |
1755 | + |
1756 | +for spec in $TOBUILD; do |
1757 | + PKGNAME="$(grep Name: "./packaging/${spec}"|cut -d' ' -f2-|tr -d ' ')" |
1758 | + yum-builddep -y "./packaging/${spec}" |
1759 | + |
1760 | + cp "./packaging/${spec}" "${RPMDIR}/SPECS/" |
1761 | + cp ./packaging/*.tar.gz "${RPMDIR}/SOURCES/" || : |
1762 | + cp ./packaging/*.patch "${RPMDIR}/SOURCES/" || : |
1763 | + |
1764 | + sed -i"" "/^Version/aVcs: ${COMMITURL}" "${RPMDIR}/SPECS/${spec}" |
1765 | + |
1766 | + tar czvf "${RPMDIR}/SOURCES/${PKGNAME}_${VERSION}.orig.tar.gz" \ |
1767 | + --exclude .git --exclude packaging \ |
1768 | + --transform "s/^\./${PKGNAME}-${VERSION}/" . |
1769 | + |
1770 | + rpmbuild --define "_topdir ${RPMDIR}/" --define "_version ${VERSION}" \ |
1771 | + --define "_go ${GO:-"UNSET"}" --define "_gopath ${GOPATH:-"UNSET"}" \ |
1772 | + -ba "${RPMDIR}/SPECS/${spec}" |
1773 | + |
1774 | + SRPM_FILE=$(find ${RPMDIR}/SRPMS -iname "${PKGNAME}*.src.rpm") |
1775 | +done |
1776 | + |
1777 | +rpms=$(find ${RPMDIR}/{S,}RPMS -iname "${PKGNAME}*.rpm") |
1778 | +for rpm in $rpms; do |
1779 | + rpm -qpilR $rpm |
1780 | +done |
1781 | +echo "copying ${rpms} to $GCS_PATH/" |
1782 | +gsutil cp -n ${rpms} "$GCS_PATH/" |
1783 | +build_success "Built $(echo ${rpms}|xargs)" |
1784 | diff --git a/packagebuild/test_build.sh b/packagebuild/test_build.sh |
1785 | new file mode 100755 |
1786 | index 0000000..0ee7432 |
1787 | --- /dev/null |
1788 | +++ b/packagebuild/test_build.sh |
1789 | @@ -0,0 +1,69 @@ |
1790 | +#!/bin/bash |
1791 | +# |
1792 | +# Produce a single package build with a custom version. |
1793 | +# |
1794 | +# - Package TYPEs can be found in the 'workflows/' directory as |
1795 | +# a filename pattern build_{{type}}.wf.json |
1796 | +# - Package Version should start with a digit |
1797 | +# |
1798 | +# Example (requires daisy tool somewhere in the PATH): |
1799 | +# build rpm package for el9 like distributions and version 0.0.1 for |
1800 | +# the last commit in the 'trustca-sync' branch of vorakl/guest-oslogin |
1801 | +# fork on Github and save results to gs://vorakl-dev-builds/packages/ bucket. |
1802 | +# |
1803 | +# env TYPE=el9 \ |
1804 | +# PROJECT=vorakl-dev \ |
1805 | +# ZONE=us-west1-a \ |
1806 | +# OWNER=vorakl \ |
1807 | +# REPO=guest-oslogin \ |
1808 | +# GIT_REF=trustca-sync \ |
1809 | +# GCS_PATH=gs://vorakl-dev-builds/packages \ |
1810 | +# VERSION=0.01 \ |
1811 | +# BUILD_DIR=. \ |
1812 | +# ./test_build.sh |
1813 | + |
1814 | +DEFAULT_TYPE='deb11' |
1815 | +DEFAULT_PROJECT='gcp-guest' |
1816 | +DEFAULT_ZONE='us-west1-a' |
1817 | +DEFAULT_OWNER='GoogleCloudPlatform' |
1818 | +DEFAULT_GIT_REF='master' |
1819 | +DEFAULT_GCS_PATH='${SCRATCHPATH}/packages' |
1820 | +DEFAULT_BUILD_DIR='.' |
1821 | +DEFAULT_VERSION='1dummy' |
1822 | + |
1823 | +[[ -z "${TYPE}" ]] && read -p "Build type [${DEFAULT_TYPE}]: " TYPE |
1824 | +[[ -z "${PROJECT}" ]] && read -p "Build project [${DEFAULT_PROJECT}]: " PROJECT |
1825 | +[[ -z "${ZONE}" ]] && read -p "Build zone [${DEFAULT_ZONE}]: " ZONE |
1826 | +[[ -z "${OWNER}" ]] && read -p "Repo owner or org [${DEFAULT_OWNER}]: " OWNER |
1827 | +[[ -z "${GIT_REF}" ]] && read -p "Ref [${DEFAULT_GIT_REF}]: " GIT_REF |
1828 | +[[ -z "${GCS_PATH}" ]] && read -p "GCS Path to upload to [${DEFAULT_GCS_PATH}]: " GCS_PATH |
1829 | +[[ -z "${BUILD_DIR}" ]] && read -p "Directory to build from [${DEFAULT_BUILD_DIR}]: " BUILD_DIR |
1830 | +[[ -z "${REPO}" ]] && read -p "Repo name: " REPO |
1831 | + |
1832 | +[[ -z "${TYPE}" ]] && TYPE=${DEFAULT_TYPE} |
1833 | +[[ -z "${PROJECT}" ]] && PROJECT=${DEFAULT_PROJECT} |
1834 | +[[ -z "${ZONE}" ]] && ZONE=${DEFAULT_ZONE} |
1835 | +[[ -z "${OWNER}" ]] && OWNER=${DEFAULT_OWNER} |
1836 | +[[ -z "${GIT_REF}" ]] && GIT_REF=${DEFAULT_GIT_REF} |
1837 | +[[ -z "${GCS_PATH}" ]] && GCS_PATH=${DEFAULT_GCS_PATH} |
1838 | +[[ -z "${BUILD_DIR}" ]] && BUILD_DIR=${DEFAULT_BUILD_DIR} |
1839 | +[[ -z "${VERSION}" ]] && VERSION=${DEFAULT_VERSION} |
1840 | + |
1841 | +WF="workflows/build_${TYPE}.wf.json" |
1842 | + |
1843 | +if [[ ! -f "${WF}" ]]; then |
1844 | + echo "Unknown build type ${TYPE}" |
1845 | + exit 1 |
1846 | +fi |
1847 | + |
1848 | +set -x |
1849 | +daisy \ |
1850 | + -project ${PROJECT} \ |
1851 | + -zone ${ZONE} \ |
1852 | + -var:gcs_path=${GCS_PATH} \ |
1853 | + -var:repo_owner=${OWNER} \ |
1854 | + -var:repo_name=${REPO} \ |
1855 | + -var:git_ref=${GIT_REF} \ |
1856 | + -var:build_dir=${BUILD_DIR} \ |
1857 | + -var:version=${VERSION} \ |
1858 | + "${WF}" |
1859 | diff --git a/packagebuild/workflows/build_deb10.wf.json b/packagebuild/workflows/build_deb10.wf.json |
1860 | new file mode 100644 |
1861 | index 0000000..8976dcf |
1862 | --- /dev/null |
1863 | +++ b/packagebuild/workflows/build_deb10.wf.json |
1864 | @@ -0,0 +1,42 @@ |
1865 | +{ |
1866 | + "Name": "deb10", |
1867 | + "Timeout": "20m", |
1868 | + "Vars": { |
1869 | + "gcs_path": { |
1870 | + "Required": true |
1871 | + }, |
1872 | + "repo_owner": { |
1873 | + "Required": true |
1874 | + }, |
1875 | + "repo_name": { |
1876 | + "Required": true |
1877 | + }, |
1878 | + "git_ref": { |
1879 | + "Required": true |
1880 | + }, |
1881 | + "version": { |
1882 | + "Required": true |
1883 | + }, |
1884 | + "build_dir": { |
1885 | + "Required": true |
1886 | + } |
1887 | + }, |
1888 | + "Steps": { |
1889 | + "build-package": { |
1890 | + "Timeout": "20m", |
1891 | + "SubWorkflow": { |
1892 | + "Path": "./build_package.wf.json", |
1893 | + "Vars": { |
1894 | + "type": "deb", |
1895 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-10", |
1896 | + "gcs_path": "${gcs_path}", |
1897 | + "repo_owner": "${repo_owner}", |
1898 | + "repo_name": "${repo_name}", |
1899 | + "git_ref": "${git_ref}", |
1900 | + "build_dir": "${build_dir}", |
1901 | + "version": "${version}" |
1902 | + } |
1903 | + } |
1904 | + } |
1905 | + } |
1906 | +} |
1907 | diff --git a/packagebuild/workflows/build_deb11.wf.json b/packagebuild/workflows/build_deb11.wf.json |
1908 | new file mode 100644 |
1909 | index 0000000..53e4c79 |
1910 | --- /dev/null |
1911 | +++ b/packagebuild/workflows/build_deb11.wf.json |
1912 | @@ -0,0 +1,42 @@ |
1913 | +{ |
1914 | + "Name": "deb11", |
1915 | + "Timeout": "20m", |
1916 | + "Vars": { |
1917 | + "gcs_path": { |
1918 | + "Required": true |
1919 | + }, |
1920 | + "repo_owner": { |
1921 | + "Required": true |
1922 | + }, |
1923 | + "repo_name": { |
1924 | + "Required": true |
1925 | + }, |
1926 | + "git_ref": { |
1927 | + "Required": true |
1928 | + }, |
1929 | + "version": { |
1930 | + "Required": true |
1931 | + }, |
1932 | + "build_dir": { |
1933 | + "Required": true |
1934 | + } |
1935 | + }, |
1936 | + "Steps": { |
1937 | + "build-package": { |
1938 | + "Timeout": "20m", |
1939 | + "SubWorkflow": { |
1940 | + "Path": "./build_package.wf.json", |
1941 | + "Vars": { |
1942 | + "type": "deb", |
1943 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-11", |
1944 | + "gcs_path": "${gcs_path}", |
1945 | + "repo_owner": "${repo_owner}", |
1946 | + "repo_name": "${repo_name}", |
1947 | + "git_ref": "${git_ref}", |
1948 | + "build_dir": "${build_dir}", |
1949 | + "version": "${version}" |
1950 | + } |
1951 | + } |
1952 | + } |
1953 | + } |
1954 | +} |
1955 | diff --git a/packagebuild/workflows/build_deb11_arm64.wf.json b/packagebuild/workflows/build_deb11_arm64.wf.json |
1956 | new file mode 100644 |
1957 | index 0000000..8c0c04a |
1958 | --- /dev/null |
1959 | +++ b/packagebuild/workflows/build_deb11_arm64.wf.json |
1960 | @@ -0,0 +1,44 @@ |
1961 | +{ |
1962 | + "Name": "deb11", |
1963 | + "Timeout": "20m", |
1964 | + "Vars": { |
1965 | + "gcs_path": { |
1966 | + "Required": true |
1967 | + }, |
1968 | + "repo_owner": { |
1969 | + "Required": true |
1970 | + }, |
1971 | + "repo_name": { |
1972 | + "Required": true |
1973 | + }, |
1974 | + "git_ref": { |
1975 | + "Required": true |
1976 | + }, |
1977 | + "version": { |
1978 | + "Required": true |
1979 | + }, |
1980 | + "build_dir": { |
1981 | + "Required": true |
1982 | + } |
1983 | + }, |
1984 | + "Steps": { |
1985 | + "build-package": { |
1986 | + "Timeout": "20m", |
1987 | + "SubWorkflow": { |
1988 | + "Path": "./build_package.wf.json", |
1989 | + "Vars": { |
1990 | + "type": "deb", |
1991 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-11-arm64", |
1992 | + "gcs_path": "${gcs_path}", |
1993 | + "repo_owner": "${repo_owner}", |
1994 | + "repo_name": "${repo_name}", |
1995 | + "git_ref": "${git_ref}", |
1996 | + "build_dir": "${build_dir}", |
1997 | + "machine_type": "t2a-standard-2", |
1998 | + "zone": "us-central1-a", |
1999 | + "version": "${version}" |
2000 | + } |
2001 | + } |
2002 | + } |
2003 | + } |
2004 | +} |
2005 | diff --git a/packagebuild/workflows/build_deb12.wf.json b/packagebuild/workflows/build_deb12.wf.json |
2006 | new file mode 100644 |
2007 | index 0000000..9b90b92 |
2008 | --- /dev/null |
2009 | +++ b/packagebuild/workflows/build_deb12.wf.json |
2010 | @@ -0,0 +1,42 @@ |
2011 | +{ |
2012 | + "Name": "deb12", |
2013 | + "Timeout": "20m", |
2014 | + "Vars": { |
2015 | + "gcs_path": { |
2016 | + "Required": true |
2017 | + }, |
2018 | + "repo_owner": { |
2019 | + "Required": true |
2020 | + }, |
2021 | + "repo_name": { |
2022 | + "Required": true |
2023 | + }, |
2024 | + "git_ref": { |
2025 | + "Required": true |
2026 | + }, |
2027 | + "version": { |
2028 | + "Required": true |
2029 | + }, |
2030 | + "build_dir": { |
2031 | + "Required": true |
2032 | + } |
2033 | + }, |
2034 | + "Steps": { |
2035 | + "build-package": { |
2036 | + "Timeout": "20m", |
2037 | + "SubWorkflow": { |
2038 | + "Path": "./build_package.wf.json", |
2039 | + "Vars": { |
2040 | + "type": "deb", |
2041 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-12", |
2042 | + "gcs_path": "${gcs_path}", |
2043 | + "repo_owner": "${repo_owner}", |
2044 | + "repo_name": "${repo_name}", |
2045 | + "git_ref": "${git_ref}", |
2046 | + "build_dir": "${build_dir}", |
2047 | + "version": "${version}" |
2048 | + } |
2049 | + } |
2050 | + } |
2051 | + } |
2052 | +} |
2053 | diff --git a/packagebuild/workflows/build_deb12_arm64.wf.json b/packagebuild/workflows/build_deb12_arm64.wf.json |
2054 | new file mode 100644 |
2055 | index 0000000..8f6f68f |
2056 | --- /dev/null |
2057 | +++ b/packagebuild/workflows/build_deb12_arm64.wf.json |
2058 | @@ -0,0 +1,44 @@ |
2059 | +{ |
2060 | + "Name": "deb12", |
2061 | + "Timeout": "20m", |
2062 | + "Vars": { |
2063 | + "gcs_path": { |
2064 | + "Required": true |
2065 | + }, |
2066 | + "repo_owner": { |
2067 | + "Required": true |
2068 | + }, |
2069 | + "repo_name": { |
2070 | + "Required": true |
2071 | + }, |
2072 | + "git_ref": { |
2073 | + "Required": true |
2074 | + }, |
2075 | + "version": { |
2076 | + "Required": true |
2077 | + }, |
2078 | + "build_dir": { |
2079 | + "Required": true |
2080 | + } |
2081 | + }, |
2082 | + "Steps": { |
2083 | + "build-package": { |
2084 | + "Timeout": "20m", |
2085 | + "SubWorkflow": { |
2086 | + "Path": "./build_package.wf.json", |
2087 | + "Vars": { |
2088 | + "type": "deb", |
2089 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-12-arm64", |
2090 | + "gcs_path": "${gcs_path}", |
2091 | + "repo_owner": "${repo_owner}", |
2092 | + "repo_name": "${repo_name}", |
2093 | + "git_ref": "${git_ref}", |
2094 | + "build_dir": "${build_dir}", |
2095 | + "machine_type": "t2a-standard-2", |
2096 | + "zone": "us-central1-a", |
2097 | + "version": "${version}" |
2098 | + } |
2099 | + } |
2100 | + } |
2101 | + } |
2102 | +} |
2103 | diff --git a/packagebuild/workflows/build_el6.wf.json b/packagebuild/workflows/build_el6.wf.json |
2104 | new file mode 100644 |
2105 | index 0000000..fbbc600 |
2106 | --- /dev/null |
2107 | +++ b/packagebuild/workflows/build_el6.wf.json |
2108 | @@ -0,0 +1,42 @@ |
2109 | +{ |
2110 | + "Name": "el6", |
2111 | + "Timeout": "20m", |
2112 | + "Vars": { |
2113 | + "gcs_path": { |
2114 | + "Required": true |
2115 | + }, |
2116 | + "repo_owner": { |
2117 | + "Required": true |
2118 | + }, |
2119 | + "repo_name": { |
2120 | + "Required": true |
2121 | + }, |
2122 | + "git_ref": { |
2123 | + "Required": true |
2124 | + }, |
2125 | + "version": { |
2126 | + "Required": true |
2127 | + }, |
2128 | + "build_dir": { |
2129 | + "Required": true |
2130 | + } |
2131 | + }, |
2132 | + "Steps": { |
2133 | + "build-package": { |
2134 | + "Timeout": "20m", |
2135 | + "SubWorkflow": { |
2136 | + "Path": "./build_package.wf.json", |
2137 | + "Vars": { |
2138 | + "type": "rpm", |
2139 | + "sourceImage": "projects/centos-cloud/global/images/family/centos-6", |
2140 | + "gcs_path": "${gcs_path}", |
2141 | + "repo_owner": "${repo_owner}", |
2142 | + "repo_name": "${repo_name}", |
2143 | + "git_ref": "${git_ref}", |
2144 | + "build_dir": "${build_dir}", |
2145 | + "version": "${version}" |
2146 | + } |
2147 | + } |
2148 | + } |
2149 | + } |
2150 | +} |
2151 | diff --git a/packagebuild/workflows/build_el7.wf.json b/packagebuild/workflows/build_el7.wf.json |
2152 | new file mode 100644 |
2153 | index 0000000..ce75870 |
2154 | --- /dev/null |
2155 | +++ b/packagebuild/workflows/build_el7.wf.json |
2156 | @@ -0,0 +1,42 @@ |
2157 | +{ |
2158 | + "Name": "el7", |
2159 | + "Timeout": "20m", |
2160 | + "Vars": { |
2161 | + "gcs_path": { |
2162 | + "Required": true |
2163 | + }, |
2164 | + "repo_owner": { |
2165 | + "Required": true |
2166 | + }, |
2167 | + "repo_name": { |
2168 | + "Required": true |
2169 | + }, |
2170 | + "git_ref": { |
2171 | + "Required": true |
2172 | + }, |
2173 | + "version": { |
2174 | + "Required": true |
2175 | + }, |
2176 | + "build_dir": { |
2177 | + "Required": true |
2178 | + } |
2179 | + }, |
2180 | + "Steps": { |
2181 | + "build-package": { |
2182 | + "Timeout": "20m", |
2183 | + "SubWorkflow": { |
2184 | + "Path": "./build_package.wf.json", |
2185 | + "Vars": { |
2186 | + "type": "rpm", |
2187 | + "sourceImage": "projects/centos-cloud/global/images/family/centos-7", |
2188 | + "gcs_path": "${gcs_path}", |
2189 | + "repo_owner": "${repo_owner}", |
2190 | + "repo_name": "${repo_name}", |
2191 | + "git_ref": "${git_ref}", |
2192 | + "build_dir": "${build_dir}", |
2193 | + "version": "${version}" |
2194 | + } |
2195 | + } |
2196 | + } |
2197 | + } |
2198 | +} |
2199 | diff --git a/packagebuild/workflows/build_el8.wf.json b/packagebuild/workflows/build_el8.wf.json |
2200 | new file mode 100644 |
2201 | index 0000000..61dba50 |
2202 | --- /dev/null |
2203 | +++ b/packagebuild/workflows/build_el8.wf.json |
2204 | @@ -0,0 +1,42 @@ |
2205 | +{ |
2206 | + "Name": "el8", |
2207 | + "Timeout": "20m", |
2208 | + "Vars": { |
2209 | + "gcs_path": { |
2210 | + "Required": true |
2211 | + }, |
2212 | + "repo_owner": { |
2213 | + "Required": true |
2214 | + }, |
2215 | + "repo_name": { |
2216 | + "Required": true |
2217 | + }, |
2218 | + "git_ref": { |
2219 | + "Required": true |
2220 | + }, |
2221 | + "version": { |
2222 | + "Required": true |
2223 | + }, |
2224 | + "build_dir": { |
2225 | + "Required": true |
2226 | + } |
2227 | + }, |
2228 | + "Steps": { |
2229 | + "build-package": { |
2230 | + "Timeout": "20m", |
2231 | + "SubWorkflow": { |
2232 | + "Path": "./build_package.wf.json", |
2233 | + "Vars": { |
2234 | + "type": "rpm", |
2235 | + "sourceImage": "projects/rhel-cloud/global/images/family/rhel-8", |
2236 | + "gcs_path": "${gcs_path}", |
2237 | + "repo_owner": "${repo_owner}", |
2238 | + "repo_name": "${repo_name}", |
2239 | + "git_ref": "${git_ref}", |
2240 | + "build_dir": "${build_dir}", |
2241 | + "version": "${version}" |
2242 | + } |
2243 | + } |
2244 | + } |
2245 | + } |
2246 | +} |
2247 | diff --git a/packagebuild/workflows/build_el8_arm64.wf.json b/packagebuild/workflows/build_el8_arm64.wf.json |
2248 | new file mode 100644 |
2249 | index 0000000..f97ed8e |
2250 | --- /dev/null |
2251 | +++ b/packagebuild/workflows/build_el8_arm64.wf.json |
2252 | @@ -0,0 +1,44 @@ |
2253 | +{ |
2254 | + "Name": "el8", |
2255 | + "Timeout": "20m", |
2256 | + "Vars": { |
2257 | + "gcs_path": { |
2258 | + "Required": true |
2259 | + }, |
2260 | + "repo_owner": { |
2261 | + "Required": true |
2262 | + }, |
2263 | + "repo_name": { |
2264 | + "Required": true |
2265 | + }, |
2266 | + "git_ref": { |
2267 | + "Required": true |
2268 | + }, |
2269 | + "version": { |
2270 | + "Required": true |
2271 | + }, |
2272 | + "build_dir": { |
2273 | + "Required": true |
2274 | + } |
2275 | + }, |
2276 | + "Steps": { |
2277 | + "build-package": { |
2278 | + "Timeout": "20m", |
2279 | + "SubWorkflow": { |
2280 | + "Path": "./build_package.wf.json", |
2281 | + "Vars": { |
2282 | + "type": "rpm", |
2283 | + "sourceImage": "projects/rocky-linux-cloud/global/images/family/rocky-linux-8-optimized-gcp-arm64", |
2284 | + "gcs_path": "${gcs_path}", |
2285 | + "repo_owner": "${repo_owner}", |
2286 | + "repo_name": "${repo_name}", |
2287 | + "git_ref": "${git_ref}", |
2288 | + "build_dir": "${build_dir}", |
2289 | + "machine_type": "t2a-standard-2", |
2290 | + "zone": "us-central1-a", |
2291 | + "version": "${version}" |
2292 | + } |
2293 | + } |
2294 | + } |
2295 | + } |
2296 | +} |
2297 | diff --git a/packagebuild/workflows/build_el9.wf.json b/packagebuild/workflows/build_el9.wf.json |
2298 | new file mode 100644 |
2299 | index 0000000..682297e |
2300 | --- /dev/null |
2301 | +++ b/packagebuild/workflows/build_el9.wf.json |
2302 | @@ -0,0 +1,42 @@ |
2303 | +{ |
2304 | + "Name": "el9", |
2305 | + "Timeout": "40m", |
2306 | + "Vars": { |
2307 | + "gcs_path": { |
2308 | + "Required": true |
2309 | + }, |
2310 | + "repo_owner": { |
2311 | + "Required": true |
2312 | + }, |
2313 | + "repo_name": { |
2314 | + "Required": true |
2315 | + }, |
2316 | + "git_ref": { |
2317 | + "Required": true |
2318 | + }, |
2319 | + "version": { |
2320 | + "Required": true |
2321 | + }, |
2322 | + "build_dir": { |
2323 | + "Required": true |
2324 | + } |
2325 | + }, |
2326 | + "Steps": { |
2327 | + "build-package": { |
2328 | + "Timeout": "40m", |
2329 | + "SubWorkflow": { |
2330 | + "Path": "./build_package.wf.json", |
2331 | + "Vars": { |
2332 | + "type": "rpm", |
2333 | + "sourceImage": "projects/rhel-cloud/global/images/family/rhel-9", |
2334 | + "gcs_path": "${gcs_path}", |
2335 | + "repo_owner": "${repo_owner}", |
2336 | + "repo_name": "${repo_name}", |
2337 | + "git_ref": "${git_ref}", |
2338 | + "build_dir": "${build_dir}", |
2339 | + "version": "${version}" |
2340 | + } |
2341 | + } |
2342 | + } |
2343 | + } |
2344 | +} |
2345 | diff --git a/packagebuild/workflows/build_el9_arm64.wf.json b/packagebuild/workflows/build_el9_arm64.wf.json |
2346 | new file mode 100644 |
2347 | index 0000000..5c41c89 |
2348 | --- /dev/null |
2349 | +++ b/packagebuild/workflows/build_el9_arm64.wf.json |
2350 | @@ -0,0 +1,44 @@ |
2351 | +{ |
2352 | + "Name": "el9", |
2353 | + "Timeout": "20m", |
2354 | + "Vars": { |
2355 | + "gcs_path": { |
2356 | + "Required": true |
2357 | + }, |
2358 | + "repo_owner": { |
2359 | + "Required": true |
2360 | + }, |
2361 | + "repo_name": { |
2362 | + "Required": true |
2363 | + }, |
2364 | + "git_ref": { |
2365 | + "Required": true |
2366 | + }, |
2367 | + "version": { |
2368 | + "Required": true |
2369 | + }, |
2370 | + "build_dir": { |
2371 | + "Required": true |
2372 | + } |
2373 | + }, |
2374 | + "Steps": { |
2375 | + "build-package": { |
2376 | + "Timeout": "20m", |
2377 | + "SubWorkflow": { |
2378 | + "Path": "./build_package.wf.json", |
2379 | + "Vars": { |
2380 | + "type": "rpm", |
2381 | + "sourceImage": "projects/rhel-cloud/global/images/family/rhel-9-arm64", |
2382 | + "gcs_path": "${gcs_path}", |
2383 | + "repo_owner": "${repo_owner}", |
2384 | + "repo_name": "${repo_name}", |
2385 | + "git_ref": "${git_ref}", |
2386 | + "build_dir": "${build_dir}", |
2387 | + "machine_type": "t2a-standard-2", |
2388 | + "zone": "us-central1-a", |
2389 | + "version": "${version}" |
2390 | + } |
2391 | + } |
2392 | + } |
2393 | + } |
2394 | +} |
2395 | diff --git a/packagebuild/workflows/build_goo.wf.json b/packagebuild/workflows/build_goo.wf.json |
2396 | new file mode 100644 |
2397 | index 0000000..31efd40 |
2398 | --- /dev/null |
2399 | +++ b/packagebuild/workflows/build_goo.wf.json |
2400 | @@ -0,0 +1,40 @@ |
2401 | +{ |
2402 | + "Name": "goo", |
2403 | + "Vars": { |
2404 | + "gcs_path": { |
2405 | + "Required": true |
2406 | + }, |
2407 | + "repo_owner": { |
2408 | + "Required": true |
2409 | + }, |
2410 | + "repo_name": { |
2411 | + "Required": true |
2412 | + }, |
2413 | + "git_ref": { |
2414 | + "Required": true |
2415 | + }, |
2416 | + "version": { |
2417 | + "Required": true |
2418 | + }, |
2419 | + "build_dir": { |
2420 | + "Required": true |
2421 | + } |
2422 | + }, |
2423 | + "Steps": { |
2424 | + "build-package": { |
2425 | + "SubWorkflow": { |
2426 | + "Path": "./build_package.wf.json", |
2427 | + "Vars": { |
2428 | + "type": "goo", |
2429 | + "sourceImage": "projects/debian-cloud/global/images/family/debian-10", |
2430 | + "gcs_path": "${gcs_path}", |
2431 | + "repo_owner": "${repo_owner}", |
2432 | + "repo_name": "${repo_name}", |
2433 | + "git_ref": "${git_ref}", |
2434 | + "build_dir": "${build_dir}", |
2435 | + "version": "${version}" |
2436 | + } |
2437 | + } |
2438 | + } |
2439 | + } |
2440 | +} |
2441 | diff --git a/packagebuild/workflows/build_package.wf.json b/packagebuild/workflows/build_package.wf.json |
2442 | new file mode 100644 |
2443 | index 0000000..7a90994 |
2444 | --- /dev/null |
2445 | +++ b/packagebuild/workflows/build_package.wf.json |
2446 | @@ -0,0 +1,101 @@ |
2447 | +{ |
2448 | + "Name": "build-package", |
2449 | + "DefaultTimeout": "30m", |
2450 | + "Sources": { |
2451 | + "startup.sh": "../daisy_startupscript_${type}.sh", |
2452 | + "common.sh": "../common.sh" |
2453 | + }, |
2454 | + "Vars": { |
2455 | + "type": { |
2456 | + "Description": "Type of workflow: rpm,deb, or goo", |
2457 | + "Required": true |
2458 | + }, |
2459 | + "sourceImage": { |
2460 | + "Description": "Source image to use for instance", |
2461 | + "Required": true |
2462 | + }, |
2463 | + "gcs_path": { |
2464 | + "Required": true |
2465 | + }, |
2466 | + "repo_owner": { |
2467 | + "Required": true |
2468 | + }, |
2469 | + "repo_name": { |
2470 | + "Required": true |
2471 | + }, |
2472 | + "git_ref": { |
2473 | + "Required": true |
2474 | + }, |
2475 | + "version": { |
2476 | + "Required": true |
2477 | + }, |
2478 | + "build_dir": { |
2479 | + "Required": true |
2480 | + }, |
2481 | + "machine_type": { |
2482 | + "Value": "e2-standard-2", |
2483 | + "Description": "machine type" |
2484 | + }, |
2485 | + "zone": { |
2486 | + "Value": "us-central1-a", |
2487 | + "Description": "zone" |
2488 | + } |
2489 | + }, |
2490 | + "Steps": { |
2491 | + "build-packages": { |
2492 | + "CreateInstances": [ |
2493 | + { |
2494 | + "disks": [ |
2495 | + { |
2496 | + "initializeParams": { |
2497 | + "diskType": "pd-ssd", |
2498 | + "sourceImage": "${sourceImage}" |
2499 | + } |
2500 | + } |
2501 | + ], |
2502 | + "Metadata": { |
2503 | + "repo-owner": "${repo_owner}", |
2504 | + "repo-name": "${repo_name}", |
2505 | + "git-ref": "${git_ref}", |
2506 | + "build-dir": "${build_dir}", |
2507 | + "version": "${version}" |
2508 | + }, |
2509 | + "machineType": "${machine_type}", |
2510 | + "zone": "${zone}", |
2511 | + "name": "inst-build-pkg", |
2512 | + "StartupScript": "startup.sh", |
2513 | + "Scopes": ["https://www.googleapis.com/auth/devstorage.read_write"] |
2514 | + } |
2515 | + ] |
2516 | + }, |
2517 | + "wait-for-build": { |
2518 | + "Timeout": "30m", |
2519 | + "WaitForInstancesSignal": [ |
2520 | + { |
2521 | + "Name": "inst-build-pkg", |
2522 | + "SerialOutput": { |
2523 | + "Port": 1, |
2524 | + "SuccessMatch": "Build succeeded", |
2525 | + "FailureMatch": "Build failed" |
2526 | + } |
2527 | + } |
2528 | + ] |
2529 | + }, |
2530 | + "copy-packages": { |
2531 | + "CopyGCSObjects": [ |
2532 | + { |
2533 | + "Source": "${OUTSPATH}/", |
2534 | + "Destination": "${gcs_path}/" |
2535 | + } |
2536 | + ] |
2537 | + } |
2538 | + }, |
2539 | + "Dependencies": { |
2540 | + "wait-for-build": [ |
2541 | + "build-packages" |
2542 | + ], |
2543 | + "copy-packages": [ |
2544 | + "wait-for-build" |
2545 | + ] |
2546 | + } |
2547 | +} |
2548 | diff --git a/policies/apt.go b/policies/apt.go |
2549 | index d326398..74d619f 100644 |
2550 | --- a/policies/apt.go |
2551 | +++ b/policies/apt.go |
2552 | @@ -22,9 +22,11 @@ import ( |
2553 | "io" |
2554 | "net/http" |
2555 | "sort" |
2556 | + "strconv" |
2557 | "strings" |
2558 | |
2559 | "github.com/GoogleCloudPlatform/osconfig/clog" |
2560 | + "github.com/GoogleCloudPlatform/osconfig/osinfo" |
2561 | "github.com/GoogleCloudPlatform/osconfig/packages" |
2562 | "golang.org/x/crypto/openpgp" |
2563 | "golang.org/x/crypto/openpgp/armor" |
2564 | @@ -83,6 +85,52 @@ func containsEntity(es []*openpgp.Entity, e *openpgp.Entity) bool { |
2565 | return false |
2566 | } |
2567 | |
2568 | +func readInstanceOsInfo() (string, float64, error) { |
2569 | + oi, err := osinfo.Get() |
2570 | + if err != nil { |
2571 | + return "", 0, fmt.Errorf("error getting osinfo: %v", err) |
2572 | + } |
2573 | + |
2574 | + osVersion, err := strconv.ParseFloat(oi.Version, 64) |
2575 | + if err != nil { |
2576 | + osVersion = 0 |
2577 | + } |
2578 | + |
2579 | + return oi.ShortName, osVersion, nil |
2580 | +} |
2581 | + |
2582 | +func shouldUseSignedBy() bool { |
2583 | + osShortName, osVersion, err := readInstanceOsInfo() |
2584 | + if err != nil { |
2585 | + return false // Default to not using signed-by approach |
2586 | + } |
2587 | + |
2588 | + if (osShortName == "debian" && osVersion >= 12) || (osShortName == "ubuntu" && osVersion >= 24) { |
2589 | + return true |
2590 | + } |
2591 | + return false |
2592 | +} |
2593 | + |
2594 | +func getAptRepoLine(repo *agentendpointpb.AptRepository, useSignedBy bool) string { |
2595 | + archiveType, ok := debArchiveTypeMap[repo.ArchiveType] |
2596 | + if !ok { |
2597 | + archiveType = "deb" |
2598 | + } |
2599 | + |
2600 | + line := fmt.Sprintf("\n%s", archiveType) |
2601 | + |
2602 | + if useSignedBy { |
2603 | + line = fmt.Sprintf("%s [signed-by=%s]", line, aptGPGFile) |
2604 | + } |
2605 | + |
2606 | + line = fmt.Sprintf("%s %s %s", line, repo.Uri, repo.Distribution) |
2607 | + for _, c := range repo.Components { |
2608 | + line = fmt.Sprintf("%s %s", line, c) |
2609 | + } |
2610 | + |
2611 | + return line |
2612 | +} |
2613 | + |
2614 | func aptRepositories(ctx context.Context, repos []*agentendpointpb.AptRepository, repoFile string) error { |
2615 | var es []*openpgp.Entity |
2616 | var keys []string |
2617 | @@ -124,18 +172,18 @@ func aptRepositories(ctx context.Context, repos []*agentendpointpb.AptRepository |
2618 | # Repo file managed by Google OSConfig agent |
2619 | deb http://repo1-url/ repo1 main |
2620 | deb http://repo1-url/ repo2 main contrib non-free |
2621 | + |
2622 | + # For now, 'signed-by' keyring approach will be used for Debian 12+ and Ubuntu 24+ only. |
2623 | + To avoid conflicting repos for old stable OSes versions |
2624 | + e.g. deb [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo1-url/ repo1 main |
2625 | + NOTE: suggested by ofca@ |
2626 | */ |
2627 | var buf bytes.Buffer |
2628 | buf.WriteString("# Repo file managed by Google OSConfig agent\n") |
2629 | + |
2630 | + shouldUseSignedByBool := shouldUseSignedBy() |
2631 | for _, repo := range repos { |
2632 | - archiveType, ok := debArchiveTypeMap[repo.ArchiveType] |
2633 | - if !ok { |
2634 | - archiveType = "deb" |
2635 | - } |
2636 | - line := fmt.Sprintf("\n%s %s %s", archiveType, repo.Uri, repo.Distribution) |
2637 | - for _, c := range repo.Components { |
2638 | - line = fmt.Sprintf("%s %s", line, c) |
2639 | - } |
2640 | + line := getAptRepoLine(repo, shouldUseSignedByBool) |
2641 | buf.WriteString(line + "\n") |
2642 | } |
2643 | |
2644 | diff --git a/policies/apt_test.go b/policies/apt_test.go |
2645 | index fc9f651..6a8c2d3 100644 |
2646 | --- a/policies/apt_test.go |
2647 | +++ b/policies/apt_test.go |
2648 | @@ -108,3 +108,31 @@ func TestGetAptGPGKey(t *testing.T) { |
2649 | t.Errorf("Expected to find Artifact Registry key in Google Cloud Public GPG key, but its missed.") |
2650 | } |
2651 | } |
2652 | + |
2653 | +func TestUseSignedBy(t *testing.T) { |
2654 | + tests := []struct { |
2655 | + desc string |
2656 | + repo *agentendpointpb.AptRepository |
2657 | + want string |
2658 | + }{ |
2659 | + { |
2660 | + "1 repo", |
2661 | + &agentendpointpb.AptRepository{Uri: "http://repo1-url/", Distribution: "distribution", Components: []string{"component1"}}, |
2662 | + "\ndeb [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo1-url/ distribution component1", |
2663 | + }, |
2664 | + { |
2665 | + "2 components", |
2666 | + &agentendpointpb.AptRepository{Uri: "http://repo2-url/", Distribution: "distribution", Components: []string{"component1", "component2"}, ArchiveType: agentendpointpb.AptRepository_DEB}, |
2667 | + "\ndeb [signed-by=/etc/apt/trusted.gpg.d/osconfig_agent_managed.gpg] http://repo2-url/ distribution component1 component2", |
2668 | + }, |
2669 | + } |
2670 | + |
2671 | + useSignedBy := true |
2672 | + for _, tt := range tests { |
2673 | + aptRepoLine := getAptRepoLine(tt.repo, useSignedBy) |
2674 | + |
2675 | + if aptRepoLine != tt.want { |
2676 | + t.Errorf("%s: got:\n%q\nwant:\n%q", tt.desc, aptRepoLine, tt.want) |
2677 | + } |
2678 | + } |
2679 | +} |
Hey, thank you for the MP. It looks good but it's a bit hard to understand why commit b7a00db is needed. Could you please update the description of the commit and the patch as well?