Merge ~dashmage/charm-sysconfig:bug-2012581/inherit-existing-kernel-params into charm-sysconfig:master
- Git
- lp:~dashmage/charm-sysconfig
- bug-2012581/inherit-existing-kernel-params
- Merge into master
Status: | Merged |
---|---|
Approved by: | Eric Chen |
Approved revision: | 0e5531c372b4776fcde2f541565fc17c6d8f9886 |
Merged at revision: | a2e49a8d488bbf5eadcf84dc5934731b5ec44743 |
Proposed branch: | ~dashmage/charm-sysconfig:bug-2012581/inherit-existing-kernel-params |
Merge into: | charm-sysconfig:master |
Diff against target: |
149 lines (+71/-7) 5 files modified
src/actions.yaml (+1/-1) src/config.yaml (+3/-0) src/lib/lib_sysconfig.py (+26/-1) src/templates/grub.j2 (+0/-5) src/tests/unit/test_lib.py (+41/-0) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
JamesLin | Approve | ||
🤖 prod-jenkaas-bootstack (community) | continuous-integration | Approve | |
Eric Chen | Approve | ||
Erhan Sunar (community) | Needs Fixing | ||
BootStack Reviewers | Pending | ||
Review via email: mp+441304@code.launchpad.net |
Commit message
Inherit pre-existing kernel cmdline parameters by default.
- Also prevents overwrite of existing console parameters by the charm.
Description of the change
Fix for issue #4 in BSENG-1027 [1] for the LP bugs #2012581 [2], #1854135 [3]. Addresses this comment on the LP bug as well [4].
[1]: https:/
[2]: https:/
[3]: https:/
[4]: https:/
Erhan Sunar (esunar) wrote : | # |
Can you link the bug reports to MR
Ashley James (dashmage) wrote : | # |
> It's LGTM for implementation part, but I think the unit test for it is
> required.
There seems to be unit tests already for the `update_grub_file` function [1]. Would I need to add something additionally to test for the change made in the template file?
[1]: https:/
> Also please provide more detail to explain why this change works.
```
-{% else -%}
-GRUB_CMDLINE_
```
This is to allow inheriting the pre-existing kernel cmdline parameters the default option. With the charm no longer resetting the value of `GRUB_CMDLINE_
```
-GRUB_CMDLINE_
+GRUB_CMDLINE_
```
These parameters have been switched around so that the pre-existing console values (ex: coming from MAAS) will not be overwritten by the charm. According to my understanding, the console parameters (if defined) from `$GRUB_
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
This merge proposal is being monitored by mergebot. Change the status to Approved to merge.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:2291db2609a
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Ashley James (dashmage) wrote : | # |
Got feedback from @mfo who implemented the initial logic in the grub template file:
```
It looks like that MR is good.
AFAICT it
1) fixes the case I had wrong (do not erase $GRUB... if charm config option is not set), and
2) also addresses the case there are MAAS kernel options.
I haven't done any testing (I know you would / already have, for both issues), but it looks good from what I see.
```
JamesLin (jneo8) wrote (last edit ): | # |
> There seems to be unit tests already for the `update_grub_file` function [1]. Would I need to add something additionally to test for the change made in the template file?
The unit test currently doesn't cover the bug case and it should be.
Eric Chen (eric-chen) wrote : | # |
> > It's LGTM for implementation part, but I think the unit test for it is
> > required.
>
> There seems to be unit tests already for the `update_grub_file` function [1].
> Would I need to add something additionally to test for the change made in the
> template file?
>
> [1]: https:/
> sysconfig/
>
> > Also please provide more detail to explain why this change works.
>
> ```
> -{% else -%}
> -GRUB_CMDLINE_
> ```
>
> This is to allow inheriting the pre-existing kernel cmdline parameters the
> default option. With the charm no longer resetting the value of
> `GRUB_CMDLINE_
> already existing values (ex: coming from MAAS).
>
> ```
> -GRUB_CMDLINE_
> console=tty0"
> +GRUB_CMDLINE_
> $GRUB_CMDLINE_
> ```
>
> These parameters have been switched around so that the pre-existing console
> values (ex: coming from MAAS) will not be overwritten by the charm. According
> to my understanding, the console parameters (if defined) from
> `$GRUB_
> last.
This is a little tricky to me. There already a precedence design under /etc/grub.d/ folder.
If we want to put "console=
I don't know if other people have the same feeling. I suggest that we should find the stakeholders to discuss which one should have higher precedence and solve it in a correct way.
Ashley James (dashmage) wrote (last edit ): | # |
After discussing with @nobuto, the real issue faced is not that the MAAS console parameters are being overwritten.
```
the original issue was that even if the machine already had ttyS0 and tty, charm added those additionally and it triggered a reboot required flag
as long as the reboot required flag is not triggered, I'm happy with any solutions
```
The charm will certainly trigger a reboot notification at the time of deployment since the template file has minor changes from the original config file on the unit. But this only happens at the time of initial deployment of the charm. Could a possible solution be to have the charm run the `clear-
In case the charm does need to support a way to prevent overwriting the MAAS kernel parameters, I've listed down some possible solutions and my proposal in this document here: https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:225a3b36c61
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Ashley James (dashmage) wrote : | # |
What this MR does:
- Inherit pre-existing kernel parameters (eg: from MAAS) by default
- Remove hard-coded console parameters
- Introduce new "console" config option
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:954c28f4bc7
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:e31dbba8a86
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:954c28f4bc7
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:e31dbba8a86
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) wrote : | # |
1. No unit test for prepare_
2. some inline comment, please fix it.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:856dfa29029
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Ashley James (dashmage) wrote : | # |
Updated MR:
- Inherit pre-existing kernel parameters (eg: from MAAS) by default
- Remove hard-coded console parameters
- Rework config parser function to support console parameters
- Unit tests for config parser function
JamesLin (jneo8) wrote : | # |
Please also update the config and doc-string.
Eric Chen (eric-chen) : | # |
Eric Chen (eric-chen) wrote : | # |
Please see inline comment.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:2ca99b5719a
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Ashley James (dashmage) wrote : | # |
Resolved comments by @jneo8 and @eric-chen
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:71aa123b8e1
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:2ca99b5719a
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:71aa123b8e1
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) wrote : | # |
please check inline comment and fix it
Eric Chen (eric-chen) wrote : | # |
one more inline comment
Ashley James (dashmage) wrote : | # |
> I think a string with "=" assignments is not the key point in case 3. There are also "=" in case 1&2. The major different is double quote. Please mention that the subkey and value can be assigned by double quote.
There are "=" in the prev 2 cases but they do not occur in the values. Here in case 3,
'key1="subkey1=val1 subkey2=val2", key2=val3' -- the double quotes are just to make the assignment between key value pairs more clearer. The key point is that the subkey values themselves contain assignments that include "=". Even if the double quotes aren't present, the parser will correctly parse it into a dictionary.
> This case is weird. Don't we need the double quote?
'key1="
Here as well, the double quotes aren't really necessary but just to aid in identifying the key and value clearer. This case is to show how there's "=" assignments and the subkey is assigned multiple values separated by commas unlike the previous case which just contained a single value. The commas between val1 and val2 in "subkey1=val1,val2" is what this case is supposed to capture.
I've also added another unit test where the double quotes aren't present. I've addressed the other inline comments as well.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:41ba0f611fd
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:41ba0f611fd
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) wrote : | # |
Please check my inline comment.
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:41ba0f611fd
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:081d772a25f
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:081d772a25f
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:081d772a25f
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
FAILED: Continuous integration, rev:081d772a25f
https:/
Executed test runs:
FAILURE: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:081d772a25f
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) wrote : | # |
Please check inline comment. You miss some of my comments before.
Ashley James (dashmage) wrote : | # |
@eric-chen -- addressed all your latest comments
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:c5f37ade0c6
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
Eric Chen (eric-chen) : | # |
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : | # |
PASSED: Continuous integration, rev:0e5531c372b
https:/
Executed test runs:
SUCCESS: https:/
None: https:/
Click here to trigger a rebuild:
https:/
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : | # |
Change successfully merged at revision a2e49a8d488bbf5
Preview Diff
1 | diff --git a/src/actions.yaml b/src/actions.yaml |
2 | index 1dcaedb..93282ae 100644 |
3 | --- a/src/actions.yaml |
4 | +++ b/src/actions.yaml |
5 | @@ -6,7 +6,7 @@ clear-notification: |
6 | By doing so, unit will change the description to 'ready' |
7 | update-grub: |
8 | description: | |
9 | - This action will run 'update-grub' to the generate a GRUB configuration file. |
10 | + This action will run 'update-grub' to generate a GRUB configuration file. |
11 | check-update-grub: |
12 | description: | |
13 | This action will test if there is any GRUB configuration changes available. |
14 | diff --git a/src/config.yaml b/src/config.yaml |
15 | index ba4a0a6..f7455e4 100644 |
16 | --- a/src/config.yaml |
17 | +++ b/src/config.yaml |
18 | @@ -121,6 +121,9 @@ options: |
19 | to avoid them being overwritten, for example |
20 | grub-config-flags='GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT |
21 | nvme_core.multipath=0' |
22 | + Also supports adding values with special characters in them(`$`, `=`, `,` etc.) |
23 | + but they need to be enclosed in double quotes. For example, grub-config-flags='GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT |
24 | + console=ttyS0,115200 console=tty0",GRUB_TIMEOUT=0' |
25 | isolcpus: |
26 | default: "" |
27 | type: string |
28 | diff --git a/src/lib/lib_sysconfig.py b/src/lib/lib_sysconfig.py |
29 | index 1a9875a..caf4c00 100644 |
30 | --- a/src/lib/lib_sysconfig.py |
31 | +++ b/src/lib/lib_sysconfig.py |
32 | @@ -5,6 +5,7 @@ Manage grub, systemd, coufrequtils and kernel version configuration. |
33 | import filecmp |
34 | import hashlib |
35 | import os |
36 | +import re |
37 | import subprocess |
38 | from datetime import datetime, timedelta, timezone |
39 | |
40 | @@ -35,10 +36,34 @@ IRQBALANCE_CONF = "/etc/default/irqbalance" |
41 | def parse_config_flags(config_flags): |
42 | """Parse config flags into a dict. |
43 | |
44 | + This parsing function supports a few different formats |
45 | + depending on the config flags being provided. |
46 | + |
47 | + 1. Simple string with key=value pairs. For example, |
48 | + a string in the format of 'key1=val1, key2=val2' will return a dict of: |
49 | + {'key1': 'val1', 'key2': 'val2'}. |
50 | + |
51 | + 2. A string in the above format, but supporting a comma-delimited |
52 | + list of values for the same key. For example, a string in the format of |
53 | + 'key1=val1, key2=val2,val3,val4' will return a dict of: |
54 | + {'key1': 'val1', 'key2': 'val2,val3,val4'} |
55 | + |
56 | + 3. A string with special characters (`=`, `,`) in the value. |
57 | + Note: This case requires double quotes to properly separate the key and value pairs |
58 | + since there are special characters that need to be protected. Adding the double |
59 | + quotes explicitly ensures that the quotes are present in the dict output as well. |
60 | + For example, a string in the format of 'key1="subkey1=val1,val2 subkey2=val3"' |
61 | + will return a dict of: {'key1': '"subkey1=val1,val2 subkey2=val3"'} |
62 | + |
63 | :param config_flags: key pairs list. Format: key1=value1,key2=value2 |
64 | :return dict: format {'key1': 'value1', 'key2': 'value2'} |
65 | """ |
66 | - key_value_pairs = config_flags.split(",") |
67 | + # This regular expression is used to split the config flags |
68 | + # into a list while preserving the content within the |
69 | + # double quotes even if it contains commas |
70 | + # For example, it splits 'val1, val2, "val3,val4", val5' |
71 | + # as ['val1', 'val2', '"val3, val4"', 'val5] |
72 | + key_value_pairs = re.split(r',(?=(?:[^"]*"[^"]*")*[^"]*$)', config_flags) |
73 | parsed_config_flags = {} |
74 | for index, pair in enumerate(key_value_pairs): |
75 | if "=" in pair: |
76 | diff --git a/src/templates/grub.j2 b/src/templates/grub.j2 |
77 | index a76ca57..5d7cf79 100644 |
78 | --- a/src/templates/grub.j2 |
79 | +++ b/src/templates/grub.j2 |
80 | @@ -5,18 +5,13 @@ |
81 | # For full documentation of the options in this file, see: |
82 | # info -f grub -n 'Simple configuration' |
83 | |
84 | -# Evaluate grub-config-flags first, in case it uses $GRUB_CMDLINE_LINUX_DEFAULT. |
85 | -# However, if grub-config-flags is not specified, ignore that (current behavior.) |
86 | {% if grub_config_flags is defined and grub_config_flags -%} |
87 | # config-flags |
88 | {% for key, value in grub_config_flags.items() -%} |
89 | {{ key }}={{ value }} |
90 | {% endfor -%} |
91 | -{% else -%} |
92 | -GRUB_CMDLINE_LINUX_DEFAULT="" |
93 | {% endif -%} |
94 | |
95 | -GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT console=ttyS0,115200 console=tty0" |
96 | {% if isolcpus is defined and isolcpus -%} |
97 | GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT isolcpus={{ isolcpus }}" |
98 | {% endif -%} |
99 | diff --git a/src/tests/unit/test_lib.py b/src/tests/unit/test_lib.py |
100 | index 9721313..b2b5354 100644 |
101 | --- a/src/tests/unit/test_lib.py |
102 | +++ b/src/tests/unit/test_lib.py |
103 | @@ -45,6 +45,47 @@ def test_check_update_grub_unavailable(check_output, cmp_file): |
104 | assert "No available grub updates found." in message |
105 | |
106 | |
107 | +@pytest.mark.parametrize( |
108 | + "config_flags, expected", |
109 | + [ |
110 | + ("key1=val1,key2=val2", {"key1": "val1", "key2": "val2"}), |
111 | + # whitespace between key value pairs after comma |
112 | + ("key1=val1, key2=val2", {"key1": "val1", "key2": "val2"}), |
113 | + # multiple values assigned to key |
114 | + ("key1=val1,val2,key2=val3", {"key1": "val1,val2", "key2": "val3"}), |
115 | + ("key1=val1 val2 val3", {"key1": "val1 val2 val3"}), |
116 | + # Subkey assigned to key has a single value (val) assigned with "=" |
117 | + ('key="$key subkey=val"', {"key": '"$key subkey=val"'}), |
118 | + ( |
119 | + 'key1="subkey1=val1 val2",key2="$key2 val3"', |
120 | + {"key1": '"subkey1=val1 val2"', "key2": '"$key2 val3"'}, |
121 | + ), |
122 | + # Subkey has multiple values (val1, val2) assigned each separated by comma |
123 | + ( |
124 | + 'key1="subkey1=val1,val2 subkey2=val3",key2=val4', |
125 | + {"key1": '"subkey1=val1,val2 subkey2=val3"', "key2": "val4"}, |
126 | + ), |
127 | + ( |
128 | + 'key1="$key1 subkey1=val1,val2 subkey2=val3", key2=val4', |
129 | + {"key1": '"$key1 subkey1=val1,val2 subkey2=val3"', "key2": "val4"}, |
130 | + ), |
131 | + ( |
132 | + ( |
133 | + 'key1="$key1 subkey1=val1,val2 subkey2=val3", key2="$key2 ' |
134 | + 'subkey1=val1 subkey2=val2"' |
135 | + ), |
136 | + { |
137 | + "key1": '"$key1 subkey1=val1,val2 subkey2=val3"', |
138 | + "key2": '"$key2 subkey1=val1 subkey2=val2"', |
139 | + }, |
140 | + ), |
141 | + ], |
142 | +) |
143 | +def test_parse_config_flags(config_flags, expected): |
144 | + """Test parsing function for config flags.""" |
145 | + assert lib_sysconfig.parse_config_flags(config_flags) == expected |
146 | + |
147 | + |
148 | class TestBootResourceState: |
149 | """Test BootResourceState class.""" |
150 |
It's LGTM for implementation part, but I think the unit test for it is required.
Also please provide more detail to explain why this change works.