Merge ~artivis/lpci:feature/ros-plugins into lpci:main

Proposed by jeremie
Status: Needs review
Proposed branch: ~artivis/lpci:feature/ros-plugins
Merge into: lpci:main
Diff against target: 1087 lines (+1041/-2)
2 files modified
lpcraft/plugin/tests/test_plugins.py (+765/-2)
lpcraft/plugins/plugins.py (+276/-0)
Reviewer Review Type Date Requested Status
Jürgen Gmach Approve
Review via email: mp+435371@code.launchpad.net

Description of the change

This implements three new plugins for building and testing ROS & ROS 2 projects, namely,

- CatkinPlugin - based on the 'catkin' build tool
- CatkinToolsPlugin - based on the 'catkin-tools' build tool
- ColconPlugin - based on the 'colcon' build tool

These plugins all share a common base class - RosBasePlugin - factoring some common code.

The three plugins follow the same principle,

- Install the ROS project's dependencies during the 'run-before' step
- Build the ROS project during the 'run' step
- Build and run the tests during the 'run-after' step (optional)

The later is optional and depends on a plugin config parameter.

Beside some ROS-specific environment variables, the base class define the following env. vars.,

- ROS_PROJECT_BASE - the parent directory of the lpcraft build tree
- ROS_PROJECT_SRC - the directory of the project
- ROS_PROJECT_BUILD - the directory of the plugins build tree
- ROS_PROJECT_INSTALL - the directory of the plugins install tree

They somewhat mimic env. vars. in snapcraft allowing for out-of-source build/install. They also allow the end user to more easily override the run commands not having to figure out the internal paths. They could be replaced in the future by lpcraft-wide env. vars.

To post a comment you must log in.
Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Jeremie, thanks for the merge proposal.

CI currently fails as there are two uncovered lines, see buildlog below (below `unmerged commits`):

```
:: lpcraft/plugins/plugins.py 284 2 64 1 99% 482, 491
```

Could you please fix these?

As lpcraft is both the CI runner for Launchpad and a standalone tool, I'd like to know how you are planning to use it. Do you use or plan to use it in Launchpad CI?

I am asking as this would influence the plugin story. For a standalone lpcraft we could easily add setuptools based plugins ( https://pluggy.readthedocs.io/en/stable/#loading-setuptools-entry-points ), so we could have the plugin in a standalone plugin package. For Launchpad CI this is more elaborate, and currently out of scope.

I am discussing these options, as at least for me ROS is a business domain I have never touched before, and including these plugins means I (we) have to maintain them.

Fortunately, the code looks pretty straightforward. I will discuss this MP at today's standup and will get back to you.

Afterwards, I will also give you a detailed review. At very least you would need to add some descriptive docstrings which explain the used business terminology.

Also, does this MP depend on https://code.launchpad.net/~artivis/lpcraft/+git/lpcraft/+merge/435370 or would merging this MP on its own already be helpful?

Revision history for this message
jeremie (artivis) wrote :

Hello,
I've already tackled the missed lines in coverage and the CI is happy as far as I can tell.
We are indeed planning on using those plugins in Launchpad CI.
This MP is stand-alone and totally independent of https://code.launchpad.net/~artivis/lpcraft/+git/lpcraft/+merge/435370 .
Looking forward to your review :+1:

Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Looks good with some minor comments!

As we have no merge bot activated for lpcraft, I think I need to merge the MP.

Just tell me when you think it is ready to be merged.

Note for myself:
- In a follow-up MP we should split the plugins and the tests into submodules.
- In a follow-up MP we should revisit the currently used exceptions.

review: Approve

Unmerged commits

b435946... by jeremie

plugins: add ros plugins

Signed-off-by: artivis <email address hidden>

Succeeded
[SUCCEEDED] test:0 (build)
[SUCCEEDED] build:0 (build)
12 of 2 results

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/lpcraft/plugin/tests/test_plugins.py b/lpcraft/plugin/tests/test_plugins.py
2index 1def73b..f7ba1b8 100644
3--- a/lpcraft/plugin/tests/test_plugins.py
4+++ b/lpcraft/plugin/tests/test_plugins.py
5@@ -5,7 +5,7 @@ import os
6 import subprocess
7 from pathlib import Path, PosixPath
8 from textwrap import dedent
9-from typing import List, Optional, Union, cast
10+from typing import Any, Dict, List, Optional, Union, cast
11 from unittest.mock import ANY, Mock, call, patch
12
13 from craft_providers.lxd import launch
14@@ -17,7 +17,7 @@ from lpcraft.commands.tests import CommandBaseTestCase
15 from lpcraft.errors import ConfigurationError
16 from lpcraft.plugin.manager import get_plugin_manager
17 from lpcraft.plugins import register
18-from lpcraft.plugins.plugins import BaseConfig, BasePlugin
19+from lpcraft.plugins.plugins import BaseConfig, BasePlugin, RosBasePlugin
20 from lpcraft.providers.tests import makeLXDProvider
21
22
23@@ -1080,3 +1080,766 @@ class TestPlugins(CommandBaseTestCase):
24 lpcraft.config.Config.load,
25 config_path,
26 )
27+
28+ def _get_ros_before_run_call(self, expected_env: Dict[str, str]) -> Any:
29+ return call(
30+ [
31+ "bash",
32+ "--noprofile",
33+ "--norc",
34+ "-ec",
35+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
36+ "\nif [ ! -f /etc/ros/rosdep/sources.list.d/20-default.list ]; then\nrosdep init\nfi" # noqa:E501
37+ "\nrosdep update --rosdistro ${ROS_DISTRO}"
38+ "\nrosdep install -i -y --rosdistro ${ROS_DISTRO} --from-paths .", # noqa:E501
39+ ],
40+ cwd=PosixPath("/build/lpcraft/project"),
41+ env=expected_env,
42+ stdout=ANY,
43+ stderr=ANY,
44+ )
45+
46+ def _get_ros_packages_call(
47+ self, expected_env: Dict[str, str], packages: List[str]
48+ ) -> Any:
49+ return call(
50+ [
51+ "apt",
52+ "install",
53+ "-y",
54+ "build-essential",
55+ "cmake",
56+ "g++",
57+ "python3-rosdep",
58+ ]
59+ + packages,
60+ cwd=PosixPath("/build/lpcraft/project"),
61+ env=expected_env,
62+ stdout=ANY,
63+ stderr=ANY,
64+ )
65+
66+ def _get_ros_expected_env(
67+ self, ros_distro: str, ros_python_version: str, ros_version: str
68+ ) -> Dict[str, str]:
69+ return {
70+ "ROS_PROJECT_BASE": "/build/lpcraft",
71+ "ROS_PROJECT_SRC": "/build/lpcraft/project",
72+ "ROS_PROJECT_BUILD": "/build/lpcraft/build",
73+ "ROS_PROJECT_INSTALL": "/build/lpcraft/install",
74+ "ROS_DISTRO": ros_distro,
75+ "ROS_PYTHON_VERSION": ros_python_version,
76+ "ROS_VERSION": ros_version,
77+ }
78+
79+ def test_fake_ros_plugin_no_version(self):
80+ config = dedent(
81+ """
82+ pipeline:
83+ - build
84+
85+ jobs:
86+ build:
87+ plugin: fake-ros-plugin
88+ series: bionic
89+ architectures: amd64
90+ """
91+ )
92+ config_path = Path(".launchpad.yaml")
93+ config_path.write_text(config)
94+
95+ @register(name="fake-ros-plugin")
96+ class FakeRosPlugin(RosBasePlugin):
97+ pass
98+
99+ config_obj = lpcraft.config.Config.load(config_path)
100+ job = config_obj.jobs["build"][0]
101+ pm = get_plugin_manager(job)
102+ plugins = pm.get_plugins()
103+ fake_plugin = [
104+ _ for _ in plugins if _.__class__.__name__ == "FakeRosPlugin"
105+ ]
106+ self.assertEqual(job.plugin, "fake-ros-plugin")
107+ self.assertRaises(NotImplementedError, fake_plugin[0]._get_ros_version)
108+
109+ def test_fake_ros_plugin_bad_version(self):
110+ config = dedent(
111+ """
112+ pipeline:
113+ - build
114+
115+ jobs:
116+ build:
117+ plugin: fake-ros-plugin
118+ series: bionic
119+ architectures: amd64
120+ """
121+ )
122+ config_path = Path(".launchpad.yaml")
123+ config_path.write_text(config)
124+
125+ @register(name="fake-ros-plugin")
126+ class FakeRosPlugin(RosBasePlugin):
127+ def _get_ros_version(self) -> int:
128+ return 3
129+
130+ config_obj = lpcraft.config.Config.load(config_path)
131+ job = config_obj.jobs["build"][0]
132+ pm = get_plugin_manager(job)
133+ plugins = pm.get_plugins()
134+ fake_plugin = [
135+ _ for _ in plugins if _.__class__.__name__ == "FakeRosPlugin"
136+ ]
137+ self.assertEqual(job.plugin, "fake-ros-plugin")
138+ self.assertEqual(3, fake_plugin[0]._get_ros_version())
139+ self.assertRaises(Exception, fake_plugin[0]._get_ros_distro)
140+
141+ @patch("lpcraft.commands.run.get_provider")
142+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
143+ def test_catkin_plugin_bionic(
144+ self, mock_get_host_architecture, mock_get_provider
145+ ):
146+ launcher = Mock(spec=launch)
147+ provider = makeLXDProvider(lxd_launcher=launcher)
148+ mock_get_provider.return_value = provider
149+ execute_run = launcher.return_value.execute_run
150+ execute_run.return_value = subprocess.CompletedProcess([], 0)
151+ config = dedent(
152+ """
153+ pipeline:
154+ - build
155+
156+ jobs:
157+ build:
158+ plugin: catkin
159+ series: bionic
160+ architectures: amd64
161+ """
162+ )
163+ Path(".launchpad.yaml").write_text(config)
164+
165+ expected_env = self._get_ros_expected_env("melodic", "2", "1")
166+
167+ self.run_command("run")
168+ self.assertEqual(
169+ [
170+ call(
171+ ["apt", "update"],
172+ cwd=PosixPath("/build/lpcraft/project"),
173+ env=expected_env,
174+ stdout=ANY,
175+ stderr=ANY,
176+ ),
177+ self._get_ros_packages_call(
178+ expected_env, ["ros-melodic-catkin"]
179+ ),
180+ self._get_ros_before_run_call(expected_env),
181+ call(
182+ [
183+ "bash",
184+ "--noprofile",
185+ "--norc",
186+ "-ec",
187+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
188+ "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
189+ ],
190+ cwd=PosixPath("/build/lpcraft/project"),
191+ env=expected_env,
192+ stdout=ANY,
193+ stderr=ANY,
194+ ),
195+ ],
196+ execute_run.call_args_list,
197+ )
198+
199+ @patch("lpcraft.commands.run.get_provider")
200+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
201+ def test_catkin_plugin(
202+ self, mock_get_host_architecture, mock_get_provider
203+ ):
204+ launcher = Mock(spec=launch)
205+ provider = makeLXDProvider(lxd_launcher=launcher)
206+ mock_get_provider.return_value = provider
207+ execute_run = launcher.return_value.execute_run
208+ execute_run.return_value = subprocess.CompletedProcess([], 0)
209+ config = dedent(
210+ """
211+ pipeline:
212+ - build
213+
214+ jobs:
215+ build:
216+ plugin: catkin
217+ series: focal
218+ architectures: amd64
219+ """
220+ )
221+ Path(".launchpad.yaml").write_text(config)
222+
223+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
224+
225+ self.run_command("run")
226+ self.assertEqual(
227+ [
228+ call(
229+ ["apt", "update"],
230+ cwd=PosixPath("/build/lpcraft/project"),
231+ env=expected_env,
232+ stdout=ANY,
233+ stderr=ANY,
234+ ),
235+ self._get_ros_packages_call(
236+ expected_env, ["ros-noetic-catkin"]
237+ ),
238+ self._get_ros_before_run_call(expected_env),
239+ call(
240+ [
241+ "bash",
242+ "--noprofile",
243+ "--norc",
244+ "-ec",
245+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
246+ "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
247+ ],
248+ cwd=PosixPath("/build/lpcraft/project"),
249+ env=expected_env,
250+ stdout=ANY,
251+ stderr=ANY,
252+ ),
253+ ],
254+ execute_run.call_args_list,
255+ )
256+
257+ @patch("lpcraft.commands.run.get_provider")
258+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
259+ def test_catkin_plugin_with_tests(
260+ self, mock_get_host_architecture, mock_get_provider
261+ ):
262+ launcher = Mock(spec=launch)
263+ provider = makeLXDProvider(lxd_launcher=launcher)
264+ mock_get_provider.return_value = provider
265+ execute_run = launcher.return_value.execute_run
266+ execute_run.return_value = subprocess.CompletedProcess([], 0)
267+ config = dedent(
268+ """
269+ pipeline:
270+ - build
271+
272+ jobs:
273+ build:
274+ plugin: catkin
275+ run-tests: True
276+ series: focal
277+ architectures: amd64
278+ packages: [file, git]
279+ """
280+ )
281+ Path(".launchpad.yaml").write_text(config)
282+
283+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
284+
285+ self.run_command("run")
286+ self.assertEqual(
287+ [
288+ call(
289+ ["apt", "update"],
290+ cwd=PosixPath("/build/lpcraft/project"),
291+ env=expected_env,
292+ stdout=ANY,
293+ stderr=ANY,
294+ ),
295+ self._get_ros_packages_call(
296+ expected_env, ["ros-noetic-catkin", "file", "git"]
297+ ),
298+ self._get_ros_before_run_call(expected_env),
299+ call(
300+ [
301+ "bash",
302+ "--noprofile",
303+ "--norc",
304+ "-ec",
305+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
306+ "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
307+ ],
308+ cwd=PosixPath("/build/lpcraft/project"),
309+ env=expected_env,
310+ stdout=ANY,
311+ stderr=ANY,
312+ ),
313+ call(
314+ [
315+ "bash",
316+ "--noprofile",
317+ "--norc",
318+ "-ec",
319+ "\nsource ${ROS_PROJECT_BASE}/devel_isolated/setup.sh"
320+ "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE} --force-cmake --catkin-make-args run_tests" # noqa:E501
321+ "\ncatkin_test_results --verbose ${ROS_PROJECT_BASE}/build_isolated/", # noqa:E501
322+ ],
323+ cwd=PosixPath("/build/lpcraft/project"),
324+ env=expected_env,
325+ stdout=ANY,
326+ stderr=ANY,
327+ ),
328+ ],
329+ execute_run.call_args_list,
330+ )
331+
332+ @patch("lpcraft.commands.run.get_provider")
333+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
334+ def test_catkin_plugin_user_command(
335+ self, mock_get_host_architecture, mock_get_provider
336+ ):
337+ launcher = Mock(spec=launch)
338+ provider = makeLXDProvider(lxd_launcher=launcher)
339+ mock_get_provider.return_value = provider
340+ execute_run = launcher.return_value.execute_run
341+ execute_run.return_value = subprocess.CompletedProcess([], 0)
342+ config = dedent(
343+ """
344+ pipeline:
345+ - build
346+
347+ jobs:
348+ build:
349+ plugin: catkin
350+ run-tests: True
351+ series: focal
352+ architectures: amd64
353+ packages: [file, git]
354+ run-before: echo 'hello'
355+ run: echo 'robot'
356+ # run-after: "we shouldn't get anything unless uncommented"
357+ """
358+ )
359+ Path(".launchpad.yaml").write_text(config)
360+
361+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
362+
363+ self.run_command("run")
364+ self.assertEqual(
365+ [
366+ call(
367+ ["apt", "update"],
368+ cwd=PosixPath("/build/lpcraft/project"),
369+ env=expected_env,
370+ stdout=ANY,
371+ stderr=ANY,
372+ ),
373+ self._get_ros_packages_call(
374+ expected_env, ["ros-noetic-catkin", "file", "git"]
375+ ),
376+ call(
377+ ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
378+ cwd=PosixPath("/build/lpcraft/project"),
379+ env=expected_env,
380+ stdout=ANY,
381+ stderr=ANY,
382+ ),
383+ call(
384+ ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
385+ cwd=PosixPath("/build/lpcraft/project"),
386+ env=expected_env,
387+ stdout=ANY,
388+ stderr=ANY,
389+ ),
390+ ],
391+ execute_run.call_args_list,
392+ )
393+
394+ @patch("lpcraft.commands.run.get_provider")
395+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
396+ def test_catkin_tools_plugin(
397+ self, mock_get_host_architecture, mock_get_provider
398+ ):
399+ launcher = Mock(spec=launch)
400+ provider = makeLXDProvider(lxd_launcher=launcher)
401+ mock_get_provider.return_value = provider
402+ execute_run = launcher.return_value.execute_run
403+ execute_run.return_value = subprocess.CompletedProcess([], 0)
404+ config = dedent(
405+ """
406+ pipeline:
407+ - build
408+
409+ jobs:
410+ build:
411+ plugin: catkin-tools
412+ series: focal
413+ architectures: amd64
414+ """
415+ )
416+ Path(".launchpad.yaml").write_text(config)
417+
418+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
419+
420+ self.run_command("run")
421+ self.assertEqual(
422+ [
423+ call(
424+ ["apt", "update"],
425+ cwd=PosixPath("/build/lpcraft/project"),
426+ env=expected_env,
427+ stdout=ANY,
428+ stderr=ANY,
429+ ),
430+ self._get_ros_packages_call(
431+ expected_env, ["python3-catkin-tools"]
432+ ),
433+ self._get_ros_before_run_call(expected_env),
434+ call(
435+ [
436+ "bash",
437+ "--noprofile",
438+ "--norc",
439+ "-ec",
440+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
441+ "catkin init --workspace ${ROS_PROJECT_BASE}\n"
442+ "catkin profile add --force lpcraft\n"
443+ "catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}\n" # noqa:E501
444+ "catkin build --no-notify --profile lpcraft", # noqa:E501
445+ ],
446+ cwd=PosixPath("/build/lpcraft/project"),
447+ env=expected_env,
448+ stdout=ANY,
449+ stderr=ANY,
450+ ),
451+ ],
452+ execute_run.call_args_list,
453+ )
454+
455+ @patch("lpcraft.commands.run.get_provider")
456+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
457+ def test_catkin_tools_plugin_with_tests(
458+ self, mock_get_host_architecture, mock_get_provider
459+ ):
460+ launcher = Mock(spec=launch)
461+ provider = makeLXDProvider(lxd_launcher=launcher)
462+ mock_get_provider.return_value = provider
463+ execute_run = launcher.return_value.execute_run
464+ execute_run.return_value = subprocess.CompletedProcess([], 0)
465+ config = dedent(
466+ """
467+ pipeline:
468+ - build
469+
470+ jobs:
471+ build:
472+ plugin: catkin-tools
473+ run-tests: True
474+ series: focal
475+ architectures: amd64
476+ packages: [file, git]
477+ """
478+ )
479+ Path(".launchpad.yaml").write_text(config)
480+
481+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
482+
483+ self.run_command("run")
484+ self.assertEqual(
485+ [
486+ call(
487+ ["apt", "update"],
488+ cwd=PosixPath("/build/lpcraft/project"),
489+ env=expected_env,
490+ stdout=ANY,
491+ stderr=ANY,
492+ ),
493+ self._get_ros_packages_call(
494+ expected_env, ["python3-catkin-tools", "file", "git"]
495+ ),
496+ self._get_ros_before_run_call(expected_env),
497+ call(
498+ [
499+ "bash",
500+ "--noprofile",
501+ "--norc",
502+ "-ec",
503+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
504+ "catkin init --workspace ${ROS_PROJECT_BASE}\n"
505+ "catkin profile add --force lpcraft\n"
506+ "catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}\n" # noqa:E501
507+ "catkin build --no-notify --profile lpcraft", # noqa:E501
508+ ],
509+ cwd=PosixPath("/build/lpcraft/project"),
510+ env=expected_env,
511+ stdout=ANY,
512+ stderr=ANY,
513+ ),
514+ call(
515+ [
516+ "bash",
517+ "--noprofile",
518+ "--norc",
519+ "-ec",
520+ "\nsource ${ROS_PROJECT_BASE}/devel/setup.sh\n"
521+ "catkin test --profile lpcraft --summary",
522+ ],
523+ cwd=PosixPath("/build/lpcraft/project"),
524+ env=expected_env,
525+ stdout=ANY,
526+ stderr=ANY,
527+ ),
528+ ],
529+ execute_run.call_args_list,
530+ )
531+
532+ @patch("lpcraft.commands.run.get_provider")
533+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
534+ def test_catkin_tools_plugin_user_command(
535+ self, mock_get_host_architecture, mock_get_provider
536+ ):
537+ launcher = Mock(spec=launch)
538+ provider = makeLXDProvider(lxd_launcher=launcher)
539+ mock_get_provider.return_value = provider
540+ execute_run = launcher.return_value.execute_run
541+ execute_run.return_value = subprocess.CompletedProcess([], 0)
542+ config = dedent(
543+ """
544+ pipeline:
545+ - build
546+
547+ jobs:
548+ build:
549+ plugin: catkin-tools
550+ run-tests: True
551+ series: focal
552+ architectures: amd64
553+ packages: [file, git]
554+ run-before: echo 'hello'
555+ run: echo 'robot'
556+ # run-after: "we shouldn't get anything unless uncommented"
557+ """
558+ )
559+ Path(".launchpad.yaml").write_text(config)
560+
561+ expected_env = self._get_ros_expected_env("noetic", "3", "1")
562+
563+ self.run_command("run")
564+ self.assertEqual(
565+ [
566+ call(
567+ ["apt", "update"],
568+ cwd=PosixPath("/build/lpcraft/project"),
569+ env=expected_env,
570+ stdout=ANY,
571+ stderr=ANY,
572+ ),
573+ self._get_ros_packages_call(
574+ expected_env, ["python3-catkin-tools", "file", "git"]
575+ ),
576+ call(
577+ ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
578+ cwd=PosixPath("/build/lpcraft/project"),
579+ env=expected_env,
580+ stdout=ANY,
581+ stderr=ANY,
582+ ),
583+ call(
584+ ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
585+ cwd=PosixPath("/build/lpcraft/project"),
586+ env=expected_env,
587+ stdout=ANY,
588+ stderr=ANY,
589+ ),
590+ ],
591+ execute_run.call_args_list,
592+ )
593+
594+ @patch("lpcraft.commands.run.get_provider")
595+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
596+ def test_colcon_plugin(
597+ self, mock_get_host_architecture, mock_get_provider
598+ ):
599+ launcher = Mock(spec=launch)
600+ provider = makeLXDProvider(lxd_launcher=launcher)
601+ mock_get_provider.return_value = provider
602+ execute_run = launcher.return_value.execute_run
603+ execute_run.return_value = subprocess.CompletedProcess([], 0)
604+ config = dedent(
605+ """
606+ pipeline:
607+ - build
608+
609+ jobs:
610+ build:
611+ plugin: colcon
612+ series: focal
613+ architectures: amd64
614+ """
615+ )
616+ Path(".launchpad.yaml").write_text(config)
617+
618+ expected_env = self._get_ros_expected_env("foxy", "3", "2")
619+
620+ self.run_command("run")
621+ self.assertEqual(
622+ [
623+ call(
624+ ["apt", "update"],
625+ cwd=PosixPath("/build/lpcraft/project"),
626+ env=expected_env,
627+ stdout=ANY,
628+ stderr=ANY,
629+ ),
630+ self._get_ros_packages_call(
631+ expected_env, ["python3-colcon-common-extensions"]
632+ ),
633+ self._get_ros_before_run_call(expected_env),
634+ call(
635+ [
636+ "bash",
637+ "--noprofile",
638+ "--norc",
639+ "-ec",
640+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
641+ "colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+", # noqa:E501
642+ ],
643+ cwd=PosixPath("/build/lpcraft/project"),
644+ env=expected_env,
645+ stdout=ANY,
646+ stderr=ANY,
647+ ),
648+ ],
649+ execute_run.call_args_list,
650+ )
651+
652+ @patch("lpcraft.commands.run.get_provider")
653+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
654+ def test_colcon_plugin_with_tests(
655+ self, mock_get_host_architecture, mock_get_provider
656+ ):
657+ launcher = Mock(spec=launch)
658+ provider = makeLXDProvider(lxd_launcher=launcher)
659+ mock_get_provider.return_value = provider
660+ execute_run = launcher.return_value.execute_run
661+ execute_run.return_value = subprocess.CompletedProcess([], 0)
662+ config = dedent(
663+ """
664+ pipeline:
665+ - build
666+
667+ jobs:
668+ build:
669+ plugin: colcon
670+ run-tests: True
671+ series: focal
672+ architectures: amd64
673+ packages: [file, git]
674+ """
675+ )
676+ Path(".launchpad.yaml").write_text(config)
677+
678+ expected_env = self._get_ros_expected_env("foxy", "3", "2")
679+
680+ self.run_command("run")
681+ self.assertEqual(
682+ [
683+ call(
684+ ["apt", "update"],
685+ cwd=PosixPath("/build/lpcraft/project"),
686+ env=expected_env,
687+ stdout=ANY,
688+ stderr=ANY,
689+ ),
690+ self._get_ros_packages_call(
691+ expected_env,
692+ ["python3-colcon-common-extensions", "file", "git"],
693+ ),
694+ self._get_ros_before_run_call(expected_env),
695+ call(
696+ [
697+ "bash",
698+ "--noprofile",
699+ "--norc",
700+ "-ec",
701+ "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
702+ "colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+", # noqa:E501
703+ ],
704+ cwd=PosixPath("/build/lpcraft/project"),
705+ env=expected_env,
706+ stdout=ANY,
707+ stderr=ANY,
708+ ),
709+ call(
710+ [
711+ "bash",
712+ "--noprofile",
713+ "--norc",
714+ "-ec",
715+ "\nsource ${ROS_PROJECT_INSTALL}/setup.sh\n"
716+ "colcon test --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+\n" # noqa:E501
717+ "colcon test-result --all --verbose --test-result-base ${ROS_PROJECT_BUILD}", # noqa:E501
718+ ],
719+ cwd=PosixPath("/build/lpcraft/project"),
720+ env=expected_env,
721+ stdout=ANY,
722+ stderr=ANY,
723+ ),
724+ ],
725+ execute_run.call_args_list,
726+ )
727+
728+ @patch("lpcraft.commands.run.get_provider")
729+ @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
730+ def test_colcon_plugin_user_command(
731+ self, mock_get_host_architecture, mock_get_provider
732+ ):
733+ launcher = Mock(spec=launch)
734+ provider = makeLXDProvider(lxd_launcher=launcher)
735+ mock_get_provider.return_value = provider
736+ execute_run = launcher.return_value.execute_run
737+ execute_run.return_value = subprocess.CompletedProcess([], 0)
738+ config = dedent(
739+ """
740+ pipeline:
741+ - build
742+
743+ jobs:
744+ build:
745+ plugin: colcon
746+ run-tests: True
747+ series: focal
748+ architectures: amd64
749+ packages: [file, git]
750+ run-before: echo 'hello'
751+ run: echo 'robot'
752+ # run-after: "we shouldn't get anything unless uncommented"
753+ """
754+ )
755+ Path(".launchpad.yaml").write_text(config)
756+
757+ expected_env = self._get_ros_expected_env("foxy", "3", "2")
758+
759+ self.run_command("run")
760+ self.assertEqual(
761+ [
762+ call(
763+ ["apt", "update"],
764+ cwd=PosixPath("/build/lpcraft/project"),
765+ env=expected_env,
766+ stdout=ANY,
767+ stderr=ANY,
768+ ),
769+ self._get_ros_packages_call(
770+ expected_env,
771+ ["python3-colcon-common-extensions", "file", "git"],
772+ ),
773+ call(
774+ ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
775+ cwd=PosixPath("/build/lpcraft/project"),
776+ env=expected_env,
777+ stdout=ANY,
778+ stderr=ANY,
779+ ),
780+ call(
781+ ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
782+ cwd=PosixPath("/build/lpcraft/project"),
783+ env=expected_env,
784+ stdout=ANY,
785+ stderr=ANY,
786+ ),
787+ ],
788+ execute_run.call_args_list,
789+ )
790diff --git a/lpcraft/plugins/plugins.py b/lpcraft/plugins/plugins.py
791index d4464ee..2a6e41e 100644
792--- a/lpcraft/plugins/plugins.py
793+++ b/lpcraft/plugins/plugins.py
794@@ -9,6 +9,9 @@ __all__ = [
795 "MiniCondaPlugin",
796 "CondaBuildPlugin",
797 "GolangPlugin",
798+ "CatkinPlugin",
799+ "CatkinToolsPlugin",
800+ "ColconPlugin",
801 ]
802
803 import textwrap
804@@ -18,6 +21,7 @@ from typing import TYPE_CHECKING, ClassVar, Dict, List, Optional, cast
805 import pydantic
806 from pydantic import StrictStr
807
808+from lpcraft.env import get_managed_environment_project_path
809 from lpcraft.plugin import hookimpl
810 from lpcraft.plugins import register
811
812@@ -451,3 +455,275 @@ class GolangPlugin(BasePlugin):
813 export PATH=/usr/lib/go-{version}/bin/:$PATH
814 {run_command}"""
815 )
816+
817+
818+class RosBasePlugin(BasePlugin):
819+ """A base class for ROS-related plugins."""
820+
821+ _MAP_SERIES_TO_ROSDISTRO = {
822+ "xenial": "kinetic",
823+ "bionic": "melodic",
824+ "focal": "noetic",
825+ }
826+
827+ _MAP_SERIES_TO_ROS2DISTRO = {
828+ "bionic": "dashing",
829+ "focal": "foxy",
830+ "jammy": "humble",
831+ }
832+
833+ class Config(BaseConfig):
834+ run_tests: Optional[bool]
835+
836+ def get_plugin_config(self) -> "RosBasePlugin.Config":
837+ return cast(RosBasePlugin.Config, self.config.plugin_config)
838+
839+ def _get_ros_version(self) -> int:
840+ raise NotImplementedError
841+
842+ def _get_ros_distro(self) -> str:
843+ """Get the ROS distro associated to the Ubuntu series in use."""
844+ if self._get_ros_version() == 1:
845+ return self._MAP_SERIES_TO_ROSDISTRO[self.config.series]
846+ elif self._get_ros_version() == 2:
847+ return self._MAP_SERIES_TO_ROS2DISTRO[self.config.series]
848+ else:
849+ raise Exception("Unknown ROS version.")
850+
851+ def _get_project_path(self) -> Path:
852+ """Get the project path."""
853+ return get_managed_environment_project_path()
854+
855+ def _get_build_path(self) -> Path:
856+ """Get the out-of-source build path."""
857+ return get_managed_environment_project_path().parent / "build"
858+
859+ def _get_install_path(self) -> Path:
860+ """Get the out-of-source install path."""
861+ return get_managed_environment_project_path().parent / "install"
862+
863+ def _get_ros_workspace_activation(self) -> str:
864+ """Get the ROS system workspace activation command."""
865+ # There should be only one ROS distro installed at any time
866+ # therefore let's make use of the wildcard
867+ return "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
868+
869+ @hookimpl # type: ignore
870+ def lpcraft_set_environment(self) -> dict[str, str | None]:
871+ ros_distro = self._get_ros_distro()
872+ python_version = (
873+ "3" if ros_distro not in ["kinetic", "melodic"] else "2"
874+ )
875+ return {
876+ "ROS_DISTRO": ros_distro,
877+ "ROS_PROJECT_BASE": str(self._get_project_path().parent),
878+ "ROS_PROJECT_SRC": str(self._get_project_path()),
879+ "ROS_PROJECT_BUILD": str(self._get_build_path()),
880+ "ROS_PROJECT_INSTALL": str(self._get_install_path()),
881+ "ROS_PYTHON_VERSION": python_version,
882+ "ROS_VERSION": f"{self._get_ros_version()}",
883+ }
884+
885+ @hookimpl # type: ignore
886+ def lpcraft_install_packages(self) -> list[str]:
887+ return ["build-essential", "cmake", "g++", "python3-rosdep"]
888+
889+ @hookimpl # type: ignore
890+ def lpcraft_execute_before_run(self) -> str:
891+ return self._get_ros_workspace_activation() + textwrap.dedent(
892+ """
893+ if [ ! -f /etc/ros/rosdep/sources.list.d/20-default.list ]; then
894+ rosdep init
895+ fi
896+ rosdep update --rosdistro ${ROS_DISTRO}
897+ rosdep install -i -y --rosdistro ${ROS_DISTRO} --from-paths ."""
898+ )
899+
900+
901+@register(name="catkin")
902+class CatkinPlugin(RosBasePlugin):
903+ """Installs ROS dependencies, builds and (optionally) tests with catkin.
904+
905+ Usage:
906+ In `.launchpad.yaml`, create the following structure.
907+
908+ .. code-block:: yaml
909+
910+ pipeline:
911+ - build
912+
913+ jobs:
914+ build:
915+ plugin: catkin
916+ run-tests: True # optional
917+ series: focal
918+ architectures: amd64
919+ packages: [file, git]
920+ package-repositories:
921+ - components: [main]
922+ formats: [deb]
923+ suites: [focal]
924+ type: apt
925+ url: http://packages.ros.org/ros/ubuntu
926+ trusted: True
927+
928+ Please note that the ROS repository must be
929+ set up in `package-repositories`.
930+ """
931+
932+ def _get_ros_version(self) -> int:
933+ return 1
934+
935+ @hookimpl # type: ignore
936+ def lpcraft_install_packages(self) -> list[str]:
937+ # XXX artivis 2022-12-9: mypy is struggling with the super() call
938+ packages: list[str] = super().lpcraft_install_packages() + [
939+ # "ros-${ROS_DISTRO}-catkin"
940+ f"ros-{self._get_ros_distro()}-catkin"
941+ ]
942+ return packages
943+
944+ @hookimpl # type: ignore
945+ def lpcraft_execute_run(self) -> str:
946+ return self._get_ros_workspace_activation() + textwrap.dedent(
947+ """
948+ catkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}""" # noqa:E501
949+ )
950+
951+ @hookimpl # type: ignore
952+ def lpcraft_execute_after_run(self) -> str:
953+ if not self.get_plugin_config().run_tests or self.config.run:
954+ return ""
955+
956+ return textwrap.dedent(
957+ """
958+ source ${ROS_PROJECT_BASE}/devel_isolated/setup.sh
959+ catkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE} --force-cmake --catkin-make-args run_tests
960+ catkin_test_results --verbose ${ROS_PROJECT_BASE}/build_isolated/""" # noqa:E501
961+ )
962+
963+
964+@register(name="catkin-tools")
965+class CatkinToolsPlugin(RosBasePlugin):
966+ """Installs ROS dependencies, builds and (optionally) tests with catkin-tools.
967+
968+ Usage:
969+ In `.launchpad.yaml`, create the following structure.
970+
971+ .. code-block:: yaml
972+
973+ pipeline:
974+ - build
975+
976+ jobs:
977+ build:
978+ plugin: catkin-tools
979+ run-tests: True # optional
980+ series: focal
981+ architectures: amd64
982+ packages: [file, git]
983+ package-repositories:
984+ - components: [main]
985+ formats: [deb]
986+ suites: [focal]
987+ type: apt
988+ url: http://packages.ros.org/ros/ubuntu
989+ trusted: True
990+
991+ Please note that the ROS repository must be
992+ set up in `package-repositories`.
993+ """
994+
995+ def _get_ros_version(self) -> int:
996+ return 1
997+
998+ @hookimpl # type: ignore
999+ def lpcraft_install_packages(self) -> list[str]:
1000+ # XXX artivis 2022-12-9: mypy is struggling with the super() call
1001+ packages: list[str] = super().lpcraft_install_packages() + [
1002+ "python3-catkin-tools"
1003+ ]
1004+ return packages
1005+
1006+ @hookimpl # type: ignore
1007+ def lpcraft_execute_run(self) -> str:
1008+ return self._get_ros_workspace_activation() + textwrap.dedent(
1009+ """
1010+ catkin init --workspace ${ROS_PROJECT_BASE}
1011+ catkin profile add --force lpcraft
1012+ catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}
1013+ catkin build --no-notify --profile lpcraft""" # noqa:E501
1014+ )
1015+
1016+ @hookimpl # type: ignore
1017+ def lpcraft_execute_after_run(self) -> str:
1018+ if not self.get_plugin_config().run_tests or self.config.run:
1019+ return ""
1020+
1021+ return textwrap.dedent(
1022+ """
1023+ source ${ROS_PROJECT_BASE}/devel/setup.sh
1024+ catkin test --profile lpcraft --summary"""
1025+ )
1026+
1027+
1028+@register(name="colcon")
1029+class ColconPlugin(RosBasePlugin):
1030+ """Installs ROS dependencies, builds and (optionally) tests with colcon.
1031+
1032+ Usage:
1033+ In `.launchpad.yaml`, create the following structure.
1034+
1035+ .. code-block:: yaml
1036+
1037+ pipeline:
1038+ - build
1039+
1040+ jobs:
1041+ build:
1042+ plugin: colcon
1043+ run-tests: True # optional
1044+ series: focal
1045+ architectures: amd64
1046+ packages: [file, git]
1047+ package-repositories:
1048+ - components: [main]
1049+ formats: [deb]
1050+ suites: [focal]
1051+ type: apt
1052+ url: http://repo.ros2.org/ubuntu/main/
1053+ trusted: True
1054+
1055+ Please note that the ROS 2 repository must be
1056+ set up in `package-repositories`.
1057+ """
1058+
1059+ def _get_ros_version(self) -> int:
1060+ return 2
1061+
1062+ @hookimpl # type: ignore
1063+ def lpcraft_install_packages(self) -> list[str]:
1064+ # XXX artivis 2022-12-9: mypy is struggling with the super() call
1065+ packages: list[str] = super().lpcraft_install_packages() + [
1066+ "python3-colcon-common-extensions"
1067+ ]
1068+ return packages
1069+
1070+ @hookimpl # type: ignore
1071+ def lpcraft_execute_run(self) -> str:
1072+ return self._get_ros_workspace_activation() + textwrap.dedent(
1073+ """
1074+ colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+""" # noqa:E501
1075+ )
1076+
1077+ @hookimpl # type: ignore
1078+ def lpcraft_execute_after_run(self) -> str:
1079+ if not self.get_plugin_config().run_tests or self.config.run:
1080+ return ""
1081+
1082+ return textwrap.dedent(
1083+ """
1084+ source ${ROS_PROJECT_INSTALL}/setup.sh
1085+ colcon test --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+
1086+ colcon test-result --all --verbose --test-result-base ${ROS_PROJECT_BUILD}""" # noqa:E501
1087+ )

Subscribers

People subscribed via source and target branches