Merge ~artivis/lpci:feature/ros-plugins into lpci:main

Proposed by jeremie
Status: Needs review
Proposed branch: ~artivis/lpci:feature/ros-plugins
Merge into: lpci:main
Diff against target: 1087 lines (+1041/-2)
2 files modified
lpcraft/plugin/tests/test_plugins.py (+765/-2)
lpcraft/plugins/plugins.py (+276/-0)
Reviewer Review Type Date Requested Status
Jürgen Gmach Approve
Review via email: mp+435371@code.launchpad.net

Description of the change

This implements three new plugins for building and testing ROS & ROS 2 projects, namely,

- CatkinPlugin - based on the 'catkin' build tool
- CatkinToolsPlugin - based on the 'catkin-tools' build tool
- ColconPlugin - based on the 'colcon' build tool

These plugins all share a common base class - RosBasePlugin - factoring some common code.

The three plugins follow the same principle,

- Install the ROS project's dependencies during the 'run-before' step
- Build the ROS project during the 'run' step
- Build and run the tests during the 'run-after' step (optional)

The later is optional and depends on a plugin config parameter.

Beside some ROS-specific environment variables, the base class define the following env. vars.,

- ROS_PROJECT_BASE - the parent directory of the lpcraft build tree
- ROS_PROJECT_SRC - the directory of the project
- ROS_PROJECT_BUILD - the directory of the plugins build tree
- ROS_PROJECT_INSTALL - the directory of the plugins install tree

They somewhat mimic env. vars. in snapcraft allowing for out-of-source build/install. They also allow the end user to more easily override the run commands not having to figure out the internal paths. They could be replaced in the future by lpcraft-wide env. vars.

To post a comment you must log in.
Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Jeremie, thanks for the merge proposal.

CI currently fails as there are two uncovered lines, see buildlog below (below `unmerged commits`):

```
:: lpcraft/plugins/plugins.py 284 2 64 1 99% 482, 491
```

Could you please fix these?

As lpcraft is both the CI runner for Launchpad and a standalone tool, I'd like to know how you are planning to use it. Do you use or plan to use it in Launchpad CI?

I am asking as this would influence the plugin story. For a standalone lpcraft we could easily add setuptools based plugins ( https://pluggy.readthedocs.io/en/stable/#loading-setuptools-entry-points ), so we could have the plugin in a standalone plugin package. For Launchpad CI this is more elaborate, and currently out of scope.

I am discussing these options, as at least for me ROS is a business domain I have never touched before, and including these plugins means I (we) have to maintain them.

Fortunately, the code looks pretty straightforward. I will discuss this MP at today's standup and will get back to you.

Afterwards, I will also give you a detailed review. At very least you would need to add some descriptive docstrings which explain the used business terminology.

Also, does this MP depend on https://code.launchpad.net/~artivis/lpcraft/+git/lpcraft/+merge/435370 or would merging this MP on its own already be helpful?

Revision history for this message
jeremie (artivis) wrote :

Hello,
I've already tackled the missed lines in coverage and the CI is happy as far as I can tell.
We are indeed planning on using those plugins in Launchpad CI.
This MP is stand-alone and totally independent of https://code.launchpad.net/~artivis/lpcraft/+git/lpcraft/+merge/435370 .
Looking forward to your review :+1:

Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Looks good with some minor comments!

As we have no merge bot activated for lpcraft, I think I need to merge the MP.

Just tell me when you think it is ready to be merged.

Note for myself:
- In a follow-up MP we should split the plugins and the tests into submodules.
- In a follow-up MP we should revisit the currently used exceptions.

review: Approve

Unmerged commits

b435946... by jeremie

plugins: add ros plugins

Signed-off-by: artivis <email address hidden>

Succeeded
[SUCCEEDED] test:0 (build)
[SUCCEEDED] build:0 (build)
12 of 2 results

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/lpcraft/plugin/tests/test_plugins.py b/lpcraft/plugin/tests/test_plugins.py
index 1def73b..f7ba1b8 100644
--- a/lpcraft/plugin/tests/test_plugins.py
+++ b/lpcraft/plugin/tests/test_plugins.py
@@ -5,7 +5,7 @@ import os
5import subprocess5import subprocess
6from pathlib import Path, PosixPath6from pathlib import Path, PosixPath
7from textwrap import dedent7from textwrap import dedent
8from typing import List, Optional, Union, cast8from typing import Any, Dict, List, Optional, Union, cast
9from unittest.mock import ANY, Mock, call, patch9from unittest.mock import ANY, Mock, call, patch
1010
11from craft_providers.lxd import launch11from craft_providers.lxd import launch
@@ -17,7 +17,7 @@ from lpcraft.commands.tests import CommandBaseTestCase
17from lpcraft.errors import ConfigurationError17from lpcraft.errors import ConfigurationError
18from lpcraft.plugin.manager import get_plugin_manager18from lpcraft.plugin.manager import get_plugin_manager
19from lpcraft.plugins import register19from lpcraft.plugins import register
20from lpcraft.plugins.plugins import BaseConfig, BasePlugin20from lpcraft.plugins.plugins import BaseConfig, BasePlugin, RosBasePlugin
21from lpcraft.providers.tests import makeLXDProvider21from lpcraft.providers.tests import makeLXDProvider
2222
2323
@@ -1080,3 +1080,766 @@ class TestPlugins(CommandBaseTestCase):
1080 lpcraft.config.Config.load,1080 lpcraft.config.Config.load,
1081 config_path,1081 config_path,
1082 )1082 )
1083
1084 def _get_ros_before_run_call(self, expected_env: Dict[str, str]) -> Any:
1085 return call(
1086 [
1087 "bash",
1088 "--noprofile",
1089 "--norc",
1090 "-ec",
1091 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
1092 "\nif [ ! -f /etc/ros/rosdep/sources.list.d/20-default.list ]; then\nrosdep init\nfi" # noqa:E501
1093 "\nrosdep update --rosdistro ${ROS_DISTRO}"
1094 "\nrosdep install -i -y --rosdistro ${ROS_DISTRO} --from-paths .", # noqa:E501
1095 ],
1096 cwd=PosixPath("/build/lpcraft/project"),
1097 env=expected_env,
1098 stdout=ANY,
1099 stderr=ANY,
1100 )
1101
1102 def _get_ros_packages_call(
1103 self, expected_env: Dict[str, str], packages: List[str]
1104 ) -> Any:
1105 return call(
1106 [
1107 "apt",
1108 "install",
1109 "-y",
1110 "build-essential",
1111 "cmake",
1112 "g++",
1113 "python3-rosdep",
1114 ]
1115 + packages,
1116 cwd=PosixPath("/build/lpcraft/project"),
1117 env=expected_env,
1118 stdout=ANY,
1119 stderr=ANY,
1120 )
1121
1122 def _get_ros_expected_env(
1123 self, ros_distro: str, ros_python_version: str, ros_version: str
1124 ) -> Dict[str, str]:
1125 return {
1126 "ROS_PROJECT_BASE": "/build/lpcraft",
1127 "ROS_PROJECT_SRC": "/build/lpcraft/project",
1128 "ROS_PROJECT_BUILD": "/build/lpcraft/build",
1129 "ROS_PROJECT_INSTALL": "/build/lpcraft/install",
1130 "ROS_DISTRO": ros_distro,
1131 "ROS_PYTHON_VERSION": ros_python_version,
1132 "ROS_VERSION": ros_version,
1133 }
1134
1135 def test_fake_ros_plugin_no_version(self):
1136 config = dedent(
1137 """
1138 pipeline:
1139 - build
1140
1141 jobs:
1142 build:
1143 plugin: fake-ros-plugin
1144 series: bionic
1145 architectures: amd64
1146 """
1147 )
1148 config_path = Path(".launchpad.yaml")
1149 config_path.write_text(config)
1150
1151 @register(name="fake-ros-plugin")
1152 class FakeRosPlugin(RosBasePlugin):
1153 pass
1154
1155 config_obj = lpcraft.config.Config.load(config_path)
1156 job = config_obj.jobs["build"][0]
1157 pm = get_plugin_manager(job)
1158 plugins = pm.get_plugins()
1159 fake_plugin = [
1160 _ for _ in plugins if _.__class__.__name__ == "FakeRosPlugin"
1161 ]
1162 self.assertEqual(job.plugin, "fake-ros-plugin")
1163 self.assertRaises(NotImplementedError, fake_plugin[0]._get_ros_version)
1164
1165 def test_fake_ros_plugin_bad_version(self):
1166 config = dedent(
1167 """
1168 pipeline:
1169 - build
1170
1171 jobs:
1172 build:
1173 plugin: fake-ros-plugin
1174 series: bionic
1175 architectures: amd64
1176 """
1177 )
1178 config_path = Path(".launchpad.yaml")
1179 config_path.write_text(config)
1180
1181 @register(name="fake-ros-plugin")
1182 class FakeRosPlugin(RosBasePlugin):
1183 def _get_ros_version(self) -> int:
1184 return 3
1185
1186 config_obj = lpcraft.config.Config.load(config_path)
1187 job = config_obj.jobs["build"][0]
1188 pm = get_plugin_manager(job)
1189 plugins = pm.get_plugins()
1190 fake_plugin = [
1191 _ for _ in plugins if _.__class__.__name__ == "FakeRosPlugin"
1192 ]
1193 self.assertEqual(job.plugin, "fake-ros-plugin")
1194 self.assertEqual(3, fake_plugin[0]._get_ros_version())
1195 self.assertRaises(Exception, fake_plugin[0]._get_ros_distro)
1196
1197 @patch("lpcraft.commands.run.get_provider")
1198 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1199 def test_catkin_plugin_bionic(
1200 self, mock_get_host_architecture, mock_get_provider
1201 ):
1202 launcher = Mock(spec=launch)
1203 provider = makeLXDProvider(lxd_launcher=launcher)
1204 mock_get_provider.return_value = provider
1205 execute_run = launcher.return_value.execute_run
1206 execute_run.return_value = subprocess.CompletedProcess([], 0)
1207 config = dedent(
1208 """
1209 pipeline:
1210 - build
1211
1212 jobs:
1213 build:
1214 plugin: catkin
1215 series: bionic
1216 architectures: amd64
1217 """
1218 )
1219 Path(".launchpad.yaml").write_text(config)
1220
1221 expected_env = self._get_ros_expected_env("melodic", "2", "1")
1222
1223 self.run_command("run")
1224 self.assertEqual(
1225 [
1226 call(
1227 ["apt", "update"],
1228 cwd=PosixPath("/build/lpcraft/project"),
1229 env=expected_env,
1230 stdout=ANY,
1231 stderr=ANY,
1232 ),
1233 self._get_ros_packages_call(
1234 expected_env, ["ros-melodic-catkin"]
1235 ),
1236 self._get_ros_before_run_call(expected_env),
1237 call(
1238 [
1239 "bash",
1240 "--noprofile",
1241 "--norc",
1242 "-ec",
1243 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
1244 "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
1245 ],
1246 cwd=PosixPath("/build/lpcraft/project"),
1247 env=expected_env,
1248 stdout=ANY,
1249 stderr=ANY,
1250 ),
1251 ],
1252 execute_run.call_args_list,
1253 )
1254
1255 @patch("lpcraft.commands.run.get_provider")
1256 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1257 def test_catkin_plugin(
1258 self, mock_get_host_architecture, mock_get_provider
1259 ):
1260 launcher = Mock(spec=launch)
1261 provider = makeLXDProvider(lxd_launcher=launcher)
1262 mock_get_provider.return_value = provider
1263 execute_run = launcher.return_value.execute_run
1264 execute_run.return_value = subprocess.CompletedProcess([], 0)
1265 config = dedent(
1266 """
1267 pipeline:
1268 - build
1269
1270 jobs:
1271 build:
1272 plugin: catkin
1273 series: focal
1274 architectures: amd64
1275 """
1276 )
1277 Path(".launchpad.yaml").write_text(config)
1278
1279 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1280
1281 self.run_command("run")
1282 self.assertEqual(
1283 [
1284 call(
1285 ["apt", "update"],
1286 cwd=PosixPath("/build/lpcraft/project"),
1287 env=expected_env,
1288 stdout=ANY,
1289 stderr=ANY,
1290 ),
1291 self._get_ros_packages_call(
1292 expected_env, ["ros-noetic-catkin"]
1293 ),
1294 self._get_ros_before_run_call(expected_env),
1295 call(
1296 [
1297 "bash",
1298 "--noprofile",
1299 "--norc",
1300 "-ec",
1301 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
1302 "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
1303 ],
1304 cwd=PosixPath("/build/lpcraft/project"),
1305 env=expected_env,
1306 stdout=ANY,
1307 stderr=ANY,
1308 ),
1309 ],
1310 execute_run.call_args_list,
1311 )
1312
1313 @patch("lpcraft.commands.run.get_provider")
1314 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1315 def test_catkin_plugin_with_tests(
1316 self, mock_get_host_architecture, mock_get_provider
1317 ):
1318 launcher = Mock(spec=launch)
1319 provider = makeLXDProvider(lxd_launcher=launcher)
1320 mock_get_provider.return_value = provider
1321 execute_run = launcher.return_value.execute_run
1322 execute_run.return_value = subprocess.CompletedProcess([], 0)
1323 config = dedent(
1324 """
1325 pipeline:
1326 - build
1327
1328 jobs:
1329 build:
1330 plugin: catkin
1331 run-tests: True
1332 series: focal
1333 architectures: amd64
1334 packages: [file, git]
1335 """
1336 )
1337 Path(".launchpad.yaml").write_text(config)
1338
1339 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1340
1341 self.run_command("run")
1342 self.assertEqual(
1343 [
1344 call(
1345 ["apt", "update"],
1346 cwd=PosixPath("/build/lpcraft/project"),
1347 env=expected_env,
1348 stdout=ANY,
1349 stderr=ANY,
1350 ),
1351 self._get_ros_packages_call(
1352 expected_env, ["ros-noetic-catkin", "file", "git"]
1353 ),
1354 self._get_ros_before_run_call(expected_env),
1355 call(
1356 [
1357 "bash",
1358 "--noprofile",
1359 "--norc",
1360 "-ec",
1361 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
1362 "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}", # noqa:E501
1363 ],
1364 cwd=PosixPath("/build/lpcraft/project"),
1365 env=expected_env,
1366 stdout=ANY,
1367 stderr=ANY,
1368 ),
1369 call(
1370 [
1371 "bash",
1372 "--noprofile",
1373 "--norc",
1374 "-ec",
1375 "\nsource ${ROS_PROJECT_BASE}/devel_isolated/setup.sh"
1376 "\ncatkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE} --force-cmake --catkin-make-args run_tests" # noqa:E501
1377 "\ncatkin_test_results --verbose ${ROS_PROJECT_BASE}/build_isolated/", # noqa:E501
1378 ],
1379 cwd=PosixPath("/build/lpcraft/project"),
1380 env=expected_env,
1381 stdout=ANY,
1382 stderr=ANY,
1383 ),
1384 ],
1385 execute_run.call_args_list,
1386 )
1387
1388 @patch("lpcraft.commands.run.get_provider")
1389 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1390 def test_catkin_plugin_user_command(
1391 self, mock_get_host_architecture, mock_get_provider
1392 ):
1393 launcher = Mock(spec=launch)
1394 provider = makeLXDProvider(lxd_launcher=launcher)
1395 mock_get_provider.return_value = provider
1396 execute_run = launcher.return_value.execute_run
1397 execute_run.return_value = subprocess.CompletedProcess([], 0)
1398 config = dedent(
1399 """
1400 pipeline:
1401 - build
1402
1403 jobs:
1404 build:
1405 plugin: catkin
1406 run-tests: True
1407 series: focal
1408 architectures: amd64
1409 packages: [file, git]
1410 run-before: echo 'hello'
1411 run: echo 'robot'
1412 # run-after: "we shouldn't get anything unless uncommented"
1413 """
1414 )
1415 Path(".launchpad.yaml").write_text(config)
1416
1417 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1418
1419 self.run_command("run")
1420 self.assertEqual(
1421 [
1422 call(
1423 ["apt", "update"],
1424 cwd=PosixPath("/build/lpcraft/project"),
1425 env=expected_env,
1426 stdout=ANY,
1427 stderr=ANY,
1428 ),
1429 self._get_ros_packages_call(
1430 expected_env, ["ros-noetic-catkin", "file", "git"]
1431 ),
1432 call(
1433 ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
1434 cwd=PosixPath("/build/lpcraft/project"),
1435 env=expected_env,
1436 stdout=ANY,
1437 stderr=ANY,
1438 ),
1439 call(
1440 ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
1441 cwd=PosixPath("/build/lpcraft/project"),
1442 env=expected_env,
1443 stdout=ANY,
1444 stderr=ANY,
1445 ),
1446 ],
1447 execute_run.call_args_list,
1448 )
1449
1450 @patch("lpcraft.commands.run.get_provider")
1451 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1452 def test_catkin_tools_plugin(
1453 self, mock_get_host_architecture, mock_get_provider
1454 ):
1455 launcher = Mock(spec=launch)
1456 provider = makeLXDProvider(lxd_launcher=launcher)
1457 mock_get_provider.return_value = provider
1458 execute_run = launcher.return_value.execute_run
1459 execute_run.return_value = subprocess.CompletedProcess([], 0)
1460 config = dedent(
1461 """
1462 pipeline:
1463 - build
1464
1465 jobs:
1466 build:
1467 plugin: catkin-tools
1468 series: focal
1469 architectures: amd64
1470 """
1471 )
1472 Path(".launchpad.yaml").write_text(config)
1473
1474 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1475
1476 self.run_command("run")
1477 self.assertEqual(
1478 [
1479 call(
1480 ["apt", "update"],
1481 cwd=PosixPath("/build/lpcraft/project"),
1482 env=expected_env,
1483 stdout=ANY,
1484 stderr=ANY,
1485 ),
1486 self._get_ros_packages_call(
1487 expected_env, ["python3-catkin-tools"]
1488 ),
1489 self._get_ros_before_run_call(expected_env),
1490 call(
1491 [
1492 "bash",
1493 "--noprofile",
1494 "--norc",
1495 "-ec",
1496 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
1497 "catkin init --workspace ${ROS_PROJECT_BASE}\n"
1498 "catkin profile add --force lpcraft\n"
1499 "catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}\n" # noqa:E501
1500 "catkin build --no-notify --profile lpcraft", # noqa:E501
1501 ],
1502 cwd=PosixPath("/build/lpcraft/project"),
1503 env=expected_env,
1504 stdout=ANY,
1505 stderr=ANY,
1506 ),
1507 ],
1508 execute_run.call_args_list,
1509 )
1510
1511 @patch("lpcraft.commands.run.get_provider")
1512 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1513 def test_catkin_tools_plugin_with_tests(
1514 self, mock_get_host_architecture, mock_get_provider
1515 ):
1516 launcher = Mock(spec=launch)
1517 provider = makeLXDProvider(lxd_launcher=launcher)
1518 mock_get_provider.return_value = provider
1519 execute_run = launcher.return_value.execute_run
1520 execute_run.return_value = subprocess.CompletedProcess([], 0)
1521 config = dedent(
1522 """
1523 pipeline:
1524 - build
1525
1526 jobs:
1527 build:
1528 plugin: catkin-tools
1529 run-tests: True
1530 series: focal
1531 architectures: amd64
1532 packages: [file, git]
1533 """
1534 )
1535 Path(".launchpad.yaml").write_text(config)
1536
1537 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1538
1539 self.run_command("run")
1540 self.assertEqual(
1541 [
1542 call(
1543 ["apt", "update"],
1544 cwd=PosixPath("/build/lpcraft/project"),
1545 env=expected_env,
1546 stdout=ANY,
1547 stderr=ANY,
1548 ),
1549 self._get_ros_packages_call(
1550 expected_env, ["python3-catkin-tools", "file", "git"]
1551 ),
1552 self._get_ros_before_run_call(expected_env),
1553 call(
1554 [
1555 "bash",
1556 "--noprofile",
1557 "--norc",
1558 "-ec",
1559 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
1560 "catkin init --workspace ${ROS_PROJECT_BASE}\n"
1561 "catkin profile add --force lpcraft\n"
1562 "catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}\n" # noqa:E501
1563 "catkin build --no-notify --profile lpcraft", # noqa:E501
1564 ],
1565 cwd=PosixPath("/build/lpcraft/project"),
1566 env=expected_env,
1567 stdout=ANY,
1568 stderr=ANY,
1569 ),
1570 call(
1571 [
1572 "bash",
1573 "--noprofile",
1574 "--norc",
1575 "-ec",
1576 "\nsource ${ROS_PROJECT_BASE}/devel/setup.sh\n"
1577 "catkin test --profile lpcraft --summary",
1578 ],
1579 cwd=PosixPath("/build/lpcraft/project"),
1580 env=expected_env,
1581 stdout=ANY,
1582 stderr=ANY,
1583 ),
1584 ],
1585 execute_run.call_args_list,
1586 )
1587
1588 @patch("lpcraft.commands.run.get_provider")
1589 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1590 def test_catkin_tools_plugin_user_command(
1591 self, mock_get_host_architecture, mock_get_provider
1592 ):
1593 launcher = Mock(spec=launch)
1594 provider = makeLXDProvider(lxd_launcher=launcher)
1595 mock_get_provider.return_value = provider
1596 execute_run = launcher.return_value.execute_run
1597 execute_run.return_value = subprocess.CompletedProcess([], 0)
1598 config = dedent(
1599 """
1600 pipeline:
1601 - build
1602
1603 jobs:
1604 build:
1605 plugin: catkin-tools
1606 run-tests: True
1607 series: focal
1608 architectures: amd64
1609 packages: [file, git]
1610 run-before: echo 'hello'
1611 run: echo 'robot'
1612 # run-after: "we shouldn't get anything unless uncommented"
1613 """
1614 )
1615 Path(".launchpad.yaml").write_text(config)
1616
1617 expected_env = self._get_ros_expected_env("noetic", "3", "1")
1618
1619 self.run_command("run")
1620 self.assertEqual(
1621 [
1622 call(
1623 ["apt", "update"],
1624 cwd=PosixPath("/build/lpcraft/project"),
1625 env=expected_env,
1626 stdout=ANY,
1627 stderr=ANY,
1628 ),
1629 self._get_ros_packages_call(
1630 expected_env, ["python3-catkin-tools", "file", "git"]
1631 ),
1632 call(
1633 ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
1634 cwd=PosixPath("/build/lpcraft/project"),
1635 env=expected_env,
1636 stdout=ANY,
1637 stderr=ANY,
1638 ),
1639 call(
1640 ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
1641 cwd=PosixPath("/build/lpcraft/project"),
1642 env=expected_env,
1643 stdout=ANY,
1644 stderr=ANY,
1645 ),
1646 ],
1647 execute_run.call_args_list,
1648 )
1649
1650 @patch("lpcraft.commands.run.get_provider")
1651 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1652 def test_colcon_plugin(
1653 self, mock_get_host_architecture, mock_get_provider
1654 ):
1655 launcher = Mock(spec=launch)
1656 provider = makeLXDProvider(lxd_launcher=launcher)
1657 mock_get_provider.return_value = provider
1658 execute_run = launcher.return_value.execute_run
1659 execute_run.return_value = subprocess.CompletedProcess([], 0)
1660 config = dedent(
1661 """
1662 pipeline:
1663 - build
1664
1665 jobs:
1666 build:
1667 plugin: colcon
1668 series: focal
1669 architectures: amd64
1670 """
1671 )
1672 Path(".launchpad.yaml").write_text(config)
1673
1674 expected_env = self._get_ros_expected_env("foxy", "3", "2")
1675
1676 self.run_command("run")
1677 self.assertEqual(
1678 [
1679 call(
1680 ["apt", "update"],
1681 cwd=PosixPath("/build/lpcraft/project"),
1682 env=expected_env,
1683 stdout=ANY,
1684 stderr=ANY,
1685 ),
1686 self._get_ros_packages_call(
1687 expected_env, ["python3-colcon-common-extensions"]
1688 ),
1689 self._get_ros_before_run_call(expected_env),
1690 call(
1691 [
1692 "bash",
1693 "--noprofile",
1694 "--norc",
1695 "-ec",
1696 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
1697 "colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+", # noqa:E501
1698 ],
1699 cwd=PosixPath("/build/lpcraft/project"),
1700 env=expected_env,
1701 stdout=ANY,
1702 stderr=ANY,
1703 ),
1704 ],
1705 execute_run.call_args_list,
1706 )
1707
1708 @patch("lpcraft.commands.run.get_provider")
1709 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1710 def test_colcon_plugin_with_tests(
1711 self, mock_get_host_architecture, mock_get_provider
1712 ):
1713 launcher = Mock(spec=launch)
1714 provider = makeLXDProvider(lxd_launcher=launcher)
1715 mock_get_provider.return_value = provider
1716 execute_run = launcher.return_value.execute_run
1717 execute_run.return_value = subprocess.CompletedProcess([], 0)
1718 config = dedent(
1719 """
1720 pipeline:
1721 - build
1722
1723 jobs:
1724 build:
1725 plugin: colcon
1726 run-tests: True
1727 series: focal
1728 architectures: amd64
1729 packages: [file, git]
1730 """
1731 )
1732 Path(".launchpad.yaml").write_text(config)
1733
1734 expected_env = self._get_ros_expected_env("foxy", "3", "2")
1735
1736 self.run_command("run")
1737 self.assertEqual(
1738 [
1739 call(
1740 ["apt", "update"],
1741 cwd=PosixPath("/build/lpcraft/project"),
1742 env=expected_env,
1743 stdout=ANY,
1744 stderr=ANY,
1745 ),
1746 self._get_ros_packages_call(
1747 expected_env,
1748 ["python3-colcon-common-extensions", "file", "git"],
1749 ),
1750 self._get_ros_before_run_call(expected_env),
1751 call(
1752 [
1753 "bash",
1754 "--noprofile",
1755 "--norc",
1756 "-ec",
1757 "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi\n" # noqa:E501
1758 "colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+", # noqa:E501
1759 ],
1760 cwd=PosixPath("/build/lpcraft/project"),
1761 env=expected_env,
1762 stdout=ANY,
1763 stderr=ANY,
1764 ),
1765 call(
1766 [
1767 "bash",
1768 "--noprofile",
1769 "--norc",
1770 "-ec",
1771 "\nsource ${ROS_PROJECT_INSTALL}/setup.sh\n"
1772 "colcon test --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+\n" # noqa:E501
1773 "colcon test-result --all --verbose --test-result-base ${ROS_PROJECT_BUILD}", # noqa:E501
1774 ],
1775 cwd=PosixPath("/build/lpcraft/project"),
1776 env=expected_env,
1777 stdout=ANY,
1778 stderr=ANY,
1779 ),
1780 ],
1781 execute_run.call_args_list,
1782 )
1783
1784 @patch("lpcraft.commands.run.get_provider")
1785 @patch("lpcraft.commands.run.get_host_architecture", return_value="amd64")
1786 def test_colcon_plugin_user_command(
1787 self, mock_get_host_architecture, mock_get_provider
1788 ):
1789 launcher = Mock(spec=launch)
1790 provider = makeLXDProvider(lxd_launcher=launcher)
1791 mock_get_provider.return_value = provider
1792 execute_run = launcher.return_value.execute_run
1793 execute_run.return_value = subprocess.CompletedProcess([], 0)
1794 config = dedent(
1795 """
1796 pipeline:
1797 - build
1798
1799 jobs:
1800 build:
1801 plugin: colcon
1802 run-tests: True
1803 series: focal
1804 architectures: amd64
1805 packages: [file, git]
1806 run-before: echo 'hello'
1807 run: echo 'robot'
1808 # run-after: "we shouldn't get anything unless uncommented"
1809 """
1810 )
1811 Path(".launchpad.yaml").write_text(config)
1812
1813 expected_env = self._get_ros_expected_env("foxy", "3", "2")
1814
1815 self.run_command("run")
1816 self.assertEqual(
1817 [
1818 call(
1819 ["apt", "update"],
1820 cwd=PosixPath("/build/lpcraft/project"),
1821 env=expected_env,
1822 stdout=ANY,
1823 stderr=ANY,
1824 ),
1825 self._get_ros_packages_call(
1826 expected_env,
1827 ["python3-colcon-common-extensions", "file", "git"],
1828 ),
1829 call(
1830 ["bash", "--noprofile", "--norc", "-ec", "echo 'hello'"],
1831 cwd=PosixPath("/build/lpcraft/project"),
1832 env=expected_env,
1833 stdout=ANY,
1834 stderr=ANY,
1835 ),
1836 call(
1837 ["bash", "--noprofile", "--norc", "-ec", "echo 'robot'"],
1838 cwd=PosixPath("/build/lpcraft/project"),
1839 env=expected_env,
1840 stdout=ANY,
1841 stderr=ANY,
1842 ),
1843 ],
1844 execute_run.call_args_list,
1845 )
diff --git a/lpcraft/plugins/plugins.py b/lpcraft/plugins/plugins.py
index d4464ee..2a6e41e 100644
--- a/lpcraft/plugins/plugins.py
+++ b/lpcraft/plugins/plugins.py
@@ -9,6 +9,9 @@ __all__ = [
9 "MiniCondaPlugin",9 "MiniCondaPlugin",
10 "CondaBuildPlugin",10 "CondaBuildPlugin",
11 "GolangPlugin",11 "GolangPlugin",
12 "CatkinPlugin",
13 "CatkinToolsPlugin",
14 "ColconPlugin",
12]15]
1316
14import textwrap17import textwrap
@@ -18,6 +21,7 @@ from typing import TYPE_CHECKING, ClassVar, Dict, List, Optional, cast
18import pydantic21import pydantic
19from pydantic import StrictStr22from pydantic import StrictStr
2023
24from lpcraft.env import get_managed_environment_project_path
21from lpcraft.plugin import hookimpl25from lpcraft.plugin import hookimpl
22from lpcraft.plugins import register26from lpcraft.plugins import register
2327
@@ -451,3 +455,275 @@ class GolangPlugin(BasePlugin):
451 export PATH=/usr/lib/go-{version}/bin/:$PATH455 export PATH=/usr/lib/go-{version}/bin/:$PATH
452 {run_command}"""456 {run_command}"""
453 )457 )
458
459
460class RosBasePlugin(BasePlugin):
461 """A base class for ROS-related plugins."""
462
463 _MAP_SERIES_TO_ROSDISTRO = {
464 "xenial": "kinetic",
465 "bionic": "melodic",
466 "focal": "noetic",
467 }
468
469 _MAP_SERIES_TO_ROS2DISTRO = {
470 "bionic": "dashing",
471 "focal": "foxy",
472 "jammy": "humble",
473 }
474
475 class Config(BaseConfig):
476 run_tests: Optional[bool]
477
478 def get_plugin_config(self) -> "RosBasePlugin.Config":
479 return cast(RosBasePlugin.Config, self.config.plugin_config)
480
481 def _get_ros_version(self) -> int:
482 raise NotImplementedError
483
484 def _get_ros_distro(self) -> str:
485 """Get the ROS distro associated to the Ubuntu series in use."""
486 if self._get_ros_version() == 1:
487 return self._MAP_SERIES_TO_ROSDISTRO[self.config.series]
488 elif self._get_ros_version() == 2:
489 return self._MAP_SERIES_TO_ROS2DISTRO[self.config.series]
490 else:
491 raise Exception("Unknown ROS version.")
492
493 def _get_project_path(self) -> Path:
494 """Get the project path."""
495 return get_managed_environment_project_path()
496
497 def _get_build_path(self) -> Path:
498 """Get the out-of-source build path."""
499 return get_managed_environment_project_path().parent / "build"
500
501 def _get_install_path(self) -> Path:
502 """Get the out-of-source install path."""
503 return get_managed_environment_project_path().parent / "install"
504
505 def _get_ros_workspace_activation(self) -> str:
506 """Get the ROS system workspace activation command."""
507 # There should be only one ROS distro installed at any time
508 # therefore let's make use of the wildcard
509 return "if [ -f /opt/ros/*/setup.sh ]; then source /opt/ros/*/setup.sh; fi" # noqa:E501
510
511 @hookimpl # type: ignore
512 def lpcraft_set_environment(self) -> dict[str, str | None]:
513 ros_distro = self._get_ros_distro()
514 python_version = (
515 "3" if ros_distro not in ["kinetic", "melodic"] else "2"
516 )
517 return {
518 "ROS_DISTRO": ros_distro,
519 "ROS_PROJECT_BASE": str(self._get_project_path().parent),
520 "ROS_PROJECT_SRC": str(self._get_project_path()),
521 "ROS_PROJECT_BUILD": str(self._get_build_path()),
522 "ROS_PROJECT_INSTALL": str(self._get_install_path()),
523 "ROS_PYTHON_VERSION": python_version,
524 "ROS_VERSION": f"{self._get_ros_version()}",
525 }
526
527 @hookimpl # type: ignore
528 def lpcraft_install_packages(self) -> list[str]:
529 return ["build-essential", "cmake", "g++", "python3-rosdep"]
530
531 @hookimpl # type: ignore
532 def lpcraft_execute_before_run(self) -> str:
533 return self._get_ros_workspace_activation() + textwrap.dedent(
534 """
535 if [ ! -f /etc/ros/rosdep/sources.list.d/20-default.list ]; then
536 rosdep init
537 fi
538 rosdep update --rosdistro ${ROS_DISTRO}
539 rosdep install -i -y --rosdistro ${ROS_DISTRO} --from-paths ."""
540 )
541
542
543@register(name="catkin")
544class CatkinPlugin(RosBasePlugin):
545 """Installs ROS dependencies, builds and (optionally) tests with catkin.
546
547 Usage:
548 In `.launchpad.yaml`, create the following structure.
549
550 .. code-block:: yaml
551
552 pipeline:
553 - build
554
555 jobs:
556 build:
557 plugin: catkin
558 run-tests: True # optional
559 series: focal
560 architectures: amd64
561 packages: [file, git]
562 package-repositories:
563 - components: [main]
564 formats: [deb]
565 suites: [focal]
566 type: apt
567 url: http://packages.ros.org/ros/ubuntu
568 trusted: True
569
570 Please note that the ROS repository must be
571 set up in `package-repositories`.
572 """
573
574 def _get_ros_version(self) -> int:
575 return 1
576
577 @hookimpl # type: ignore
578 def lpcraft_install_packages(self) -> list[str]:
579 # XXX artivis 2022-12-9: mypy is struggling with the super() call
580 packages: list[str] = super().lpcraft_install_packages() + [
581 # "ros-${ROS_DISTRO}-catkin"
582 f"ros-{self._get_ros_distro()}-catkin"
583 ]
584 return packages
585
586 @hookimpl # type: ignore
587 def lpcraft_execute_run(self) -> str:
588 return self._get_ros_workspace_activation() + textwrap.dedent(
589 """
590 catkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE}""" # noqa:E501
591 )
592
593 @hookimpl # type: ignore
594 def lpcraft_execute_after_run(self) -> str:
595 if not self.get_plugin_config().run_tests or self.config.run:
596 return ""
597
598 return textwrap.dedent(
599 """
600 source ${ROS_PROJECT_BASE}/devel_isolated/setup.sh
601 catkin_make_isolated --source-space ${ROS_PROJECT_SRC} --directory ${ROS_PROJECT_BASE} --force-cmake --catkin-make-args run_tests
602 catkin_test_results --verbose ${ROS_PROJECT_BASE}/build_isolated/""" # noqa:E501
603 )
604
605
606@register(name="catkin-tools")
607class CatkinToolsPlugin(RosBasePlugin):
608 """Installs ROS dependencies, builds and (optionally) tests with catkin-tools.
609
610 Usage:
611 In `.launchpad.yaml`, create the following structure.
612
613 .. code-block:: yaml
614
615 pipeline:
616 - build
617
618 jobs:
619 build:
620 plugin: catkin-tools
621 run-tests: True # optional
622 series: focal
623 architectures: amd64
624 packages: [file, git]
625 package-repositories:
626 - components: [main]
627 formats: [deb]
628 suites: [focal]
629 type: apt
630 url: http://packages.ros.org/ros/ubuntu
631 trusted: True
632
633 Please note that the ROS repository must be
634 set up in `package-repositories`.
635 """
636
637 def _get_ros_version(self) -> int:
638 return 1
639
640 @hookimpl # type: ignore
641 def lpcraft_install_packages(self) -> list[str]:
642 # XXX artivis 2022-12-9: mypy is struggling with the super() call
643 packages: list[str] = super().lpcraft_install_packages() + [
644 "python3-catkin-tools"
645 ]
646 return packages
647
648 @hookimpl # type: ignore
649 def lpcraft_execute_run(self) -> str:
650 return self._get_ros_workspace_activation() + textwrap.dedent(
651 """
652 catkin init --workspace ${ROS_PROJECT_BASE}
653 catkin profile add --force lpcraft
654 catkin config --profile lpcraft --install --source-space ${ROS_PROJECT_SRC} --build-space ${ROS_PROJECT_BUILD} --install-space ${ROS_PROJECT_INSTALL}
655 catkin build --no-notify --profile lpcraft""" # noqa:E501
656 )
657
658 @hookimpl # type: ignore
659 def lpcraft_execute_after_run(self) -> str:
660 if not self.get_plugin_config().run_tests or self.config.run:
661 return ""
662
663 return textwrap.dedent(
664 """
665 source ${ROS_PROJECT_BASE}/devel/setup.sh
666 catkin test --profile lpcraft --summary"""
667 )
668
669
670@register(name="colcon")
671class ColconPlugin(RosBasePlugin):
672 """Installs ROS dependencies, builds and (optionally) tests with colcon.
673
674 Usage:
675 In `.launchpad.yaml`, create the following structure.
676
677 .. code-block:: yaml
678
679 pipeline:
680 - build
681
682 jobs:
683 build:
684 plugin: colcon
685 run-tests: True # optional
686 series: focal
687 architectures: amd64
688 packages: [file, git]
689 package-repositories:
690 - components: [main]
691 formats: [deb]
692 suites: [focal]
693 type: apt
694 url: http://repo.ros2.org/ubuntu/main/
695 trusted: True
696
697 Please note that the ROS 2 repository must be
698 set up in `package-repositories`.
699 """
700
701 def _get_ros_version(self) -> int:
702 return 2
703
704 @hookimpl # type: ignore
705 def lpcraft_install_packages(self) -> list[str]:
706 # XXX artivis 2022-12-9: mypy is struggling with the super() call
707 packages: list[str] = super().lpcraft_install_packages() + [
708 "python3-colcon-common-extensions"
709 ]
710 return packages
711
712 @hookimpl # type: ignore
713 def lpcraft_execute_run(self) -> str:
714 return self._get_ros_workspace_activation() + textwrap.dedent(
715 """
716 colcon build --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+""" # noqa:E501
717 )
718
719 @hookimpl # type: ignore
720 def lpcraft_execute_after_run(self) -> str:
721 if not self.get_plugin_config().run_tests or self.config.run:
722 return ""
723
724 return textwrap.dedent(
725 """
726 source ${ROS_PROJECT_INSTALL}/setup.sh
727 colcon test --base-paths ${ROS_PROJECT_SRC} --build-base ${ROS_PROJECT_BUILD} --install-base ${ROS_PROJECT_INSTALL} --event-handlers console_direct+
728 colcon test-result --all --verbose --test-result-base ${ROS_PROJECT_BUILD}""" # noqa:E501
729 )

Subscribers

People subscribed via source and target branches