Merge ~liaou3/checkbox-iiotg/+git/checkbox-provider-intliotg:refactor_tsn_test_plan into ~checkbox-dev/checkbox-iiotg/+git/checkbox-provider-intliotg:master

[Vincent Liao]

Refactor TSN testplan from manual to semi-automated

# Description
All of these test cases is based on the TSN testing document[1] provided by Intel, and was simplified into the testing document[2], then I created those test jobs based on that. You could read those document if you have time (I know the document is too long to read). To put it simply, I created jobs based on each protocal in TSN.

# Difference
In the past we ran tsn tests as manual jobs
 - May be human error exist -> Hard to unified the test method
 - No log left -> Hard to track the error
Now we run tsn as semi-automated jobs
 - Every tester run the same commands -> Unify the test method
 - There's log left -> Make review easier
 - Test cases categorized into each protocal -> Make tester easy to test each protocal

# TODO
I know that we could run these tsn tests fully automated. However I would like to make this MR go into the main branch first. After that I will slowly improve the test cases from semi-automated to fully-automated.

# Submissions:
https://certification.canonical.com/hardware/202305-31589/submission/332830/

# References
[1] https://drive.google.com/file/d/1McvVL5Xo5CO2fHsqkEV6L6KUTVPcGss9/view
[2] https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit?pli=1#heading=h.cqrgvanzectb

[Rick Wu (rickwu4444)]

@Vincent,
I have one minor suggestion. Otherwise LGTM.

review: Needs Information

[Vincent Liao (liaou3)]

> @Vincent,
> I have one minor suggestion. Otherwise LGTM.
@Rick
Fixed!

[Rick Wu (rickwu4444)]

LGTM

review: Approve

[PeiYao Chang (baconyao)]

There are four duplicated paths of "/usr/share/gPTP.cfg" in tsn_time_sync.sh, I suggest replace them with a variable.

review: Needs Fixing

[Vincent Liao (liaou3)]

@Patrick
Fixed.

[Vincent Liao (liaou3)]

@Patrick
Thanks for your review, please see my latest commit.
I will raise another MR for checkbox-iiotg-classic to add the config variable `GPTP_CONFIG_PATH` into config_vars.

[PeiYao Chang (baconyao)]

LGTM

review: Approve

diff --git a/bin/tsn_test_env_setup.sh b/bin/tsn_test_env_setup.sh
new file mode 100755
index 0000000..4eaefbe
--- /dev/null
+++ b/bin/tsn_test_env_setup.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+
+set -e
+
+interface=$1
+gptp_config_path=$2
+
+echo "## Creating gPTP.cfg to $gptp_config_path..."
+echo "[global]
+gmCapable           	1
+priority1           	248
+priority2           	248
+logAnnounceInterval 	0
+logSyncInterval     	-3
+syncReceiptTimeout  	3
+neighborPropDelayThresh 800
+min_neighbor_prop_delay -20000000
+assume_two_step     	1
+path_trace_enabled  	1
+follow_up_info      	1
+transportSpecific   	0x1
+ptp_dst_mac         	01:80:C2:00:00:0E
+network_transport   	L2
+delay_mechanism     	P2P
+tx_timestamp_timeout	100" > "$gptp_config_path"
+
+echo "## Disabling EEE status to prevent the network adapter from entering EEE mode..."
+ethtool --set-eee "$interface" eee off
+echo "## Displaying EEE status..."
+ethtool --show-eee "$interface"
+
+echo "## Disabling pause options to prevent from traffic interruption..."
+ethtool -A "$interface" autoneg off rx off tx off
+
+echo "## Disabling NTP clock sync to prevent interrupt with clock sync between boards..."
+timedatectl set-ntp 0
diff --git a/bin/tsn_time_sync.sh b/bin/tsn_time_sync.sh
new file mode 100755
index 0000000..c8e3ce9
--- /dev/null
+++ b/bin/tsn_time_sync.sh
@@ -0,0 +1,27 @@
+#!/bin/bash
+
+set -e
+
+timeout=$1
+mode=$2
+config_path=$3
+eth_interface=$4
+server_user=$5
+server_ip=$6
+server_pwd=$7
+server_eth_interface=$8
+
+# This will clear any queue discipline configuration on the devices. If there was no configuration, you will get an error, but you can ignore it.
+sshpass -p "$server_pwd" ssh -o StrictHostKeyChecking=no "$server_user"@"$server_ip" "echo ${server_pwd} | sudo -S tc qdisc del dev $server_eth_interface root" || true
+tc qdisc del dev "$eth_interface" root || true
+
+if [ "$mode" == "client" ]; then
+    sshpass -p "$server_pwd" ssh -o StrictHostKeyChecking=no "$server_user"@"$server_ip" "echo ${server_pwd} | sudo -S timeout $timeout ptp4l -P2Hi $server_eth_interface -f $config_path --step_threshold=1 --first_step_threshold=0.0" &
+    timeout "$timeout" ptp4l -P2Hi "$eth_interface" -f "$config_path" --step_threshold=1 -m -s --first_step_threshold=0.0
+elif [ "$mode" == "master" ]; then
+    timeout "$timeout" ptp4l -P2Hi "$eth_interface" -f "$config_path" --step_threshold=1 --first_step_threshold=0.0 &
+    sshpass -p "$server_pwd" ssh -o StrictHostKeyChecking=no "$server_user"@"$server_ip" "echo ${server_pwd} | sudo -S timeout $timeout ptp4l -m -s -P2Hi $server_eth_interface -f $config_path --step_threshold=1 --first_step_threshold=0.0"
+else
+    echo "Wrong argument for $mode. It should be either client or master"
+    exit 1
+fi
diff --git a/bin/udp_traffic_test.sh b/bin/udp_traffic_test.sh
new file mode 100755
index 0000000..c0ac460
--- /dev/null
+++ b/bin/udp_traffic_test.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+
+eth_interface=$1
+testing_time=$2
+gptp_config_path=$3
+server_user=$4
+server_ip=$5
+server_pwd=$6
+server_eth_interface=$7
+
+echo "## Starting time synchronization..."
+tsn_time_sync.sh "$testing_time" master "${gptp_config_path}" "${eth_interface}" "${server_user}" "${server_ip}" "${server_pwd}" "${server_eth_interface}" > /dev/null &
+sleep 10
+
+echo "## Setting TAPRIO qdisc..."
+BASE=$(date +%s%N)
+tc qdisc add dev "${eth_interface}" parent root handle 100 taprio num_tc 4 map 0 1 2 3 3 3 3 3 3 3 3 3 3 3 3 3 queues 1@0 1@1 1@2 1@3 base-time "${BASE}" sched-entry S 01 5000000 sched-entry S 02 5000000 sched-entry S 04 5000000 sched-entry S 08 5000000 flags 0x2 txtime-delay 0
+sleep 10
+
+echo "## Starting iperf3 server..."
+sshpass -p "${server_user}" ssh -o StrictHostKeyChecking=no "${server_user}@${server_ip}" "echo ${server_pwd} | sudo -S iperf3 -s -p 5100 -D" &
+sshpass -p "${server_user}" ssh -o StrictHostKeyChecking=no "${server_user}@${server_ip}" "echo ${server_pwd} | sudo -S iperf3 -s -p 5101 -D" &
+sshpass -p "${server_user}" ssh -o StrictHostKeyChecking=no "${server_user}@${server_ip}" "echo ${server_pwd} | sudo -S iperf3 -s -p 5102 -D" &
+
+echo "## Setting which hardware transmit queue each iperf3 instance using via net_prio cgroups..."
+mkdir -p /sys/fs/cgroup/net_prio
+mount -t cgroup -onet_prio none /sys/fs/cgroup/net_prio
+mkdir -p /sys/fs/cgroup/net_prio/grp1
+echo "${eth_interface} 1" > /sys/fs/cgroup/net_prio/grp1/net_prio.ifpriomap
+mkdir -p /sys/fs/cgroup/net_prio/grp2
+echo "${eth_interface} 2" > /sys/fs/cgroup/net_prio/grp2/net_prio.ifpriomap
+mkdir -p /sys/fs/cgroup/net_prio/grp3
+echo "${eth_interface} 3" > /sys/fs/cgroup/net_prio/grp3/net_prio.ifpriomap
+
+echo "## Setting IP address..."
+TARGET_IP=192.168.1.22
+sshpass -p "${server_user}" ssh -o StrictHostKeyChecking=no "${server_user}@${server_ip}" "echo ${server_pwd} | sudo -S ip addr add ${TARGET_IP}/24 dev ${server_eth_interface}"
+ip addr add 192.168.1.11/24 dev "${eth_interface}"
+
+echo "## Running iperf3 client and add each instance to its owncgroup directory..."
+iperf3 -u -l 1000 -b0 -p 5100 -c ${TARGET_IP} -t $((testing_time-10)) >> /dev/null &
+echo $! > /sys/fs/cgroup/net_prio/grp1/cgroup.procs
+iperf3 -u -l 1000 -b0 -p 5101 -c ${TARGET_IP} -t $((testing_time-10)) >> /dev/null &
+echo $! > /sys/fs/cgroup/net_prio/grp2/cgroup.procs
+iperf3 -u -l 1000 -b0 -p 5102 -c ${TARGET_IP} -t $((testing_time-10)) >> /dev/null &
+echo $! > /sys/fs/cgroup/net_prio/grp3/cgroup.procs
+
+echo "## Checking results..."
+tc -s qdisc show dev "${eth_interface}"
+sleep 10
+echo ""
+echo "# 10 seconds after..."
+tc -s qdisc show dev "${eth_interface}"
+sleep 10
+echo ""
+echo "# 10 seconds after..."
+tc -s qdisc show dev "${eth_interface}"
+sshpass -p "${server_user}" ssh -o StrictHostKeyChecking=no "${server_user}@${server_ip}" "echo ${server_pwd} | sudo -S killall iperf3"
diff --git a/units/tsn/jobs.pxu b/units/tsn/jobs.pxu
index 701398b..96b1a4f 100644
--- a/units/tsn/jobs.pxu
+++ b/units/tsn/jobs.pxu
@@ -1,11 +1,30 @@
 unit: job
 category_id: tsn
-id: tsn/enabled
-_summary: Check if TSN is supported
+id: tsn/tsn-detected
+plugin: shell
+_summary: Check if there is at least one ethernet interface support TSN
 imports: from com.canonical.plainbox import manifest
 requires: manifest.has_tsn == 'True'
-flags: simple
-command: echo "TSN is supported."
+user: root
+command:
+    DEVLIST=$(lspci |grep -i "ethernet controller" |cut -d' ' -f1)
+    exit_code=1
+    for DEVICE in $DEVLIST; do
+        mystr=""
+        mystr=$(lspci -nnvv -s "$DEVICE" |grep -i 'precision time measurement')
+        if [ -n "$mystr" ]; then
+            INTERFACE=$(ls -l /sys/class/net |grep "$DEVICE" |cut -d'/' -f8)
+            echo "$INTERFACE : TSN supported!"
+            exit_code=0
+        fi
+    done
+    echo ""
+    if [ "$exit_code" -eq 0 ]; then
+        echo "PASS: Found at least one ethernet interface supporting TSN!"
+    else
+        echo "FAIL: There's no ethernet interface supporting TSN!"
+    fi
+    exit "$exit_code"
 
 unit: job
 category_id: tsn
@@ -47,7 +66,7 @@ _description:
     If it does, it will have a number identifying which clock ID it is using,\
     and a corresponding entry in /dev/ptp#.\ 
     If it does not support PTP, it will list "none", and the test will fail.
-depends: tsn/enabled
+depends: tsn/tsn-detected
 plugin: shell
 user: root
 command:
@@ -68,132 +87,220 @@ command:
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-client-time-sync
+id: tsn/test-env-setup-for-{eth-interface}
 category_id: tsn
-_summary: Verify SUT can use {eth-interface} to establish foreign clock sync
-_purpose: 
-    Use {eth-interface} on the SUT to synchronize its clock with the clock\
-    of the Grand Master of the Secondary System.
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.pzr6mq4dyb0u
+_summary: Setup TSN testing environment for Ethernet interface {eth-interface}
+_description: 
+    Setup Testing environment for TSN
+depends: tsn/tsn-detected
+plugin: shell
+user: root
+environ:
+    GPTP_CONFIG_PATH
+command:
+    tsn_test_env_setup.sh {eth-interface} "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}"
 
-unit: template
-template-resource: tsn/tsn-devices
-template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-client-sync-to-ptp-hw-clock
+unit: job
 category_id: tsn
-_summary: Verify SUT {eth-interface} can sync its hardware clock to Master
-_purpose:
-    Use {eth-interface} on the SUT to synchronize its clock with the clock\
-    of the Grand Master of the Secondary System.
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.dxubm4m7zzag
+id: tsn/test-server-env-setup
+_summary: Setup the TSN server environment
+plugin: user-interact
+user: root
+_description: 
+    Setup Testing server environment for TSN
+environ:
+    GPTP_CONFIG_PATH
+command:
+    echo "Please enter the ethernet interface (like enp1s0, eno1 etc.) you would like to enable."
+    echo "Please make sure that ethernet interface support TSN."
+    read -p "Ethernet interface: " eth_interface
+    tsn_test_env_setup.sh "$eth_interface" "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}"
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-talker-using-time-aware-shaper
+id: tsn/ieee802.1as-2011/time-sync-between-two-system-as-client-{eth-interface}
 category_id: tsn
-_summary: Verify SUT {eth-interface} can transmit using Time Aware Shaper
+_summary: Verify SUT can use {eth-interface} to establish time-sync with another machine as client (IEEE 802.1AS-2011)
 _purpose:
-   Configure Time Aware Shaper on SUT and Secondary, and verify SUT transmits\ 
-   packets in accordance to this configuration
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.2mpce0amhrf9
+    Use {eth-interface} on the SUT to achieve time sync with the second machine as client
+depends:
+    tsn/tsn-detected
+plugin: user-interact-verify
+user: root
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+    GPTP_CONFIG_PATH
+command:
+    tsn_time_sync.sh 60 client "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}" {eth-interface} "$TSN_SERVER_USER" "$TSN_SERVER_IP" "$TSN_SERVER_PWD" "$TSN_SERVER_INTERFACE"
+_verification:
+    Is the `rms` value within 100ns?
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-talker-using-time-based-shaper
+id: tsn/ieee802.1as-2011/time-sync-between-two-system-as-master-{eth-interface}
 category_id: tsn
-_summary: Verify SUT {eth-interface} can transmit at precise 1ms intervals
+_summary: Verify SUT can use {eth-interface} to establish time-sync with another machine as master (IEEE 802.1AS-2011)
 _purpose:
-   Configure Time Based Shaper on SUT and Secondary, and verify SUT transmits\
-   packets in accordance to this configuration
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.2m1zswmirdvb
+    Use {eth-interface} on the SUT to achieve time sync with the second machine as master
+depends:
+    tsn/tsn-detected
+plugin: user-interact-verify
+user: root
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+    GPTP_CONFIG_PATH
+command:
+    tsn_time_sync.sh 60 master "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}" {eth-interface} "$TSN_SERVER_USER" "$TSN_SERVER_IP" "$TSN_SERVER_PWD" "$TSN_SERVER_INTERFACE"
+_verification:
+    Is the `rms` value within 100ns?
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-configured-with-frame-preemption
+id: tsn/ieee802.1as-2011/time-sync-system-clock-with-PHC-as-client-{eth-interface}
 category_id: tsn
-_summary: Verify that Frame-Preemption can be configured on the {eth-interface} of the SUT
+_summary: Verify if the system can sync time between system clock and PTP hardware clock as client(IEEE 802.1AS-2011)
+depends:
+    tsn/tsn-detected
+    tsn/ieee802.1as-2011/time-sync-between-two-system-as-client-{eth-interface}
+plugin: user-interact-verify
+user: root
 _purpose:
-   Configure Frame-Preemption on the SUT
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.ge4dauxlinm3
+    To check if the system can sync time between system clock and PTP hardware clock as client
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+    GPTP_CONFIG_PATH
+command:
+    tsn_time_sync.sh 60 client "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}" {eth-interface} "$TSN_SERVER_USER" "$TSN_SERVER_IP" "$TSN_SERVER_PWD" "$TSN_SERVER_INTERFACE" > /dev/null &
+    pmc -u -b 0 -t 1 "SET GRANDMASTER_SETTINGS_NP clockClass 248\
+    clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37\
+    leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1\
+    frequencyTraceable 0 timeSource 0xa0"
+    timeout 60 phc2sys -s {eth-interface} -O 0 -c CLOCK_REALTIME --step_threshold=1 \
+    --transportSpecific=1 -w -m --first_step_threshold=0.0
+_verification:
+    Is the `phc offset` value within `100ns`?
+    Is the `State` value equal to `S2`?
+    Is the `Path Delay` equal to `0`
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-master-time-sync
+id: tsn/ieee802.1as-2011/time-sync-system-clock-with-PHC-as-master-{eth-interface}
 category_id: tsn
-_summary: Verify SUT can use {eth-interface} to establish itself as the Clock Master
+_summary: Verify if the system can sync time between system clock and PTP hardware clock as master(IEEE 802.1AS-2011)
+depends:
+    tsn/tsn-detected
+    tsn/ieee802.1as-2011/time-sync-between-two-system-as-master-{eth-interface}
+plugin: user-interact-verify
+user: root
 _purpose:
-    Use {eth-interface} on the SUT to become the Grand Clock Master and have the Secondary\
-    System synchronize with it.
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.15f1v79u3km0
+    To check if the system can sync time between system clock and PTP hardware clock as master
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+    GPTP_CONFIG_PATH
+command:
+    tsn_time_sync.sh 60 master "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}" {eth-interface} "$TSN_SERVER_USER" "$TSN_SERVER_IP" "$TSN_SERVER_PWD" "$TSN_SERVER_INTERFACE" > /dev/null &
+    pmc -u -b 0 -t 1 "SET GRANDMASTER_SETTINGS_NP clockClass 248\
+    clockAccuracy 0xfe offsetScaledLogVariance 0xffff currentUtcOffset 37\
+    leap61 0 leap59 0 currentUtcOffsetValid 1 ptpTimescale 1 timeTraceable 1\
+    frequencyTraceable 0 timeSource 0xa0"
+    timeout 60 phc2sys -s {eth-interface} -O 0 -c CLOCK_REALTIME --step_threshold=1 \
+    --transportSpecific=1 -w -m --first_step_threshold=0.0
+_verification:
+    Is the `phc offset` value within `100ns`?
+    Is the `State` value equal to `S2`?
+    Is the `Path Delay` equal to `0`
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-master-sync-to-ptp-hw-clock
+id: tsn/time-based-shaping/udp-packet-time-check-{eth-interface}
 category_id: tsn
-_summary: Verify Secondary can sync hardware clock of the SUT {eth-interface}
+_summary: Check the packet time intervals
+depends:
+    tsn/tsn-detected
+plugin: user-interact-verify
+user: root
 _purpose:
-    Have the Secondary system to synchronize its HW clock with the HW clock on the {eth-interface} of the SUT
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.rb5hx7uq4hcr
+    To check if the system can transmit udp packet precisely at 1 ms intervals
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+command:
+    tc qdisc del dev {eth-interface} root
+    tc qdisc add dev {eth-interface} handle 8001: parent root mqprio num_tc 4 map 0 1 2 3 3 3 3 3 3 3 3 3 3 3 3 3 queues 1@0 1@1 1@2 1@3 hw 0
+    tc qdisc replace dev {eth-interface} parent 8001:4 etf offload clockid CLOCK_TAI delta 500000
+    tc qdisc show dev {eth-interface}
+    timeout 30 udp_tai -c 3 -i {eth-interface} -P 1000000 -p 90 -d 600000 > /dev/null &
+    timeout 30 sshpass -p "u" ssh -o StrictHostKeyChecking=no "$TSN_SERVER_USER"@"$TSN_SERVER_IP" "echo "$TSN_SERVER_PWD" | sudo -S tcpdump -Q in -ttt -ni "$TSN_SERVER_INTERFACE" --time-stamp-precision=nano -j adapter_unsynced port 7788 -c 20000"
+_verification:
+    Is the packets are being transmitted every 1000000ns (+/-500ns)?
+    If > 95% of packets match this criteria the test is successful.
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-listener-using-time-aware-shaper
+id: tsn/ieee802.1qbv/udp-traffic-check-{eth-interface}
 category_id: tsn
-_summary: Verify SUT {eth-interface} can receive using Time Aware Shaper
+_summary: Check if the streams packet are handled properly
+depends:
+    tsn/tsn-detected
+plugin: user-interact-verify
+user: root
 _purpose:
-   Configure Time Aware Shaper on SUT and Secondary, and verify SUT receives\
-   packets in accordance to this configuration
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.9x7mdm368f70
+    To check if the system can transmit 4 streams of iperf udp traffic
+    using TAPRIO qdisc to limit the bandwidth of each stream.
+environ:
+    TSN_SERVER_IP
+    TSN_SERVER_USER
+    TSN_SERVER_PWD
+    TSN_SERVER_INTERFACE
+    GPTP_CONFIG_PATH
+command:
+    udp_traffic_test.sh {eth-interface} 30 "${{GPTP_CONFIG_PATH:-/usr/share/gPTP.cfg}}" "$TSN_SERVER_USER" "$TSN_SERVER_IP" "$TSN_SERVER_PWD" "$TSN_SERVER_INTERFACE"
+_verification:
+    Is the send bytes and pkt are increasing for 100:1 - 100:4?
+
 
 unit: template
 template-resource: tsn/tsn-devices
 template-unit: job
-id: tsn/verify-SUT-{eth-interface}-as-listener-using-time-based-shaper
+id: tsn/ieee802.1qbu/check-frame-preemption-{eth-interface}
 category_id: tsn
-_summary: Verify SUT {eth-interface} can receive at precise 1ms intervals
+_summary: Check if could set frame preemption
+depends:
+    tsn/tsn-detected
+plugin: user-interact-verify
+user: root
 _purpose:
-   Configure Time Based Shaper on SUT and Secondary, and verify the Secondary transmits\
-   packets in accordance to this configuration
-depends: tsn/enabled
-plugin: manual
-_steps:
-    Follow the instructions as described here:
-    https://docs.google.com/document/d/14fNcjUzxh6H-mWK8tXV8HoUxrgFirOhDksUZfd44OTY/edit#heading=h.ozz3peqlqp6g
+    To check if the system can setup frame preemption
+command:
+    if ! command -v ethtool-rt.ethtool > /dev/null; then
+        echo "ERROR: snap ethtool-rt si required for this test and it's probably not installed."
+        echo "Please install it with the following command."
+        echo "$ sudo snap install ethtool-rt --edge --devmode"
+        exit 1
+    fi
+    echo "## Setting frame preemption on..."
+    ethtool-rt.ethtool --set-frame-preemption {eth-interface} fp on
+    echo "## Showing frame preemption settings..."
+    ethtool-rt.ethtool --show-frame-preemption {eth-interface}  
+_verification:
+    Is the frame preemption enabled for {eth-interface}?
diff --git a/units/tsn/test-plan.pxu b/units/tsn/test-plan.pxu
index f5d27de..1b6d405 100644
--- a/units/tsn/test-plan.pxu
+++ b/units/tsn/test-plan.pxu
@@ -13,16 +13,14 @@ _name: Time Sensitive Networking (TSN) test (manual)
 unit: test plan
 bootstrap_include:
     tsn/tsn-devices
-include: 
-    tsn/verify-SUT-.*-as-client-time-sync
-    tsn/verify-SUT-.*-as-client-sync-to-ptp-hw-clock
-    tsn/verify-SUT-.*-as-talker-using-time-aware-shaper
-    tsn/verify-SUT-.*-as-talker-using-time-based-shaper
-    tsn/verify-SUT-.*-configured-with-frame-preemption
-    tsn/verify-SUT-.*-as-master-time-sync
-    tsn/verify-SUT-.*-as-master-sync-to-ptp-hw-clock
-    tsn/verify-SUT-.*-as-listener-using-time-aware-shaper
-    tsn/verify-SUT-.*-as-listener-using-time-based-shaper
+include:
+    tsn/tsn-detected
+    tsn/test-env-setup-for-.*
+nested_part:
+    tsn-ieee802.1as-2011
+    tsn-time-based-shaping
+    tsn-ieee802.1qbv
+    tsn-ieee802.1qbu
 
 id: tsn-automated
 _name: Time Sensitive Networking (TSN) test (automated)
@@ -31,5 +29,51 @@ bootstrap_include:
     tsn/tsn-devices
 include:
     tsn/verify-PTP-for-.*
-  
-  
+
+id: tsn-ieee802.1as-2011
+_name: Time Sensitive Networking Test Plan for IEEE 802.1AS-2011
+_description:
+    This testplan is to check if it could follow IEEE 802.1AS-2011.
+     - To achieve time synchronization between two platforms.
+     - To synchronize each platforms's system clock with PHC (Physical hardware clock)
+       using hardware cross-timestamping (which is selected by phc2sys automatically).
+unit: test plan
+bootstrap_include:
+    tsn/tsn-devices
+include:
+    tsn/ieee802.1as-2011/time-sync-between-two-system-as-client-.*
+    tsn/ieee802.1as-2011/time-sync-between-two-system-as-master-.*
+    tsn/ieee802.1as-2011/time-sync-system-clock-with-PHC-as-client-.*
+    tsn/ieee802.1as-2011/time-sync-system-clock-with-PHC-as-master-.*    
+
+id: tsn-time-based-shaping
+_name: Time Sensitive Networking Test Plan for Time-Based Shaping
+_description:
+    This testplan is to check if it could do time-based Shaping.
+     - Transmit a UDP packet precisely at 1 ms intervals. The listner board
+       shows the time between packets.
+unit: test plan
+bootstrap_include:
+    tsn/tsn-devices
+include:
+    tsn/time-based-shaping/udp-packet-time-check-.*
+
+id: tsn-ieee802.1qbv
+_name: Time Sensitive Networking Test Plan for IEEE 802.1Qbv
+_description:
+    This testplan is to check if it could follow IEEE 802.1Qbv Time Aware Shaper (TAS).
+unit: test plan
+bootstrap_include:
+    tsn/tsn-devices
+include:
+    tsn/ieee802.1qbv/udp-traffic-check-.*
+
+id: tsn-ieee802.1qbu
+_name: Time Sensitive Networking Test Plan for IEEE 802.1Qbu
+_description:
+    This testplan is to check if it could follow IEEE 802.1Qbu Frame Preemption.
+unit: test plan
+bootstrap_include:
+    tsn/tsn-devices
+include:
+    tsn/ieee802.1qbu/check-frame-preemption-.*