Merge lp:~doanac/phablet-tools/rndis into lp:phablet-tools

[Andy Doan]

add rndis networking support

This sets up 2 things:

1) an upstart entry to ensure we have the proper bridged interface
in place. This interace will serve as DHCP server to all clients.

2) a udev rule that will detect when an rndis device is connected
and automatically provide that device an address

Inspired By: Jean-Baptiste Lallement and Oliver Grawert

add rndis networking support

This sets up 2 things:

1) an upstart entry to ensure we have the proper bridged interface
in place. This interace will serve as DHCP server to all clients.

2) a udev rule that will detect when an rndis device is connected
and automatically provide that device an address

Inspired By: Jean-Baptiste Lallement and Oliver Grawert

Tested On: mako and grouper

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:147
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/120/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/63
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/63
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/63

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/120/rebuild

review: Approve (continuous-integration)

[Sergio Schvezov (sergiusens)]

21 +# make sure we have proper permission when the device shows up as rndis
22 +SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", RUN+="udev-acl --action=$env{ACTION} --device=$env{DEVNAME}"

can you add this to android-tools-adb ?

I can do it, but it needs to be removed from here (doesn't belong)

review: Needs Fixing

[Sergio Schvezov (sergiusens)]

187 + adb $ADBOPTS push $tfile /var/lib/lxc/android/pre-start.sh
188 + adb $ADBOPTS shell chmod 755 /var/lib/lxc/android/pre-start.sh

failed to copy '/tmp/tmp.KSqkeJWenb' to '/var/lib/lxc/android/pre-start.sh': Read-only file system
chmod: changing permissions of '/var/lib/lxc/android/pre-start.sh': Read-only file system

This should be read from a flag in lxc-android-config

review: Needs Fixing

[Sergio Schvezov (sergiusens)]

257 + 'phablet-network-bridge-addif',

This would be better installed in libexec or something, not a stopper though

review: Needs Fixing

[Sergio Schvezov (sergiusens)]

> 21 +# make sure we have proper permission when the device shows up as
> rndis
> 22 +SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", RUN+="udev-acl
> --action=$env{ACTION} --device=$env{DEVNAME}"
>
> can you add this to android-tools-adb ?
>
> I can do it, but it needs to be removed from here (doesn't belong)

This udev rule also only works on precise.

[Andy Doan (doanac)]

On 07/30/2013 12:52 PM, Sergio Schvezov wrote:
> Review: Needs Fixing
>
> 187 + adb $ADBOPTS push $tfile /var/lib/lxc/android/pre-start.sh
> 188 + adb $ADBOPTS shell chmod 755 /var/lib/lxc/android/pre-start.sh
>
> failed to copy '/tmp/tmp.KSqkeJWenb' to '/var/lib/lxc/android/pre-start.sh': Read-only file system
> chmod: changing permissions of '/var/lib/lxc/android/pre-start.sh': Read-only file system
>
> This should be read from a flag in lxc-android-config

Not sure I follow? If this is a read-only file, we're dead in the water.
Or are you saying check so that we can give a better error message?

[Sergio Schvezov (sergiusens)]

> > 21 +# make sure we have proper permission when the device shows up as
> > rndis
> > 22 +SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", RUN+="udev-acl
> > --action=$env{ACTION} --device=$env{DEVNAME}"
> >
> > can you add this to android-tools-adb ?
> >
> > I can do it, but it needs to be removed from here (doesn't belong)
>
> This udev rule also only works on precise.

Added https://code.launchpad.net/~sergiusens/ubuntu/saucy/android-tools/lax_udev/+merge/177661

[lp:~doanac/phablet-tools/rndis]

148. By Andy Doan on 2013-07-30

remove adb permission hack

Sergio is moving this to android-tools-adbd

149. By Andy Doan on 2013-07-30

move phablet-network-bridge-addif to /lib/udev

as per sergio's review. this is where other udev scripts get located

[Andy Doan (doanac)]

> > 21 +# make sure we have proper permission when the device shows up as
> > rndis
> > 22 +SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", RUN+="udev-acl
> > --action=$env{ACTION} --device=$env{DEVNAME}"
> >
> > can you add this to android-tools-adb ?
> >
> > I can do it, but it needs to be removed from here (doesn't belong)
>
> This udev rule also only works on precise.

It works on raring for me.

[Andy Doan (doanac)]

I think I've fixed all but the lxc-android-config/read-only thing you mentioned. One question:

do we need to update the packaging so that this now depends on a specific version of android-adb-tools so we don't get adb permission issues?

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:149
http://jenkins.qa.ubuntu.com/job/phablet-tools-ci/122/
Executed test runs:
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-amd64-ci/65
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-armhf-ci/65
    SUCCESS: http://jenkins.qa.ubuntu.com/job/phablet-tools-saucy-i386-ci/65

Click here to trigger a rebuild:
http://s-jenkins:8080/job/phablet-tools-ci/122/rebuild

review: Approve (continuous-integration)

[Sergio Schvezov (sergiusens)]

On Tue, Jul 30, 2013 at 3:39 PM, Andy Doan <email address hidden> wrote:
> I think I've fixed all but the lxc-android-config/read-only thing you mentioned. One question:
>
> do we need to update the packaging so that this now depends on a specific version of android-adb-tools so we don't get adb permission issues?

The saucy versions use something like the MR I linked

-ACTION=="add|change", SUBSYSTEM=="usb", \
- ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="d001|d002", \
- TAG+="uaccess"
+ACTION=="add|change", SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", TAG+="uaccess"

Tool version is the same on all environments as long as you have
ppa:phablet-team/tools added.

[Andy Doan (doanac)]

On 07/30/2013 02:43 PM, Sergio Schvezov wrote:
> On Tue, Jul 30, 2013 at 3:39 PM, Andy Doan <email address hidden> wrote:
>> I think I've fixed all but the lxc-android-config/read-only thing you mentioned. One question:
>>
>> do we need to update the packaging so that this now depends on a specific version of android-adb-tools so we don't get adb permission issues?
>
> The saucy versions use something like the MR I linked
>
> -ACTION=="add|change", SUBSYSTEM=="usb", \
> - ATTRS{idVendor}=="18d1", ATTRS{idProduct}=="d001|d002", \
> - TAG+="uaccess"
> +ACTION=="add|change", SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", TAG+="uaccess"
>
> Tool version is the same on all environments as long as you have
> ppa:phablet-team/tools added.
>

What I was trying to say is phablet-tools now requires a specific
version android-adb-tools in order for rndis to work. ie - we merged
this today, and you ran:

  apt-get update; apt-get install phablet-tools

you'd have an outdated version of android-adb-tools, right?

Unmerged revisions

149. By Andy Doan on 2013-07-30

move phablet-network-bridge-addif to /lib/udev

as per sergio's review. this is where other udev scripts get located

148. By Andy Doan on 2013-07-30

remove adb permission hack

Sergio is moving this to android-tools-adbd

147. By Andy Doan on 2013-07-29

add rndis networking support

This sets up 2 things:

1) an upstart entry to ensure we have the proper bridged interface
in place. This interace will serve as DHCP server to all clients.

2) a udev rule that will detect when an rndis device is connected
and automatically provide that device an address

Inspired By: Jean-Baptiste Lallement and Oliver Grawert

146. By Andy Doan on 2013-07-25

break out wifi setup into its own function

This is just to help make things cleaner for adding rndis support

=== modified file 'debian/control'
--- debian/control	2013-07-14 18:46:18 +0000
+++ debian/control	2013-07-30 18:36:35 +0000
@@ -25,6 +25,7 @@
 Depends:
  android-tools-adb (>= 4.2.2),
  android-tools-fastboot (>= 4.2.2),
+ bridge-utils,
  curl,
  python-configobj,
  python-launchpadlib,

=== added file 'debian/phablet-tools.udev'
--- debian/phablet-tools.udev	1970-01-01 00:00:00 +0000
+++ debian/phablet-tools.udev	2013-07-30 18:36:35 +0000
@@ -0,0 +1,3 @@
+# Udev rules for letting adding a usb network interface to the bridge
+ACTION=="add|change", SUBSYSTEM=="net", DRIVERS=="usb", KERNEL=="usb[0-9]*", \
+ATTRS{idVendor}=="18d1", RUN+="phablet-network-bridge-addif"

=== added file 'debian/phablet-tools.upstart'
--- debian/phablet-tools.upstart	1970-01-01 00:00:00 +0000
+++ debian/phablet-tools.upstart	2013-07-30 18:36:35 +0000
@@ -0,0 +1,62 @@
+description "touch usb network"
+author "Jean-Baptiste Lallement <[e-mail]>"
+
+start on started networking
+stop on runlevel [!023456]
+
+env USE_TOUCH_BRIDGE="true"
+env TOUCH_BRIDGE="usbbr0"
+env TOUCH_ADDR="169.254.0.1"
+env TOUCH_NETMASK="255.255.255.0"
+env TOUCH_NETWORK="169.254.0.1/24"
+env TOUCH_DHCP_RANGE="169.254.0.2,169.254.0.254"
+env TOUCH_DHCP_MAX="253"
+env TOUCH_DHCP_CONFILE=""
+env varrun="/var/run/touch"
+
+pre-start script
+    [ -f /etc/default/touch-net ] && . /etc/default/touch-net
+    [ "x$USE_TOUCH_BRIDGE" = "xtrue" ] || { stop; exit 0; }
+
+    cleanup() {
+        # dnsmasq failed to start, clean up the bridge
+        iptables -t nat -D POSTROUTING -s ${TOUCH_NETWORK} ! -d ${TOUCH_NETWORK} -j MASQUERADE || true
+        ifconfig ${TOUCH_BRIDGE} down || true
+        brctl delbr ${TOUCH_BRIDGE} || true
+    }
+
+    if [ -d /sys/class/net/${TOUCH_BRIDGE} ]; then
+        if [ ! -f ${varrun}/network_up ]; then
+        # bridge exists, but we didn't start it
+            stop;
+        fi
+        exit 0;
+    fi
+
+    # set up the usb network
+    brctl addbr ${TOUCH_BRIDGE} || { echo "Missing bridge support in kernel"; stop; exit 0; }
+    echo 1 > /proc/sys/net/ipv4/ip_forward
+    mkdir -p ${varrun}
+    ifconfig ${TOUCH_BRIDGE} ${TOUCH_ADDR} netmask ${TOUCH_NETMASK} up
+    iptables -t nat -A POSTROUTING -s ${TOUCH_NETWORK} ! -d ${TOUCH_NETWORK} -j MASQUERADE
+
+    dnsmasq --strict-order --bind-interfaces --pid-file=${varrun}/dnsmasq.pid --conf-file=${TOUCH_DHCP_CONFILE} --listen-address ${TOUCH_ADDR} --dhcp-range ${TOUCH_DHCP_RANGE} --dhcp-lease-max=${TOUCH_DHCP_MAX} --dhcp-no-override --except-interface=lo --interface=${TOUCH_BRIDGE} --dhcp-leasefile=/var/lib/misc/dnsmasq.${TOUCH_BRIDGE}.leases --dhcp-authoritative || cleanup
+    [ ! -d ${varrun} ] && mkdir -p ${varrun}
+    touch ${varrun}/network_up
+end script
+
+post-stop script
+    [ -f /etc/default/touch ] && . /etc/default/touch
+    [ -f "${varrun}/network_up" ] || exit 0;
+    # if $TOUCH_BRIDGE has attached interfaces, don't shut it down
+    ls /sys/class/net/${TOUCH_BRIDGE}/brif/* > /dev/null 2>&1 && exit 0;
+
+    if [ -d /sys/class/net/${TOUCH_BRIDGE} ]; then
+        ifconfig ${TOUCH_BRIDGE} down
+        iptables -t nat -D POSTROUTING -s ${TOUCH_NETWORK} ! -d ${TOUCH_NETWORK} -j MASQUERADE || true
+        pid=`cat ${varrun}/dnsmasq.pid 2>/dev/null` && kill -9 $pid || true
+        rm -f ${varrun}/dnsmasq.pid
+        brctl delbr ${TOUCH_BRIDGE}
+    fi
+    rm -f ${varrun}/network_up
+end script

=== added file 'phablet-network-bridge-addif'
--- phablet-network-bridge-addif	1970-01-01 00:00:00 +0000
+++ phablet-network-bridge-addif	2013-07-30 18:36:35 +0000
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+# called by udev rules when an rndis device shows up
+# it works in conjunction with logic set up by phablet-network-bridge to
+# allow the target device to aquire an IP address via USB0
+
+BRIDGE=usbbr0
+
+log() {
+	logger -t phablet-tools $*
+}
+
+if [ "$ID_USB_DRIVER" != "rndis_host" ] ; then
+	log "Unknown USB interface found"
+	exit 0
+fi
+
+log "rndis interface found: $DEVPATH"
+DEVICE=$(basename $DEVPATH)
+ip link set $DEVICE up
+brctl addif $BRIDGE $DEVICE

=== modified file 'phablet-network-setup'
--- phablet-network-setup	2013-07-22 12:13:58 +0000
+++ phablet-network-setup	2013-07-30 18:36:35 +0000
@@ -27,6 +27,7 @@
   -s    Specify the serial of the device to install (see adb $ADBOPTS devices)
   -i    Install helper network packages (openssh-server, iw) in chroot
   -n    Select network file
+  -r    Use rndis based network from device to host
 
 EOF
 }
@@ -37,7 +38,7 @@
 CHROOTDIR="/data/ubuntu"
 CHROOTCMD="chroot $CHROOTDIR"
 
-while getopts "hin:s:" OPTION
+while getopts "hin:s:r" OPTION
 do
     case $OPTION in
         h)
@@ -53,6 +54,9 @@
         s)
             ADBOPTS="-s $OPTARG"
             ;;
+        r)
+            RNDIS=1
+            ;;
         ?)
             usage
             exit
@@ -99,39 +103,66 @@
     echo "$network_file"
 }
 
+setup_wifi() {
+    if [ -z "$OPTION_NETWORK_FILE" ]; then
+        network_file=$(find_active_network)
+    else
+        network_file=$OPTION_NETWORK_FILE
+    fi
+
+    if [ ! -f "$network_file" ]
+    then
+        echo "Network connection file \"$network_file\" cannot be read"
+        exit 1
+    fi
+
+    echo Network file is $network_file
+
+    TMP_FILE=$(mktemp)
+    sudo grep -v mac-address "$network_file" > $TMP_FILE
+
+    echo Provisioning network file to device
+    adb $ADBOPTS root
+    adb $ADBOPTS wait-for-device
+    target_network_file=$CHROOTDIR/$NETWORK_MANAGER/active_ws_connection.conf
+    adb $ADBOPTS push $TMP_FILE $target_network_file
+    adb $ADBOPTS shell "chmod 600 $target_network_file"
+
+    rm -f $TMP_FILE
+
+    echo
+}
+
+setup_rndis() {
+    # ensure upon reboots the android lxc container startup doesn't overwrite
+    # our persisted changes below. Also disable the container's adb.
+    tfile=$(mktemp)
+    adb $ADBOPTS pull /var/lib/lxc/android/pre-start.sh $tfile
+    sed -i -e '/# Make sure adbd is not started automatically/{n;s/.*/#PHABLET_TOOLS_RNDIS\n#&\nmv $LXC_ROOTFS_PATH\/sbin\/adbd $LXC_ROOTFS_PATH\/sbin\/adbd.disabled\n#PHABLET_TOOLS_DISABLED/;}' -e "s/#PHABLET_TOOLS_DISABLED$/sed -i '\/setprop persist.sys.usb.config adb$\/d' \$LXC_ROOTFS_PATH\/init.rc\n#PHABLET_TOOLS_RNDIS/" $tfile
+    adb $ADBOPTS push $tfile /var/lib/lxc/android/pre-start.sh
+    adb $ADBOPTS shell chmod 755 /var/lib/lxc/android/pre-start.sh
+    rm $tfile
+
+    adb $ADBOPTS shell sh -c "echo -n rndis,adb > /data/property/persist.sys.usb.config"
+    adb $ADBOPTS reboot
+    # reboots aren't instant, so wait-for-device usually needs to tries
+    sleep 4  # give time to reboot
+    adb $ADBOPTS wait-for-device
+    sleep 4  # give time to reboot
+    adb $ADBOPTS wait-for-device
+}
+
 if is_flipped; then
     echo Ubuntu is root
     CHROOTDIR=""
     CHROOTCMD=""
 fi
 
-if [ -z "$OPTION_NETWORK_FILE" ]; then
-    network_file=$(find_active_network)
+if [ -z $RNDIS ]; then
+    setup_wifi
 else
-    network_file=$OPTION_NETWORK_FILE
-fi
-
-if [ ! -f "$network_file" ]
-then
-    echo "Network connection file \"$network_file\" cannot be read"
-    exit 1
-fi
-
-echo Network file is $network_file
-
-TMP_FILE=$(mktemp)
-sudo grep -v mac-address "$network_file" > $TMP_FILE
-
-echo Provisioning network file to device
-adb $ADBOPTS root
-adb $ADBOPTS wait-for-device
-target_network_file=$CHROOTDIR/$NETWORK_MANAGER/active_ws_connection.conf
-adb $ADBOPTS push $TMP_FILE $target_network_file
-adb $ADBOPTS shell "chmod 600 $target_network_file"
-
-rm -f $TMP_FILE
-
-echo
+    setup_rndis
+fi
 echo Network setup complete
 
 if [ $OPTION_SSH -eq 1 ]; then
@@ -139,7 +170,7 @@
     # Indentation is going to be ugly here
     cat > $TMP_FILE << 'EOF'
 #!/bin/bash
-export PATH="/sbin:/usr/sbin:/bin:/usr/bin" 
+export PATH="/sbin:/usr/sbin:/bin:/usr/bin"
 export DEBIAN_FRONTEND=noninteractive
 apt-get update
 apt-get install --yes --quiet openssh-server iw

=== modified file 'setup.py'
--- setup.py	2013-07-17 16:13:33 +0000
+++ setup.py	2013-07-30 18:36:35 +0000
@@ -18,5 +18,8 @@
              'phablet-test-run',
              'repo',
             ],
+    data_files=[
+        ('/lib/udev/', ['phablet-network-bridge-addif']),
+    ],
     test_suite='tests',
 )