modem-manager crashed with SIGSEGV in g_slice_alloc() - REGRESSION: No internet access via mobile broadband

There isn't enough information here to debug it yet. What were you doing when it crashed? Plugging in a new modem?

Could you try to reproduce the crash and provide debugging logs for ModemManager, as outlined in http://wiki.ubuntu.com/DebuggingNetworkManager ?

Thanks!

Here is the missing info:

First, this bug is a regression against 12.10. Booting from a 12.10 live CD I can use the mobile broadband.

The modem is built-in (Ericsson Business Mobile Networks BV).

Id did not get the crash this time, but no connection. According to syslog, modem-manager is trying something infinitely.

I have switched to debug logging via

sudo python debug-helper.py --mm debug

Then I have captured the syslog via

tail -n0 -f /var/log/syslog > /tmp/syslog

While capturing, I have first clicked "Enable Mobile Broadband" in the Network Manager menu, near the bottom. After some seconds I got an OSD pop-up that I am registered for GSN or so. Now I clicked on "T-Mobile (Telekom) Default" in the "Mobile Broadband" section. The indicator applet icon of the Network Manager started spinning, telling that it is searched for a network. After some time it stopped spinning without connection established. After that, modem-manager is still active in the syslog. I waited some minutes before stopping to capture.

Log files attached to the following comments.

Output of "lsusb".

udevadm info --query=all --path=/sys/class/tty/ttyACM0 --attribute-walk

udevadm info --query=all --path=/sys/class/tty/ttyACM1 --attribute-walk

udevadm info --query=all --path=/sys/class/tty/ttyACM2 --attribute-walk

There are two orthogonal issues here:
 * One the connection failure, you got logs for that one. But the logs just say you were not able to get an IP address, so could really be just a temp failure.
 * The g_slice_alloc crash, what this log is about, we didn't get new info for that one.

modem-manager kept spinning as shown in the attached captured syslog until 21:25 and then it crashed with a segfault, but I did not get an Apport pop-up.

Full syslog of the system, including modem-manager segfault at 21:27:48. No new crash report got dropped in /var/crash/.

I have tried already several times and had two crash reports, this bug and the duplicate bug 1163015. Both seem to have crashed at the same point and in both the crash report popped up only several minutes after the attempt to connect via mobile broadband. So the modem-manager crash of my test run to get the logs is the same.

Your last syslog doesn't contain any crash... so not sure what you mean with crash :)

Aleksander, the syslog.txt of comment #11 contains a segfault at Apr 3, 21:27:48.

Ah, you're right, my bad :)

Relevant logs then:

Apr 3 21:26:18 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): --> 'AT+CIND?<CR>'
Apr 3 21:26:21 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): --> 'AT+CSQ<CR>'
Apr 3 21:26:21 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>+CSQ: 11,99<CR><LF>'
Apr 3 21:26:21 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>OK<CR><LF>'
Apr 3 21:26:21 till-twist modem-manager[5778]: mm_callback_info_schedule: assertion `info->called == FALSE' failed
Apr 3 21:26:48 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): --> 'AT+CIND?<CR>'
Apr 3 21:26:48 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>+CIND: 5,3,0,0,1,1,0,0,0,0,0,0<CR><LF>'
Apr 3 21:26:48 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>OK<CR><LF>'
Apr 3 21:26:57 till-twist usbmuxd[5806]: [2] Attempted to connect to nonexistent device 1
Apr 3 21:27:18 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): --> 'AT+CIND?<CR>'
Apr 3 21:27:18 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>+CIND: 5,3,0,0,1,1,0,0,0,0,0,0<CR><LF>'
Apr 3 21:27:18 till-twist modem-manager[5778]: <debug> [mm-at-serial-port.c:334] debug_log(): (ttyACM1): <-- '<CR><LF>OK<CR><LF>'
Apr 3 21:27:48 till-twist kernel: [11019.226084] modem-manager[5778]: segfault at 100000001 ip 00007fd690d58f7f sp 00007fff38b165f0 error 4 in libglib-2.0.so.0.3600.0[7fd690cf5000+f9000]

The "assertion `info->called == FALSE' failed" message is interesting...

Are you able to run modem-manager manually under valgrind? These would be the steps:

$> sudo mv /usr/sbin/modem-manager /
$> sudo killall -9 modem-manager
$> sudo G_SLICE=always-malloc valgrind --log-file=valgrind.out /modem-manager --debug

[Try to reproduce the issue, attach the valgrind.out file to the bugreport]

And then recover the original state:
$> sudo mv /modem-manager /usr/sbin

I did

sudo mv /usr/sbin/modem-manager /
sudo killall -9 modem-manager
sudo G_SLICE=always-malloc valgrind --log-file=valgrind.out /modem-manager --debug 2>&1 | tee modem-manager.out

While the last one was running I have (as on the last test) first clicked "Enable Mobile Broadband" in the Network Manager menu, near the bottom. After some seconds I got an OSD pop-up that I am registered for GSN or so. Now I clicked on "T-Mobile (Telekom) Default" in the "Mobile Broadband" section. The indicator applet icon of the Network Manager started spinning, telling that it is searched for a network. After some time it stopped spinning without connection established. After that, modem-manager still kept running for some hours (it did not crash this time) and so I pressed Ctrl+C. Also, I did not get a connection, as before.

Log files attached to the following comments.

I've fixed an issue with invalid memory handling in Ericsson MBM devices, it's in the MM_06 branch and git master:
http://cgit.freedesktop.org/ModemManager/ModemManager/commit/?h=MM_06&id=16520193f715453eac3c48eab3a648a0cadfb1b5

You may want to try the latest MM_06 branch to see if it really fixes your issues.

I couldn't really reproduce any more issues with my Ericsson MBM device here.

Thank you for the patch. I have applied it (debdiff attached), installed the resulting package but the behavior is exactly the same, no connection, a lot of logging in syslog (debug mode), but at least no crash.

debug syslog capture in next comment.

syslog (in debug mode) for patched modem-manager.

Ok, good to know :)

Now, I'm not sure why you don't get a proper connection. The DHCP request on the interface just times out, as reported by NetworkManager:

Apr 4 21:20:56 till-twist NetworkManager[1150]: <warn> (wwan0): DHCPv4 request timed out.
Apr 4 21:20:56 till-twist NetworkManager[1150]: <info> (wwan0): canceled DHCP transaction, DHCP client pid 15159
Apr 4 21:20:56 till-twist NetworkManager[1150]: <info> Activation (wwan0) Stage 4 of 5 (IPv4 Configure Timeout) scheduled...
Apr 4 21:20:56 till-twist NetworkManager[1150]: <info> Activation (wwan0) Stage 4 of 5 (IPv4 Configure Timeout) started...
Apr 4 21:20:56 till-twist NetworkManager[1150]: <info> (wwan0): device state change: ip-config -> failed (reason 'ip-config-unavailable') [70 120 5]

You say it's a regression the lack of connectivity; could you gather debug logs using an older MM version?

Also, check your APN string, is it correct for sure?

I have booted the 12.10 Desktop amd64 live CD now as I know that the mobile broadband works with this. In the live system I entered

wget http://cgit.freedesktop.org/NetworkManager/NetworkManager/plain/test/debug-helper.py
sudo python debug-helper.py --nm debug
sudo python debug-helper.py --mm debug
tail -n0 -f /var/log/syslog > syslog.txt

This switches BOTH Modem Manager and Network Manager into debug mode.

While capturing I have (as on the last test) first clicked "Enable Mobile Broadband" in the Network Manager menu, near the bottom. After some seconds I got an OSD pop-up that I am registered for GSM or so. Now I clicked on "New network" in the "Mobile Broadband" section and went through the first-time-setup wizard, choosing my operator (Germany, T-Mobile, Default). The indicator applet icon of the Network Manager started spinning, telling that it is searched for a network. After some time I got the OSD pop-up telling that I am connected. I disconnected the Ethernet cable then, opened Firefox and googled for "abcdef", to see that mobile broadband works. It does. Then I reconnected the ethernet cable and stopped capturing.

syslog.txt is attached.

Unfortunately, the capturing did not work for some reason. Therefore I am attaching the full syslog of the live CD session.

The APN string with 12.10 is "internet.t-d1.de", the "Number" is "*99#", the "Username" is empty, and the "Password" (not PIN) is "t-d1". The PIN functionality of the SIM card I had turned off before (putting the card into a cell phone for that), therefore "PIN" is empty. "Network ID" is empty, and "Type" is "Any".

On 13.04 where it does not work, the parameters (APN, ...) are exactly the same as in comment #26.

And here is the captured syslog from the not working 13.04 (with patched modem-manager) with debug logging for BOTH Modem Manager and Network Manager.

May end up being a bug in NM after all, just filled it: https://bugzilla.gnome.org/show_bug.cgi?id=697313

Could you try to use a dummy username? Like " " (1 whitespace) or "t-d1". Just run "nm-connection-editor" and edit your connection profile.

I entered "t-d1" as user name and this did not help. Still no connection.

Before you posted comment #29, I got another crash of modem-manager, this time the patched one. I attach the .crash file from the /var/crash directory. As the patched modem-manager is not an official package there was no way to do an automatic upload of the crash data.

syslog for that one?

syslog for connection attempt with user name set to "t-d1". debug mode for both Modem Manager and Network Manager.

Weird, I see nothing wrong... How about if you install an older MM 0.5.x instead of the MM 0.6.x that comes by default? I'm trying to see whether the issue is in MM or out of it...

I have tried 0.5.2 by installing the package of Precise. The user name t-d1 I have removed again. syslog is attached.

The MM package of Quantal also does not work in this Raring environment.

Ok, so we'll need to assume that the connection problem, the lack of IP while doing DHCP on the wwan port, is not MM-related...

Added network-manager task as the problem of not getting an IP seems to be in NM ...

Whoever gets this issue again, please make sure to grab SYSLOG around the time of the *crash* (not the connection issue), and attach it here :)

apport information

apport information

Status changed to 'Confirmed' because the bug affects multiple users.

Aleksander, I succeeded to get another crash of modem-manager now, with syslog of both modem-manager and network-manager in debug mode. See duplicate bug 1167288.

Ok, so the MM_06 branch in git is actually free of your problem. Your issue is fixed (99% chance :)) in the following commit:
http://cgit.freedesktop.org/ModemManager/ModemManager/commit/?h=MM_06&id=5d854a32cf2d04cbdb3d16f019d277caaba69ab4

So, either compile directly that branch, or ask Ubuntu packagers to also include that commit in the build.

Thanks for the patch. I have built the modem-manager package with this patch now and it does not crash but I still get no connection/IP.

debdiff attached.

Did another test of connecting with the modemmanager package from comment #45. This time I clicked on "T-Mobile (Telekom) Default" and after around 1 or 2 seconds the network manager indicator icon stopped spinning. I did this a second time and the same happens. The syslog of these two attempts is attached.

Now I did, still with modemmanager from comment #45:

sudo restart modemmanager
sudo python debug-helper.py --mm debug
sudo python debug-helper.py --nm debug
tail -n0 -f /var/log/syslog > /tmp/syslog

Then I clicked first "Enable Mobile Broadband", wited for the OSD, and clicked "T-Mobile (Telekom) Default" in the network manager indicator applet menu. Then the icon started spinning and I waited for it to stop (no OSD, no connection, no IP) and several more minutes. The syslog is attached.

modem-manager seems to be communication infinitely. It does not stop. But at least it does not crash. This part of the bug seems to be fixed. Problem is now to get a connection/IP.

This bug was fixed in the package modemmanager - 0.6.0.0.really-0ubuntu5

---------------
modemmanager (0.6.0.0.really-0ubuntu5) raring; urgency=low

  * debian/patches/lp1164023_try_csq_if_cind_fails_5d854a3.patch: fallback to
    trying AT+CSQ if AT+CIND fails. (LP: #1164023)
  * debian/patches/git_udev_match_both_vid_pid_e322ccf.patch: match both VID
    and PID when matching devices: If the rules to tag specific USB interface
    numbers only apply on the PID, we'll end up seeing that if the port has a
    parent with another PID, and that other PID also has a rule, port will get
    tagged multiple times.
  * debian/patches/git_mbm_remove_check_poll_when_conn_1652019.patch: mbm:
    remove connection attempt check poll explicitly when connected: Completing
    a MMCallbackInfo is done asynchronously (in an idle), which means that we
    may get the poll timeout called in between...
 -- Mathieu Trudel-Lapierre <email address hidden> Wed, 10 Apr 2013 21:20:53 -0400

The crash is indeed fixed, thank you very much.

For the connection problem I have reported bug 1167852 now.