Canonical System Image

Need to detect use of UCS-2 encoding and reflect in message count

Bug #1617961 reported by Jean-Baptiste Lallement on 2016-08-29

This bug affects 6 people

Affects		Status	Importance	Assigned to	Milestone
	Canonical System Image	Confirmed	Wishlist	Bill Filler	Canonical System Image backlog
	messaging-app (Ubuntu)	Confirmed	Undecided	Unassigned

Bug Description

current build number: 409
device name: arale
channel: ubuntu-touch/rc-proposed/meizu.en

A single SMS is charged several times while there is only 1 corresponding SMS in the messaging app.

On the report attach, several messages are charged up to 5 times. It's very similar to bug 1537735 although no one ever told me they received my SMS more than once.

See original description

Revision history for this message

Jean-Baptiste Lallement (jibel) wrote on 2016-08-29:

LP1617961.png Edit (39.3 KiB, image/png)

description:	updated
Changed in canonical-devices-system-image:
status:	New → Confirmed
description:	updated
Changed in canonical-devices-system-image:
importance:	Undecided → Critical
milestone:	none → 13
assignee:	nobody → John McAleely (john.mcaleely)

Revision history for this message

Alfonso Sanchez-Beato (alfonsosanchezbeato) wrote on 2016-08-29:

This is probably due to using characters not included in the GSM 7-bit alphabet [1]. Which implies that UCS-2 is used to encode the messages. This limits size to 70 chars per SMS. If this is confirmed, the issue would be the UI not informing the user about extra SMS needed to send the message.

[1] https://en.wikipedia.org/wiki/GSM_03.38#GSM_7-bit_default_alphabet_and_extension_table_of_3GPP_TS_23.038_.2F_GSM_03.38

Revision history for this message

Jean-Baptiste Lallement (jibel) wrote on 2016-08-29:

I think this is correct, I sent 2 SMS made of 50 accentuated characters between the same 2 devices, on the same network and same location. On the receiver end, I received the message only once but on the report from the operator I've been charged twice for each message.
Then I sent a short message without any accent and I've been charged once.

Revision history for this message

John McAleely (john.mcaleely) wrote on 2016-08-30:

ok, I've passed this over to @bfiller then.

Changed in canonical-devices-system-image:
assignee:	John McAleely (john.mcaleely) → Bill Filler (bfiller)

Pat McGowan (pat-mcgowan) on 2016-08-30

summary:

- SMS are charged several times
+ Need to detect use of UCS-2 encoding and reflect in message count

Revision history for this message

Launchpad Janitor (janitor) wrote on 2016-08-30:

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in messaging-app (Ubuntu):
status:	New → Confirmed

Pat McGowan (pat-mcgowan) on 2016-08-31

Changed in canonical-devices-system-image:
milestone:	13 → backlog
importance:	Critical → Wishlist

Revision history for this message

Pat McGowan (pat-mcgowan) wrote on 2016-08-31:

It turns out that implementing a UI for this properly is a bit complex. This is the standard way that SMS works on all smartphones based on the characters in the message.

In short several things need to be observed and accounted for:
Any character not in the GSM alphabet causes use of UCS-2 (certain escape chars, emojis)
Any extension GSM character requires an extra character. Any such escape character must have both parts in the same message and can therefore leave 1 more unused char.
Multi-part messages introduce header overhead in each message in both GSM (7 chars) and UCS-2 (3 chars)

Revision history for this message

Jean-Baptiste Lallement (jibel) wrote on 2016-09-01:

Just for reference, here is an example of such sms length calculator
https://bitbucket.org/solvam/smstools/src