Upgrade Noto Sans CJK fonts to version 2.001

Bug #1834406 reported by Gunnar Hjalmarsson
22
This bug affects 1 person
Affects Status Importance Assigned to Milestone
fonts-noto-cjk (Debian)
Fix Released
Unknown
fonts-noto-cjk (Ubuntu)
Fix Released
Wishlist
Gunnar Hjalmarsson
Bionic
Fix Released
Medium
Gunnar Hjalmarsson
Disco
Fix Released
Medium
Gunnar Hjalmarsson
language-selector (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Medium
Gunnar Hjalmarsson
Disco
Fix Released
Medium
Gunnar Hjalmarsson

Bug Description

[Impact]

Due to Japan's new era, a Unicode character has been introduced which represents the era name (Reiwa) as one single character. The version of fonts-noto-cjk in the stable releases does not support the new character, which is what motivates a backport of the eoan fonts-noto-cjk version. The proposed SRU is in line with the meta bug #1828884.

This means an upgrade of Noto Sans CJK from version 1.004 to 2.001, and it will bring a few other changes in the bargain. Most notably a new set of Hong Kong fonts has been added to the previous ones (simplified and traditional Chinese, Japanese, and Korean). Attached to this bug report please see the file noto-sans-cjk-news.txt with upstream's detailing of the changes.

[Test Case]

Install from {bionic,disco}-proposed:

- fonts-noto-cjk
- fonts-noto-cjk-extra
- language-selector-common
- language-selector-gnome

1. The Reiwa glyph
--------------------------
Visit http://people.ubuntu.com/~gunnarhj/square-era-name-reiwa.html
and find that a proper Japanese character, and not a tofu, is shown.

2. Presence and configuration of HK fonts
-----------------------------------------
(This also bears on the language-selector changes.)

* Install the Hong Kong locale:

  sudo locale-gen zh_HK.UTF-8

* Run this command:

  LC_CTYPE=zh_HK.UTF-8 fc-match

  and find that it returns:

  NotoSansCJK-Regular.ttc: "Noto Sans CJK HK" "Regular"

3. General use
--------------
Browse some web sites with Chinese, Japanese or Korean contents and
confirm that the upgrade does not cause any font rendering issues.

[Regression Potential]

This somewhat aggressive SRU proposal is based on trust in Google and Adobe as solid upstream providers of these fonts. Version 2.000 (which includes most of the changes) was released in November 2018, while version 2.001 (with the Reiwa glyph) was released in April 2019. Serious problems should have been known by now. The upstream bug tracker gives no cause for concern:

https://github.com/googlefonts/noto-cjk/issues

It may be worth mentioning that the fonts are provided as OTC files, and we don't compile those files ourselves. Hence just patching the new glyph would not be a practicable option.

It would be easy to reverse this upgrade, if regressions would be reported. Basically we are talking about 7 files on the file system.

[Other Info re. eoan]

Since Debian hasn't packaged this yet, we have created a modified .orig.tar.xz file where 7 .ttc files (plus NEWS and HISTORY) differ from version 1:20181130+repack1-1~exp1, i.e. the latest Debian upload.

There is no other Ubuntu/Debian delta, so as soon as Debian has catched up, it will be fine to start syncing again.

Changed in fonts-noto-cjk (Debian):
status: Unknown → New
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fonts-noto-cjk - 1:20190409+repack1-0ubuntu1

---------------
fonts-noto-cjk (1:20190409+repack1-0ubuntu1) eoan; urgency=medium

  * New upstream release.
    - This upgrades the Noto Sans CJK fonts to version 2.001, which
      includes glyphs for Japan's new era Reiwa (LP: #1834406).

 -- Gunnar Hjalmarsson <email address hidden> Thu, 27 Jun 2019 07:01:00 +0200

Changed in fonts-noto-cjk (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Possible backports to disco and bionic are available in this PPA:

https://launchpad.net/~gunnarhj/+archive/ubuntu/fonts-noto-cjk

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Ok, after browsing through the news and thinking about it, I feel that this could potentially be SRUable. My rationale would be: the package is in main and typically an all-or-nothing package, where we can't cherry pick single pieces easily. So to keep our users up-to-date, there's no other choice but including a new upstream release.

Anyway, with my SRU hat on, I'm giving a +1 on getting this in. But as with any risky bits like this, I'd appreciate if we could keep the package in -proposed for a bit longer and, possibly, organizing a call-for-testing for it.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Thanks Łukasz! Sure, we'll call for testing when the packages are in -proposed.

At the moment they are in the upload queues. I uploaded fonts-noto-cjk to cosmic too 'for version technical reasons'. Not sure if that was necessary...

no longer affects: language-selector (Ubuntu Cosmic)
Changed in fonts-noto-cjk (Ubuntu Bionic):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Changed in fonts-noto-cjk (Ubuntu Cosmic):
status: New → In Progress
Changed in fonts-noto-cjk (Ubuntu Disco):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Changed in language-selector (Ubuntu):
status: New → Fix Released
Changed in language-selector (Ubuntu Bionic):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Changed in language-selector (Ubuntu Disco):
assignee: nobody → Gunnar Hjalmarsson (gunnarhj)
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Mitsuya Shibata (cosmos-door) wrote :

Hi Gunnar, Łukasz,

Thank you for your consideration!

I confirmed fixed for eoan. And I'm going to test bionic/disco, if package upload to proposed.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Note to other SRU members: since this upload is generally impossible to review, what I did is check if the binaries are matching what's in eoan and if the binaries are matching the contents from the upstream tarball.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Gunnar, or anyone else affected,

Accepted fonts-noto-cjk into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fonts-noto-cjk/1:20190409+repack1-0ubuntu0.19.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in fonts-noto-cjk (Ubuntu Disco):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-disco
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Gunnar, or anyone else affected,

Accepted language-selector into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/language-selector/0.194.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in language-selector (Ubuntu Disco):
status: In Progress → Fix Committed
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Gunnar, or anyone else affected,

Accepted fonts-noto-cjk into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/fonts-noto-cjk/1:20190409+repack1-0ubuntu0.18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in fonts-noto-cjk (Ubuntu Bionic):
status: In Progress → Fix Committed
tags: added: verification-needed-bionic
Changed in language-selector (Ubuntu Bionic):
status: In Progress → Fix Committed
Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Hello Gunnar, or anyone else affected,

Accepted language-selector into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/language-selector/0.188.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Confirmed the test case for bionic and disco after having installed

- from bionic-proposed:
   fonts-noto-cjk 1:20190409+repack1-0ubuntu0.18.04
   fonts-noto-cjk-extra 1:20190409+repack1-0ubuntu0.18.04
   language-selector-common 0.188.3
   language-selector-gnome 0.188.3

- from disco-proposed:
   fonts-noto-cjk 1:20190409+repack1-0ubuntu0.19.04
   fonts-noto-cjk-extra 1:20190409+repack1-0ubuntu0.19.04
   language-selector-common 0.194.1
   language-selector-gnome 0.194.1

My tests included step 1, 2, and - FWIW - 3.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Call for testing
================

Since the proposed upgrade of fonts-noto-cjk affects most CJK users and includes changes beyond the glyphs for rendering the Reiwa character, we hope for help with testing by users who speak Chinese, Japanese and/or Korean before the packages are made available to all users via -updates.

Testing on the latest LTS, i.e. Ubuntu 18.04, is preferable, but also testing fonts-noto-cjk 1:20190409+repack1-0ubuntu* on 19.04 or even eoan helps.

These are the steps to install the new versions of the affected packages from bionic-proposed:

1. Enable bionic-proposed
-------------------------

* Open Software & Updates and select the "Developer Options" tab.

* Check the "Pre-released updates" option.

2. Update the package information on your system
------------------------------------------------

sudo apt update

3. Install/upgrade the packages
-------------------------------

sudo apt install fonts-noto-cjk* language-selector-*

4. Disable bionic-proposed
--------------------------

* Open Software & Updates and select the "Developer Options" tab.

* Uncheck the "Pre-released updates" option.

5. Update the package information on your system
------------------------------------------------

sudo apt update

With that your system is back to normal, except for those four packages from -proposed. You can confirm that you got the right versions by using this command:

dpkg-query -W fonts-noto-cjk* language-selector-*

It should show this info:

fonts-noto-cjk 1:20190409+repack1-0ubuntu0.18.04
fonts-noto-cjk-extra 1:20190409+repack1-0ubuntu0.18.04
language-selector-common 0.188.3
language-selector-gnome 0.188.3

Then I simply ask you to use your computer for your daily needs when dealing with CJK contents, and submit a comment here after a few days about your experience. Any kinds of comments are helpful, both "works well" and info about issues you stumble upon. Most important, of course, is that no new issues are introduced via this upgrade.

Thanks in advance for your help with this!

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

Would like to remind of the call för testing in comment #13.

Mitsuya has confirmed that the new Reiwa character is shown properly in the recently upgraded version of ibus-mozc (comment #6). Technically I verified the test case the other day (comment #12), and I have changed the tags accordingly. But as someone who don't speak any CJK language, my evaluation of what the characters look like is of limited value.

So it's highly desirable that some CJK users test the upgraded fonts packages, and submit comments here, before they are moved to -updates.

no longer affects: fonts-noto-cjk (Ubuntu Cosmic)
description: updated
tags: added: verification-done verification-done-bionic verification-done-disco
removed: verification-needed verification-needed-bionic verification-needed-disco
Revision history for this message
Anthony Wong (anthonywong) wrote :

I've always have proposed turned on on my 19.04 box and haven't experienced any Chinese input/output issues with the new font.

Revision history for this message
Mitsuya Shibata (cosmos-door) wrote :

Sorry for lately testing...

On disco and bionic, I found no problem with Japanese locale.

Revision history for this message
Mitsuya Shibata (cosmos-door) wrote :
Revision history for this message
Mitsuya Shibata (cosmos-door) wrote :
Revision history for this message
Brian Murray (brian-murray) wrote :

@gunnarhj- Has there been a call for testing made in places other than this bug report?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2019-07-09 19:27, Brian Murray wrote:
> Has there been a call for testing made in places other than this bug
> report?

There is this message at ubuntu-devel-discuss:

https://lists.ubuntu.com/archives/ubuntu-devel-discuss/2019-July/018393.html

But it's a reply in a thread, and not so well highlighted.

Since I subscribed three relevant teams (Ubuntu CJK Testers, Ubuntu Japanese Team and Ubuntu Korean Team) to this bug, my thought has been that using the bug report ought to be fit for purpose.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

I feel like this update has been in -proposed long enough now. Also, the fix has been verified and package used by 3 people at least (as seen in comments #14, #15 and #16), so that's already quite good. If no issues have been noticed so far, not sure if waiting any longer makes any sense.

Let me release.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package language-selector - 0.194.1

---------------
language-selector (0.194.1) disco; urgency=medium

  * fontconfig/69-language-selector-zh-hk.conf:
  * fontconfig/64-language-selector-prefer.conf:
    Changes because version 2.001 of the Noto Sans CJK fonts includes
    Noto Sans CJK HK and Noto Sans Mono CJK HK (LP: #1834406).

 -- Gunnar Hjalmarsson <email address hidden> Fri, 28 Jun 2019 19:08:00 +0200

Changed in language-selector (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for language-selector has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fonts-noto-cjk - 1:20190409+repack1-0ubuntu0.19.04

---------------
fonts-noto-cjk (1:20190409+repack1-0ubuntu0.19.04) disco; urgency=medium

  * New upstream release.
    - This upgrades the Noto Sans CJK fonts to version 2.001, which
      includes glyphs for Japan's new era Reiwa (LP: #1834406).

 -- Gunnar Hjalmarsson <email address hidden> Thu, 27 Jun 2019 07:01:00 +0200

Changed in fonts-noto-cjk (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package language-selector - 0.188.3

---------------
language-selector (0.188.3) bionic; urgency=medium

  * fontconfig/69-language-selector-zh-hk.conf:
  * fontconfig/64-language-selector-prefer.conf:
    Changes because version 2.001 of the Noto Sans CJK fonts includes
    Noto Sans CJK HK and Noto Sans Mono CJK HK (LP: #1834406).

 -- Gunnar Hjalmarsson <email address hidden> Fri, 28 Jun 2019 19:12:00 +0200

Changed in language-selector (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package fonts-noto-cjk - 1:20190409+repack1-0ubuntu0.18.04

---------------
fonts-noto-cjk (1:20190409+repack1-0ubuntu0.18.04) bionic; urgency=medium

  * New upstream release.
    - This upgrades the Noto Sans CJK fonts to version 2.001, which
      includes glyphs for Japan's new era Reiwa (LP: #1834406).

 -- Gunnar Hjalmarsson <email address hidden> Thu, 27 Jun 2019 07:01:00 +0200

Changed in fonts-noto-cjk (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

Shouldn't this be fixed in Xenial as well?

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

On 2019-07-29 20:40, Mathieu Trudel-Lapierre wrote:
> Shouldn't this be fixed in Xenial as well?

There are two reasons why I didn't propose it for Xenial too.

In Xenial Noto Sans CJK is not the default font in Ubuntu for rendering Japanese, but Takao is. So to support the rendering of the "square era name reiwa" glyph in Xenial, Takao is the package which should be handled at first hand.

Also, Xenial is the first cycle where we use fonts-noto-cjk in Ubuntu (for Chinese), and the introduction of that package with its somewhat demanding OTC packaging format forced us into a last minute upgrade of fontconfig.

https://lists.ubuntu.com/archives/ubuntu-devel/2016-April/039303.html

So without having tested, my gut feeling says that backporting fonts-noto-cjk to Xenial would be higher risk than was the case for Bionic.

Revision history for this message
Mathieu Trudel-Lapierre (cyphermox) wrote :

It *was* used in Xenial though, and appears to be in main. It should be very much evaluated whether people are using it in xenial to display Japanese, and if that's the case, then it ought to be fixed there as well. It is possible a straight backport would work well there, since it's a font.

FWIW, I'm not making the call that it *should* be SRUed, but bringing it up for discussion.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

It's in main in xenial since it's default for Chinese.

On 2019-08-01 16:55, Mathieu Trudel-Lapierre wrote:
> It should be very much evaluated whether people are using it in
> xenial to display Japanese,

I would like to see some Japanese users chime in here.

Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

One specific question is: How common is it that Japanese xenial users have uninstalled fonts-takao-pgothic and other fonts-takao packages in order to let rendering of Japanese text be handled by Noto Sans CJK JP?

Changed in fonts-noto-cjk (Debian):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.