Bug 572841

+++ This bug was initially created as a clone of Bug #499902 +++

Created an attachment (id=343146)
language specific fontconfig settings for Japanese

The CJK language fontconfig settings has been a long-standing issue ever since the beginning of using fontconfig. There hasn't been a good solution yet. The major difficulties for causing this problem, IMO, include

1. the default 65-nonlatin.conf was badly outdated with unoptimized font orders
2. there are no CJK language-specific fontconfig files, therefore, to override the default orders, one has to add font prefer list in the fontconfig settings shipped with individual CJK font package.
3. because these settings were scattered into many font packages, this further makes the problem complex, and the conflict between different CJK font packages became more and more frequent.

To solve this issue, I propose the following solution
1. update 65-nonlatin.conf and setup an optimized and up-to-date font list
2. add language specific fontconfig settings, defining font preferred orders with for a given lang tag, and assign these files a lower number than 65
3. remove all font order settings from all CJK related font packages

For solution step 1, I proposed a completely updated 65-nonlatin.conf at http://bugs.freedesktop.org/show_bug.cgi?id=20911 . My rationale of the font orders was also explained in details.

For solution step 2, I created two default settings for ja and zh locales (that for ko can be done similarly) as examples, and you can find them in the attachments. Because I used lang tag matching in the rules, these files can be installed concurrently (in this sense, it is better than the language-selector in Ubuntu, which needs to run fontconfig-voodoo to link a set of active config under a given language).

I did the following to test this proposal. From what I saw for zh-*, ja and en locales, all the wanted features were working properly.

Here are the details of my test:

First, I did this test with rawhide updated this morning. On the system, I installed chinese-support and japanese-support, including wqy-bitmap-fonts, wqy-zenhei-fonts, arphic-uming, vlgothic-fonts and some mincho Japanese font. 

To clean up all the font config settings, I "su" to root, and run the following
  cd /etc/fonts/conf.d/
  mkdir cjk
  mv *wqy* cjk
  mv *vlgo* cjk
  mv *arphic* cjk
  mv *gothic* cjk

This is just a quick way to get rid of all the font-wise preference settings. In the future, most of these files can stay, as long as they remove the blocks to set <prefer> or prepend family names.

The second step is to update (of course, make a backup first) the 65-nonlatin.conf by downloading from the attachment at http://bugs.freedesktop.org/show_bug.cgi?id=20911

Then, download the 55-language_fonts_ja-jp.conf and 55-language_fonts_zh-cn.conf from this thread and save them under /etc/fonts/conf.d/ . This completes the settings.

For Chinese users, some of them prefer bitmap glyphs for Han characters (35%), some prefer vectors (65%). This is controlled by installing wqy-bitmap-fonts or uninstalling the bitmap fonts.

I tested my proposed settings for English, Japanese and Chinese languages, with bitmap preferred settings and vector preferred settings. My screenshots for all combination can be found at this online album:
http://picasaweb.google.com/fangqq/ProposalForFontconfigSettingsForCJKLanguages

When bitmap font is installed, for en and zh desktops, fonts at sizes 9pt ~ 12pt will be rendered as bitmaps for all font aliases (sans,serif, mono); otherwise, it will be rendered by the respective vector fonts. Particularly, for en desktop, no more font-mosaic problems caused by high priority of Japanese fonts (such as http://picasaweb.google.com/fangqq/ConfigScreenshot#5333302146968242258) . When one remove the bitmap fonts, all Hanzi glyphs were rendered by the preferred vector fonts, as expected (some remaining ones are from UMing, which I did not disable).

For ja desktop, all fonts were rendered by their preferred vector fonts, no matter bitmap Chinese fonts installed or not (please ignore "驿" and "阵" as they are not defined in JIS).



I believe this approach greatly simplifies the CJK font settings by centralizing the related settings to language specific files. All the expected basic rendering order were achieved with the current proposed config files. Of course one can fine-tune them further. For Korean users, a 55-language-fonts-ko.conf file can be made similarly.

It will be really great if this proposal can be tested and agreed by all CJK font package maintainers.

--- Additional comment from fangqq on 2009-05-08 15:17:13 EDT ---

Created an attachment (id=343148)
language specific fontconfig settings for Chinese

--- Additional comment from fedora-triage-list on 2009-06-09 11:29:05 EDT ---


This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

--- Additional comment from petersen on 2010-03-03 04:20:46 EST ---

Closing since this issue has been under discussion upstream.

--- Additional comment from petersen on 2010-03-04 05:17:53 EST ---

Reopening since WQY Zenhei is still overriding vlgothic in Japanese
webpages in firefox.

--- Additional comment from fangqq on 2010-03-10 02:10:02 EST ---

I don't see how this can still happen. Please attach your FC_DEBUG output of a test command, for example
 pango-view  --waterfall --text='与返骨直' --language=ja

--- Additional comment from tagoh on 2010-03-11 08:41:30 EST ---

(In reply to comment #4)
> Reopening since WQY Zenhei is still overriding vlgothic in Japanese
> webpages in firefox.    

That's the "ll v.s. ll-cc" issue. I've tried to run firefox with FC_DEBUG=4 and have a look at the logs. I see they requested both of lang="ja-jp" and lang="ja" in the queries of the font. ideally it would be nice to fix this issue in fontconfig though, we may want to think about a workaround in firefox for that?

Or add a rule for "ll" in every fonts... dunno.

--- Additional comment from fangqq on 2010-03-11 19:58:24 EST ---

replied from gmail, but never show up, repost.

(In reply to comment #6)
> That's the "ll v.s. ll-cc" issue. I've tried to run firefox with FC_DEBUG=4 and
> have a look at the logs. I see they requested both of lang="ja-jp" and
> lang="ja" in the queries of the font. ideally it would be nice to fix this
> issue in fontconfig though, we may want to think about a workaround in firefox
> for that?
> 
> Or add a rule for "ll" in every fonts... dunno.    

If I were you, I would use the following construct
   <test name="lang" compare="contains"><string>ja</string></test>
to get around this. It is not as dangerous as some of you might think, the language tags are rather stable in Linux.

Of course, the ideal solution would be to ask fontconfig to support regular-expression in compare, something like
   <test name="lang" compare="regex"><string>^ja(-\w+)*</string></test>

--- Additional comment from tfujiwar on 2010-03-11 21:04:33 EST ---

(In reply to comment #7)
> If I were you, I would use the following construct
>    <test name="lang" compare="contains"><string>ja</string></test>
> to get around this. It is not as dangerous as some of you might think, the
> language tags are rather stable in Linux.

Me either.

--- Additional comment from tagoh on 2010-03-12 02:00:36 EST ---

(In reply to comment #7)
> If I were you, I would use the following construct
>    <test name="lang" compare="contains"><string>ja</string></test>
> to get around this. It is not as dangerous as some of you might think, the
> language tags are rather stable in Linux.

Sure. it looks good in this case but it may depends. we had experienced a bug like http://bugs.freedesktop.org/show_bug.cgi?id=23419. it may be too early saying that is stable in _Linux_.

> Of course, the ideal solution would be to ask fontconfig to support
> regular-expression in compare, something like
>    <test name="lang" compare="regex"><string>^ja(-\w+)*</string></test>    

FWIW "contains" isn't supposed to check the language name comparison only. it tries to check the code coverage with orth files first and then compare the name if it fails.

--- Additional comment from tagoh on 2010-03-12 02:03:46 EST ---

hmm, it may be good to keep this AS IS and close upstream again. I'll make a clone for vlgothic-fonts specific issue.