protonscr

Special characters look scrambled

steamopen reviewedSteam client
ValveSoftware/steam-for-linux#1889 · opened 2013-02-19 by spacelike · updated 2022-01-18 · 38 comments · github
Sspacelike 2013-02-19 github

text

In the image above the top part is how the text is displayed in steam, while the bottom shows what was pasted into the chat box.
(Here it is in case anyone wants to copy paste it into their steam to help test this: こんにちは).
NOTE: This happens with even special unicode symbols like stars and symbols.

This was tested with both STEAM_RUNTIME=0 and 1.

Also, I notice that if more characters are pasted the same characters seem to be scrambled in a different looking way.

I am using the OpenBox window manager and I do have fonts capable of displaying these particular characters as I have everything set up for displaying and typing japanese characters.


Processor Information:
Vendor: GenuineIntel
Speed: 2668 Mhz
8 logical processors
4 physical processors
HyperThreading: Supported
FCMOV: Supported
SSE2: Supported
SSE3: Supported
SSSE3: Supported
SSE4a: Unsupported
SSE41: Supported
SSE42: Supported

Network Information:
Network Speed:

Operating System Version:
"Gentoo Base System release 2.1" (64 bit)
Kernel Name: Linux
Kernel Version: 3.6.11-gentoo
X Server Vendor: The X.Org Foundation
X Server Release: 11301000
X Window Manager: Openbox
Steam Runtime Version:

Video Card:
Driver: NVIDIA Corporation GeForce GTX 295/PCIe/SSE2

Driver Version:  3.3.0 NVIDIA 313.18
Desktop Color Depth: 24 bits per pixel
Monitor Refresh Rate: 60 Hz
VendorID:  0x10de
DeviceID:  0x5e0
Number of Monitors:  1
Number of Video Cards Not Detected
Primary Display Resolution:  1920 x 1080
Desktop Resolution: 1920 x 1080
Primary Display Size: 18.78" x 10.55"  (21.54" diag)
                                        47.7cm x 26.8cm  (54.7cm diag)
Primary Bus: PCI Express 16x
Primary VRAM: 896 MB
Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
Audio device: Realtek ALC1200

Memory:
RAM: 5961 Mb

Miscellaneous:
UI Language: English
LANG: en_US.UTF-8
Microphone: Not set
Total Hard Disk Space Available: 460352 Mb
Largest Free Hard Disk Block: 365024 Mb

Installed software:

Recent Failure Reports:

TTele42 2013-02-19 github

I grabbed the test text, threw it into the chat window, and it rendered fine.

Processor Information:
    Vendor:  AuthenticAMD
    Speed: 3600 Mhz
    8 logical processors
    8 physical processors
    HyperThreading:  Unsupported
    FCMOV:  Supported
    SSE2:  Supported
    SSE3:  Supported
    SSSE3:  Supported
    SSE4a:  Supported
    SSE41:  Supported
    SSE42:  Supported

Network Information:
    Network Speed:  

Operating System Version:
    "Gentoo Base System release 2.1" (64 bit)
    Kernel Name:  Linux
    Kernel Version:  3.6.11-gentoo
    X Server Vendor:  The X.Org Foundation
    X Server Release:  11301000
    X Window Manager:  Mutter (Muffin)
    Steam Runtime Version:  <Runtime disabled>

Video Card:
    Driver:  NVIDIA Corporation GeForce GTX 460/PCIe/SSE2

    Driver Version:  4.3.0 NVIDIA 313.18
    Desktop Color Depth: 24 bits per pixel
    Monitor Refresh Rate: 60 Hz
    VendorID:  0x10de
    DeviceID:  0xe22
    Number of Monitors:  2
    Number of Video Cards Not Detected
    Primary Display Resolution:  1920 x 1080
    Desktop Resolution: 3840 x 1080
    Primary Display Size: 20.04" x 11.26"  (22.95" diag)
                                            50.9cm x 28.6cm  (58.3cm diag)
    Primary Bus: PCI Express 16x
    Primary VRAM: 1024 MB
    Supported MSAA Modes:  2x 4x 8x 16x 

Sound card:
    Audio device: Loopback Mixer

Memory:
    RAM:  16035 Mb

Miscellaneous:
    UI Language:  English
    LANG:  en_US.utf8
    Microphone:  Not set
    Total Hard Disk Space Available:  905512 Mb
    Largest Free Hard Disk Block:  27425 Mb

Installed software:

Recent Failure Reports:
Jjohndrinkwater 2013-02-19 github

Works for me too.
Screenshot from 2013-02-19 11:56:31
System info https://gist.github.com/johndrinkwater/4721913

Jjohndrinkwater 2013-02-19 github

If you put regular latin characters before and after the Hiragana, are they scrambled too, or does it just occur for those characters in a line?

HHirato 2013-02-19 github

I can confirm, and it's just those characters. They also get re-scrambled when you add additional characters. but they're the only ones drawn completely corrupted.

On a similar note, ascii/latin/whatever is also completely screwed up if you fail to install ttf-ms-fonts or ttf-win7-fonts, and it's been like that since the first days of the beta (although it no longer crashes in that case). I don't like having them around, they make text on the internet look rather horrific IMO.

I get the impression that steam is tailored for very, very specific fonts, and if you want to use something that steam wasn't specifically tested with/tailored to, you're basically out of luck.
I can't help but wonder if the texture is drawn or expanded with a bad stride or something; the asserts and stuff they've since removed talked a lot about the characters being "too high."

MMajkl578 2013-02-19 github

@johndrinkwater: When I was in that chat room, I didn't see those special characters in your name.

Jjohndrinkwater 2013-02-19 github

@Majkl578 its ok not to see them if you dont have the fonts, they should render as square blocks, see https://github.com/ValveSoftware/steam-for-linux/issues/1489#issuecomment-13477145

Jjohndrinkwater 2013-02-19 github

@Hirato it could be ttf-ms-fonts having buggy glyph variables? @spacelike do you have those fonts? Would you put the output of fc-list : family style file spacing into a gist

HHirato 2013-02-19 github

@johndrinkwater Emphasis on the word "fail," but since you had trouble parsing that, let me rephrase it.

Steam requires very specific fonts to draw text correctly.
The packages, ttf-ms-fonts and ttf-win7-fonts provide sufficient fonts to draw the latin sets correctly within Steam.
Without them the text is messed up in various inconsistent ways, ranging from pixel sized dots, rampant corruption as seen in the screenshot spacelike provided in the bug report above, or its simple absence.... but sometimes it works just fine for the library tab, but nowhere else.

Jjohndrinkwater 2013-02-19 github

@Hirato there’s no need to be a dick. I have neither of your suggested fonts and Steam renders fine, as you can see from my screenshot.
Have you considered one of your non-latin installed fonts have incorrect glyph coverage and thats why you need to install other fonts to correct it?
Whats the output from fc-match :family=Arial with & without those fonts.

HHirato 2013-02-19 github

Considering that the text is undeniably corrupt, not rendering as empty squares, and that nothing else has any text rendering issues, I have some serious qualms about that hypothesis.

The requested gist: https://gist.github.com/Hirato/4986187
If you want, I can give you a version with the --all flag on too.

Jjohndrinkwater 2013-02-19 github

Just getting useful information for the bug…

Sspacelike 2013-02-19 github

Thanks for the replies john.
Putting other characters in front doesn't change anything, and here is the gist:
https://gist.github.com/spacelike/4986854

out of curiosity, what fonts do you have that allow you to display the characters in your name? They show ups as regular boxes for me (not scrambled). Perhaps installing those could solve the main issue as well.

Jjohndrinkwater 2013-02-19 github

@Hirato the font Steam is picking for you is a bitmap font from the package xfonts-75dpi. You havent said which distribution you’re on but at the very least Liberation or DejaVu should be installed and would be picked up first…

Screenshot from 2013-02-19 15:29:01

Jjohndrinkwater 2013-02-19 github

@Spacelike I use AndroidEmoji.ttf which gives me access to those characters. I notice you have the same package installed, if you remove xfonts-75dpi does that fix the problem? You may have to run fc-cache --force and close your X session for it to take effect.

HHirato 2013-02-19 github

Oh wow, this is very interesting. Is Steam trying to open bitmap fonts as a TTF or something?

I purged xorg-fonts-75dpi and xorg-fonts-100dpi and suddenly steam has text with my preferred font, I almost can't believe my eyes! (That is DejaVu Sans by the way)
Unfortunately Japanese/Korean/Chinese characters are still corrupt though.

Distro is Arch by the way.

HHirato 2013-02-19 github

I got a request to run fc-match -v :family="WenQuanYi Zen Hei" via email, though I don't see the comment above.
Here it is anyway: https://gist.github.com/Hirato/4987390

Jjohndrinkwater 2013-02-19 github

I deleted it ’cause it was wrong, trying to locate the name of a font that would trigger on the remaining languages.
Do you have xfonts-unifont installed, and does removing it cure the issue?

HHirato 2013-02-19 github

There is one more package with fonts, xorg-fonts-misc - it appears to have bitmap fonts for the affected languages.
Just one problem, If I remove that it will end up purging just about everything I have installed; xorg, my whole desktop environment, the nvidia driver, etc.

Jjohndrinkwater 2013-02-19 github

Don’t remove that then :^) whats your output for fc-list : family style file spacing now ?

HHirato 2013-02-19 github
Sspacelike 2013-02-19 github

@johndrinkwater
I removed the 75dpi fonts and nothing changed.

HHirato 2013-02-19 github

I've also poked around fontconfig's options and ran the following sudo ln -s /etc/fonts/conf.avail/70-no-bitmaps.conf /etc/fonts/conf.d/70-no-bitmaps.conf

Bitmap fonts should no longer be cached, and I've even reinstalled the bitmap fonts I just removed; they're not interfering with the rendering of the latin/ascii characters at all, and don't show up in fc-list either.
Japanese and Chinese is still corrupt, but Korean is completely missing now. So I'm not sure if removing xorg-fonts-misc would've helped, even if I could remove it.

I'll keep playing around for now.

HHirato 2013-02-19 github

@spacelike Just wondering, what japanese/chinese/korean/whatever fonts do you have installed?

@johndrinkwater
I've just replaced ttf-sazanami with ttf-hanazono, japanese and chinese characters now appear correctly in steam. Very strange methinks that sazanami was good enough for everything else, yet not for steam.
The Korean text was similarly "corrected" by installed ttf-baekmuk, but that appears to be a genuine issue of actually missing a font. Rather interesting that it just turned into a tiny block of nothing...
pics

It appears this issue is a combination of a few things.

  • weird, and plain dumb behaviour when encountering bitmap fonts.
  • improper fallbacks when unable to render characters; It's either noise or empty space (and in the latter case, it takes up a miniscule fraction of the expected space).
  • general steamy weirdness, at least I have no other explanation for why the Sazanami fonts couldn't render even a single japanese/chinese character inside Steam, yet managed everything thrown at it outside of Steam.
Sspacelike 2013-02-20 github

@Hirato
I have the package "kochi-substitute" which gives me allows me to see japanese characters in all other programs. I also manually added the font "Meiryo".

Ggdrewb-valve maintainer 2013-02-20 github

Steam uses Freetype and isn't aware of bitmap vs. non-bitmap in its font handling. Freetype handles fallback character rendering also, so these would appear to be Freetype bugs (which is odd, since tons of code uses Freetype and such bugs would show up in lots of ways). These issues don't repro for me with or without xfonts-75dpi, so we'll have to work on figuring out exactly configuration what triggers the issue.

Sspacelike 2013-02-23 github

It's working now.

I was originally not opted into the beta, but I heard there was an update today for people in the beta. So I opted in and updated. Now characters are displayed properly.

Ggdrewb-valve maintainer 2013-02-24 github

OK, thanks. @Hirato , are you still seeing this problem?

HHirato 2013-02-24 github

Yes

If I allow bitmap fonts (aka, undoing the 70-no-bitmaps.conf symlink I did earlier)
If I use ttf-sazanami over ttf-hanazono and other japanese fonts (should read ですですですですです)

EDIT: It's the same for both available "stable" and "beta" builds

HHirato 2013-02-24 github

Also, if it helps any, I had the following font packages installed...

  • ttf-dejavu
  • gsfonts (ghostscript, needed by the gimp and others)
  • xorg-fonts-75dpi
  • xorg-fonts-100dpi
  • ttf-sazanami (for japanese/chinese characters)
  • ttf-bitstream-vera
  • ttf-freefont

The changes to my system from when it's failing to where it works

  • symlinked /etc/fonts/conf.avail/70-no-bitmaps.conf into /etc/fonts/conf.d
  • installed ttf-baekmuk (for korean characters)
  • replaced ttf-sazanami with ttf-hanazono

I also have a TONNE of LaTeX fonts, but those don't count, they're not indexed by fontconfig by default, and I never enabled the option.

Llight4 2013-07-22 github

steam

Ssahilsinha 2013-07-27 github

This looks like a unicode issue. You need unicode support in your OS. https://wiki.gentoo.org/wiki/UTF-8 gives a great overview of what unicode is and what you can do to display special characters.

Aalex7kom 2013-07-27 github

I have the same issue. Chinese and Japanese characters look scrambled, but Korean is OK. No issues with cyrillic characters either.
System Info: https://gist.github.com/Alex7Kom/458c9afea0413bcc7aa6

@sahilsinha What do you mean by 'unicode support in OS'? These characters look perfectly in any other program and even in Steam Web Browser but not in Steam UI.

Ssahilsinha 2013-07-27 github

@Alex7Kom I should have read more carefully looks like I was wrong. If that is the case perhaps people simply don't have the fonts installed? A friend had an issue with UI fonts last night which was fixed by installing font-bitstream-100dpi and corefonts.

Aalex7kom 2013-11-12 github

Removing ttf-sazanami-mincho, ttf-kochi-gothic, and ttf-kochi-mincho resolved this problem for me. Not sure how I missed this part of the discussion above.

Sspacelike 2014-10-02 github

I think the real issue behind all of the bug reports of scrambled text like this one and
https://github.com/ValveSoftware/steam-for-linux/issues/3311
is still simply bitmap fonts, as has been discussed, and nothing more.

I've recently come to understand the issue a lot better and I don't think I've seen anyone concisely summarize what is going on in any of these bug reports, so here is what I think is happening exactly.

We have been told to set 70-no-bitmaps.conf in the fontconfig to remove bitmap fonts, and that fixes the issue for some people.
However, this fontconfig setting doesn't actually prevent embedded bitmap fonts from being used.
(I'm not sure if it's supposed to do that or not as someone mentioned something about a fontconfig bug? But it definitely doesn't block embedded bitmaps on my system)

So, even though steam is selecting Arial which has no embedded bitmaps, when Arial is missing glyphs the system uses a fallback font. If that fallback font containes embedded bitmaps then the text looks scrambled.
This explains why large text is usually rendered fine by people who experience the bug because embedded bitmaps usually only exist for small sizes (around 10-16px) above and below which the truetype font is used which steam can render.

As a demonstration of this, I created a steam skin set to use as the primary font (not fallback) one that I know contains embedded bitmaps (ipamonafont). The result is that even english characters are scrambled at small sizes, yet displays fine for large sizes for which no embedded bitmap is available in the font.
http://i.imgur.com/7aROA02.png?1
On the other hand, if I set the skin to use a default font without embedded bitmaps and containing a larger unicode coverage than the default Arial (to prevent fallback) then all characters display correctly at all sizes.

One other problem to note, is that it is currently recommended to Linux users to install the font wqy-zenhei as a requirement for displaying fonts correctly in some source engine games.
However, this font also contains embedded bitmaps, and so although it fixes font rendering in-game it can break it on the client if it's used as fallback.

Once this issue is known it's fairly easy to work around it, so I wanted to leave this information here for completeness sake in case anyone needs a workaround until a proper fix comes out.

Jjdorn-eu 2015-04-30 github

Alright because I'm facing this exact same issue right now I think it's appropriate to leave my experiences with it here. I agree the cause of this is probably because of certain fonts, because it was like this for in fresh Manjaro x64 install:

  • Start Steam. Fonts are scrambled.
  • Install wqy-zenhei. Restart Steam. Fonts look fine.
  • Install ttf-ms-win8-japanese. Restart Steam. Fonts are scrambled.
  • Remove ttf-ms-win8-japanese. Restart Steam. Fonts look fine.
Nniihelium 2016-02-28 github

I guess it may be the same issue. I'm using cyrillic symbols in my nickname and it doesn't showing correctly only in main navbar https://imgur.com/pFk1r21

SSlider-Whistle 2022-01-18 github

Still having this issue, don't really want to have to uninstall any of my fonts.