Emoji support is broken in Tor Browser 5.5

It seems our font fingerprinting defense breaks emoji support: https://blog.torproject.org/blog/tor-browser-55-released#comment-153807. See comment:10:ticket:17270 for one example.

added TorBrowserTeam201603 component::applications/tor browser owner::tbb-team parent::18097 priority::medium severity::normal status::needs-revision tbb-5.5-regression tbb-fingerprinting-fonts tbb-usability-website type::defect labels

I see no problem with the face characters in that comment. Gnu/linux, 64 bits.

Trac:

Trac:

Trac:

There's a unicode emoji table here: http://apps.timwhitlock.info/emoji/tables/unicode

Here's what I get on different platforms:

=Linux=	=Mac=	=Windows=

So it looks like we need to fix Mac and Windows, but Linux is OK.

The last commit in this branch contains a fix for Windows and Mac. It has been applied on top of my fix for #18169 (moved):

https://github.com/arthuredelstein/tor-browser/commits/18169

Trac:
Status: new to needs_review
Keywords: N/A deleted, TorBrowserTeam201601R added

Replying to arthuredelstein:

So it looks like we need to fix Mac and Windows, but Linux is OK.

That's actually not true. Surprisingly I have at least one Linux box where emoticons are broken. Is is running Debian testing. That said I realized that it can be a bit hard to see this if one is upgrading from 5.0.7 as it seems to work then but after shutting the browser down and restarting it no emoticons are shown. (This reminded me at one of the failure modes of our start script. :/)

Trac:
Status: needs_review to needs_revision

Replying to gk:

Replying to arthuredelstein:

So it looks like we need to fix Mac and Windows, but Linux is OK.

That's actually not true. Surprisingly I have at least one Linux box where emoticons are broken.

Me too (uncommon Linux distribution). If the native emojis work, that's a leak of installed fonts, right (so a bug)?

Replying to gk:

Replying to arthuredelstein:

So it looks like we need to fix Mac and Windows, but Linux is OK.

That's actually not true. Surprisingly I have at least one Linux box where emoticons are broken. Is is running Debian testing. That said I realized that it can be a bit hard to see this if one is upgrading from 5.0.7 as it seems to work then but after shutting the browser down and restarting it no emoticons are shown. (This reminded me at one of the failure modes of our start script. :/)

You're right, this happened to me. Once I shutdown the browser and started it again, the Emojis were no longer available.

So here's a patch for Linux:

https://github.com/arthuredelstein/tor-browser-bundle/commit/18172

I built this version and confirmed that the Emojis are displayed, using the "Noto Emoji" font.

(I should mention, I decided against bundling Noto Color Emoji, because it is 2.3 MB. Instead I am bundling Noto Emoji, which is only 280kB.)

Replying to arthuredelstein:

Replying to gk:

Replying to arthuredelstein:

So it looks like we need to fix Mac and Windows, but Linux is OK.

That's actually not true. Surprisingly I have at least one Linux box where emoticons are broken. Is is running Debian testing. That said I realized that it can be a bit hard to see this if one is upgrading from 5.0.7 as it seems to work then but after shutting the browser down and restarting it no emoticons are shown. (This reminded me at one of the failure modes of our start script. :/)

You're right, this happened to me. Once I shutdown the browser and started it again, the Emojis were no longer available.

So here's a patch for Linux:

https://github.com/arthuredelstein/tor-browser-bundle/commit/18172

I built this version and confirmed that the Emojis are displayed, using the "Noto Emoji" font.

I think we can start with this font as a fix for this bug. Two things regarding the patch:

1. You don't use NOTOEMOJIFONT_VER anywhere, so we don't need it. Which reminds me that the same seems to hold for NOTOCJKFONT_VER.
2. Please remove the superflous newline below +- "NotoEmoji-Regular.ttf"

Carrying over review tickets.

Trac:
Keywords: TorBrowserTeam201601R deleted, TorBrowserTeam201602R added

Will this fix resolve these 9 widespread "icons" https://blogs.windows.com/windowsexperience/2016/01/27/announcing-windows-10-insider-preview-build-14251/?

Replying to gk:

Replying to arthuredelstein:

Replying to gk:

Replying to arthuredelstein:

So it looks like we need to fix Mac and Windows, but Linux is OK.

That's actually not true. Surprisingly I have at least one Linux box where emoticons are broken. Is is running Debian testing. That said I realized that it can be a bit hard to see this if one is upgrading from 5.0.7 as it seems to work then but after shutting the browser down and restarting it no emoticons are shown. (This reminded me at one of the failure modes of our start script. :/)

You're right, this happened to me. Once I shutdown the browser and started it again, the Emojis were no longer available.

So here's a patch for Linux:

https://github.com/arthuredelstein/tor-browser-bundle/commit/18172

I built this version and confirmed that the Emojis are displayed, using the "Noto Emoji" font.

I think we can start with this font as a fix for this bug. Two things regarding the patch:

1. You don't use NOTOEMOJIFONT_VER anywhere, so we don't need it. Which reminds me that the same seems to hold for NOTOCJKFONT_VER.
2. Please remove the superflous newline below +- "NotoEmoji-Regular.ttf"

Here's a patch with these things fixed: https://github.com/arthuredelstein/tor-browser-bundle/commit/18172+1

Trac:
Status: needs_revision to needs_review

Thanks, this is commit c885551e69ca5043d0af0de6940e250968e6f7e5, 7aa3abf90371fb82518edf5d54e862e91744c97b and ee6fd02653fc0e396bee5daa034bd00be890ccb5 on master/maint-5.5/hardened-builds.

Trac:
Status: needs_review to closed
Resolution: N/A to fixed

Trac:
Keywords: TorBrowserTeam201602R deleted, TorBrowserTeam201602R tbb-5.5-regression added

No NotoEmoji-Regular.ttf in TBB 5.5.1 for Win, no emoji.

Replying to bugzilla:

No NotoEmoji-Regular.ttf in TBB 5.5.1 for Win, no emoji.

It appears that the Emoji font I add to the whitelist for Windows, Segoe UI Emoji, is not always available on Windows 7. So there are a couple of alternatives:

1. Whitelist Segoe UI Symbol instead. This covers a lot of Emoji characters, although they are monochrome.
2. Whitelist both. This of course allows Windows 7 and earlier to be distinguished from later versions, but there are other ways to distinguish these fonts.
3. Bundle NotoEmoji-Regular.ttf with Windows and remove Segoe UI Emoji from the whitelist. This makes the Emoji fonts identical on all versions of Windows. The Noto Emojis don't look the quite the same as the native Windows fonts.

I'm inclined to go for the third option. Does anyone else have an opinion?

Trac:
Status: closed to reopened
Resolution: fixed to N/A

Replying to arthuredelstein:

Replying to bugzilla:

No NotoEmoji-Regular.ttf in TBB 5.5.1 for Win, no emoji.

It appears that the Emoji font I add to the whitelist for Windows, Segoe UI Emoji, is not always available on Windows 7. So there are a couple of alternatives:

Windows 8 seems to be affected as well.

1. Whitelist Segoe UI Symbol instead. This covers a lot of Emoji characters, although they are monochrome.
2. Whitelist both. This of course allows Windows 7 and earlier to be distinguished from later versions, but there are other ways to distinguish these fonts.
3. Bundle NotoEmoji-Regular.ttf with Windows and remove Segoe UI Emoji from the whitelist. This makes the Emoji fonts identical on all versions of Windows. The Noto Emojis don't look the quite the same as the native Windows fonts.

I'm inclined to go for the third option. Does anyone else have an opinion?

Hm, this is pretty unfortunate especially as we probably want to test whether the bundling mechanism is working on Windows as well properly. But, yes, I think we could try that route. I am not really inclined to wait for that for 5.5.1, though, as #18169 (moved) and #18168 (moved) are more important and I would like to see some testing with this approach.