Adventures in working with JavaScript, Dart and Emojis

I spent the whole day today working with Strings being sent between a JavaScript serverside app and a Dart clientside app. As an industry we have been doing this forever, so you’d think it’d be easy but then along came emojis to muck up my day 🤬

Instead of writing my own primer on Strings here (and doing a bad job), let me just link to Joel Spolsky’s excellent post on the subject

This really old post still does a great job of bringing us up to speed to the Unicode world we live in today. And then came Emojis

There are numerous posts of the pain of dealing with Emojis whenever you have to because it does screwy things like combining neighboring characters to form a single emoji. This means that the length of a string, if it is just a measure of the unicode CodePoints used is different from what you would count on the screen.

This gives you whacky results like “💩”.length == 2 and generally makes working with strings just a pain even to the extent of crashing your iPhone. On the flip side some things like being able to delete family members from the 4 member family emoji with every backspace are kinda amusing, since it is it’s actually 7 characters: 4 ‘normal’ people characters and 3 invisible ‘joining’ characters in between.

If you try to delete the family emoji in whatsapp you kill them off one at a time from funny

Which brings me to today. I am playing around with moving a client/server app from JavaScript everywhere to JavaScript server and a Dart client app. In the previous iteration, strings that needed to be sent had special characters that needed to be escaped and sent across: no problem. JavaScript’s escape/unescape worked pretty well.

Moving to Dart though was a challenge, because there is no escape/unescape method. Turns out escape/unescape is best avoided anyway, and encodeURI/decodeURI is a better option. Dart has a decodeFull method on the Uri class that does the job pretty well.

Except that the characters in the list also included emojis and Dart’s Uri class doesn’t work with anything more than UTF-8 characters and crashes when encountering strings with emojis that are just ‘escaped’. This, as it turns out, is as per spec and all those fancy emoji domains that I thought used Unicode in the URI, use a different idea around Internationalized Resource Identifiers and Punycode. Thankfully passing in a URI encoded string with emojis seems to work fine and emojis come out 👍on the other side of the decode process.

While this seemed to work at that point, passing the decoded string to my Yaml loader crashes the app again (is Yaml supposed to be restricted to Ascii/Utf-8 ? ). But that is a problem for a different day.

For now, I have decided to just convert emojis to shortcodes for the transit and remap them to emojis on the other side. Its not pretty but it works.

Oh and in the meanwhile, if you want to know how to loop through a String with emojis in Dart, you can do that by looking through the Runes in a String:

	String s = "😀 hello";
	s.runes.forEach((int i){
	String x = String.fromCharCode(i); // Get emoji as 1 string and now 2 CodePoints
	}

view raw emoji.dart hosted with ❤ by GitHub