RPGDX > Forums > Design and programming > Supporting Unicode In Games

etscrivner · Posted: Fri Aug 24, 2007 8:52 pm Post subject: Supporting Unicode In Games

I've been doing a lot of development on a new game engine recently, and one of my design goals was to be able to support unicode (files, text, input etc...) in a fairly transparent way. I'm currently trying to decide between the three main Unicode encodings UTF-8, UTF-16 and UTF-32, and I was curious what good/bad experiences have people here had with unicode in the past, and is it worth it for an open-source game? Also, what format did you decide on if any?

Ninkazu · Posted: Fri Aug 24, 2007 9:12 pm Post subject:

Unless your engine is good enough for the whole world to use, don't bother is my opinion. If you get enough demand, add it or let someone else add it. Just don't bake your file io around one method (abstract it) and you should be fine.

etscrivner · Posted: Sun Aug 26, 2007 7:53 pm Post subject:

Ninkazu: That's very true, plus with UTF-8 support coming in C++ I think it's not something I should be too concerned with implementing right now. It's not the world's greatest engine, but a considerable amount of thought has gone into it, and swapping out strings would be trivial if necessary.

cowgod: Actually using Unicode, especially with 8-bit ASCII is quite trivial. The UTF-8 encodings are precisely the same as long as you stay within the range 0x00 - 0x7F, which occurs so long as you aren't using an extended character set and even then it's fairly trivial to support.

Bjorn · Posted: Tue Aug 28, 2007 10:28 am Post subject:

Since Guilllaume has recently added internationalization to The Mana World, we are at the moment interested in looking into moving to UTF-8 for the strings displayed in our GUI. However, we haven't got a solution so far.

In our case, people had already started forking the project just to translate all the strings everywhere (I know of at least a complete French version). In order to keep these efforts together, adding support for internationalization is very important.

etscrivner · Posted: Wed Aug 29, 2007 1:42 am Post subject:

Bjørn: Yeah, it seems like guichan wasn't exactly designed with unicode support in mind - so you would probably have to recode some major chunks of it or roll a new gui framework...both of which are not fantastic options.

If you guys could use some help I'd be more than happy to throw together the UTF-8 code I've already written and make a more complete string class to support widening/shortening strings and all that.

Bjorn · Posted: Fri Aug 31, 2007 1:08 am Post subject:

Hmm, I think the solution came by using UTF-8 encoding for storing the strings pulled by gettext and using the UTF-8 font render calls of SDL_ttf. Now all we need is a font which supports enough characters, I think for example Dejavu will suffice for now.

Probably now we still need to also support UTF-8 input.

Hajo · Posted: Mon Sep 10, 2007 12:31 pm Post subject:

My only successful project seemed to be particularly successful in japan and honkong, so there was pressure to move to unicode. It happened finally after about 7 years of development.

So yes, it can be useful. And it helps if you think about it early, but it's not a show-stopper. Do you aim at the asian market? There you'll need it for sure. Rest of the world can live with 8 bit encodings, it seems.

Adam · Mage Joined: 30 Dec 2002 Posts: 416 Location: Australia

Another fun note is that you want to go full hardcore support of asian languages you need a 16 pixel high font size. As one of the characters has 8 horizontal lines in it...
_________________
https://numbatlogic.com

Ninkazu · Posted: Tue Sep 11, 2007 1:32 pm Post subject:

Like Adam said, or go smaller and anti alias the fuck out of the characters.