The Ur-Quan Masters Discussion Forum

Hello!

I know you guys don't want to make any serious modifications to UQM until the 1.0 release, but I and my friends just can't wait :) and the potential gain (IMHO) from the proposed changes is very noticeable, while the changes themselves are very simple. So I thought it would be good to ask your opinion on this.

My daughter likes the game alot, but alas he doesn't know English :) Also some of my frends that even know English to some extent won't play the game because they have to think hard on every phrase, and this is not fun at all; at the same time they like the game as a whole.

So I thought a little localization for the game would help spread the word about the game, and - who knows - bring new developers to help.

---

Now about the actual implementation. I thought it wouldn't be too intrusive to make it possible to translate at least the most important parts of the game to other languages. In general, the main part that matters most is the text of the dialogs. And since dialogs are separated into text files, and also every letter is separated into a .png file, it is possible to just create subdirectories like en/ de/ fi/ and so on under the same directory where text and font files are placed, and put the additional font characters and localized text files there. Something like this (I don't have the source tree at hand, so I may misspell some file names here):

065.png (letter 'A')
066.png (letter 'B')
mycoon.txt

ru/mycoon.txt (russian dialogs)
ru/128.png (cyrillic letter 'A')
ru/129.png (another cyrillic letter)

de/mycoon.txt (German dialogs)
de/155.png (some german letter)
de/159.png (perhaps another german letter)

The code will have to be modified just a little:

First there should be a command-line argument to switch to a specific language (--lang=ru), also it would be good to have some platform-specific function like GetUserLanguage() to fetch current language from environment (LANG), registry and so on. The default implementation can just return "en".

Then the dialog loader will have to be modified so that it first looks under the $LANG/ directory for the respective text file, and if it is not found, load the default (English) file. Same about the font loader - it will have to load all characters that are in main directory and additional characters that possibly are under the $LANG/ subdirectory.

These simple modifications already will allow translators work to begin. What do you think of the idea?

There was a Russian translation project (http://uqm.stack.nl/cgi-bin/yabb/YaBB.pl?board=Gendiscuss;action=display;num=1068293821) for UQM before. I don't know how does it fare now, but I have decided to participate in it once, and translated some conversations for the Ur-Quan, but abandoned it very soon. Maybe I'll return to it later though. :)

I think the .png "fonts" are a serious problem. How can one bind specific symbols to png-letters? Then there's a whole charset mess...

I think the best approach would be to give some truetype love for the dialog engine (looking there right now), then we can simply translate the messages with help of unicode and put all different fonts and nice utf-8 encoded messages into language-packs. Umm, Utopia...

There's no need to get involved in that TrueType madness... Its a lot simpler than that.

The character images are called like 42.png, 64.png and so on. You just have to provide your characters with the names corresponding to the code of the character, and ensure all text in one language is written using same encoding, thats all. The code 128 may mean one letter in a language, and a totally different letter in a different language - it's not a problem at all, since the letter images will be placed under ru/, de/, fi/ and so on directories.

Forget Unicode, TTF and all other buzzwords - they will take a lot of time and pay nothing.

Just read that old thread about translation and have some comments.

First, it is a bad thing that the XML translation is put as a obstacle for the translators. If the XML translation will be eventually automated, it's not a problem to translate several sets of files instead of one.

Second, the game can just have zero knowledge about Unicode, UTF-8 and so on. Every language can choose an appropiate (single-byte) encoding, and draw the appropiate character PNGs with appropiate filenames (which are derivates of the character code). Also, if input will be desirable, the appropiate keyboard mapping will have to be done somewhere (not a priority task though).

For languages with more than 255 characters it is possible to use something simple to decode like UTF-8. Then we'll have just more characters to load, thats all.

Right-to-left scripting is something totally unknown to me, but I presume this also can be defined as a per-language attribute, and the text renderer will have to be slightly modified to account for that. Not a top priority task (for me, at least) though.

Okay, I'll try to start doing something this evening and see what problems will pop up.

Yep, looks like trutype will not payoff. Agreed... Still some work is required to draw the per-font per-language letters. In which case, a lang-pack will contain, additional letters for png's, the translated messages and maybe the translated voices (loading of the former two needs implementation?)

Unicode also is out of the ground, because utf-8 and utf-16 are multibyte encodings and I think they're unusable, if a character is picked by it's byte number. This is ok for most european languages though to use iso-* charsets...

Update: looks like the game does not even need to know about lang-packs -- the addon-loader is supposed to _replace_ files from content packages with files from the addons, right? :) I think it can replace just anything: fonts, txts, pics (like the ones used in game menu).

Well, it's not so straight-ahead as using single-byte encodings but still it's achievable. UTF-8 is an encoding that use bytes, and the upper half of the code table (128-255) are really prefixes, and you can easily decode UTF-8 characters to a code in the range 0 ... 2^31-1.

However, multi-byte character set support is not a stringent need right now, it can be added later if the translation project turns to be viable.



Yes, looking at uio_mountDir procedure in io.c, it has options to mount a zip file on top of a existing virtual directory, I think that's exactly what we want. And mountContentDir() function actually uses this option, so addons will perfectly suit our needs. So basically the --lang switch will be equivalent to --addon.

If you're going to do it this way you don't even need a separate switch. You could use --addon and make a package that works even with the current release.

I'm the one who wrote uio btw.

Ah it looks nice indeed. I wrote once ago a very similar system called VFS (Virtual File System) for the Crystal Space project :)

I agree the --lang option could be avoided. I will try to take this route and see if no new problems will pop up..

---

Okay, here's the first problem. The addons are mounted with the uio_MOUNT_BELOW option, which means, addons can't override files from the base content. This is very bad not only from the localization point of view. Any special reasons to use the BELOW option instead of the uio_MOUNT_ABOVE? Also the base content dir is mounted as uio_MOUNT_TOP; I think it would be better to have it mounted as uio_MOUNT_BOTTOM although it shouldnt matter (or TOP/BOTTOM are absolute? or they have effect only at the uio_mountDir time?)

Heh. VFS was the first name I thought of using too, as it describes the system best. But I considered that too confusing as it's used in many places, including in the Linux kernel itself.

I have created a page on the Ultronomicon which describes uio from a user's perspective, on http://uqm.stack.nl/wiki/index.php?title=Content_Management.


This is only a problem if you use the content unzipped. If you use zipped content, the .zip files will be in the content/packages dir, which you can override.
It was an intentional design decision. The idea was that people should use the .zip files still zipped, and to use separate files only where you want to override the packages.
But I agree that it can be a problem in some circumstances, like for people who run the game directly from CVS (like myself), or for distributions that for some reason or another extract the .zip files (like Debian).
I have been planning on doing it a bit differently in a future release. Stay tuned.


uio_MOUNT_TOP and uio_MOUNT_BOTTOM are absolute. uio_MOUNT_ABOVE and uio_MOUNT_BELOW specify that the new mount should end up directly above cq. below the mount specified in the 9th argument.
And indeed, this refers to the locations at mount time. If later another directory is mounted below the same location, then it will end up above one that was mounted below that location earlier.

Thanks for clarification. I'm indeed running the CVS version, so I did a temporary hack to mount the addon packages with the ABOVE flag, and it works okay now. Waiting to see what solution will you propose in the next version :)

After that everything worked as by magic... I just added some dummy character images with codes above 128 under the respective .fon directories, and translated some text to Russian... it worked! The only problem I've encountered so far is that the dialog history (the one that pops instead of the alien when you press Space) for some reason doesn't seem to like characters with codes above 128 (just skips them), although the text that displays above the alien is painted perfectly.

I almost finished a small tool that takes the .fon directory and combines all the characters on a single canvas, and saves it into a .PNG file; also it does the reverse process (e.g. splits one PNG into multiple characters). I hope I did not reinvent the wheel. I'll start painting the Cyrillic character set as soon as time will permit, there are a lot of fonts to draw, luckily some Latin glyphs are same or very similar to some Cyrillic characters... that will simplify my job.

-------------

Okay, so I'm inviting everybody interested in the Russian translation of Star Control to start a discussion on the subject... I'd propose a irc conference, as the forums discussions proceed slow and if we disagree we can never came to a decision. For example, we can meet on irc.freenode.net, channel #uqm, I'll be there everyday from 12:00 to 24:00 Moscow time. If it makes sense, we can set up some dedicated Web page for this.

--------------

Another thing I wanted to ask knowledgeaboe StarCon people what special meaning have the names of some races... for example it was a nice surprise to discover the deciphering of the VUX name... are there any other hidden jokes inside names, etc?

I think race names are the first and one of the most important things to translate correctly. Sure, we can just transliterate, but this is not so good as it can be.

The VUX aren't actually *named* Very Ugly Xenoforms (in English). That's just what Rand called them, and it's a rather forced joke (you never hear any other extraterrestrials or aliens being called "xenoforms").

The alien names do reference things in English, but most of them are rather obscure, except for Syreen being an obvious takeoff of "siren". (Not counting the ones that are actually supposed to be Greek words: "Mycon" comes from the Greek for "fungus", "Androsynth" "constructed man", "Slylandro" "sky man". See the stuff I've been saying on Ultronomicon about how Humans appear to have a convention of using Greek to name alien things.) "Ilwrath", I remember, comes from a Scottish friend of Paul and Fred whose last name was McIlwraith, "Yehat" seems to come from the Yeehat tribe of Indians in Jack London's Call of the Wild, "Melnorme" comes from singer Mel Torme (seriously), and so on. The references are obscure enough that I wouldn't worry about "translating" them, unless you're a really hardcore Tolkien-style linguist who wants to translate the words into words that "feel" the same way in Russian, which seems to me to be an impossible task. (As in, "Ilwrath" evokes "ill" and "wrath", and "Spathi" invokes "spastic" and "pathetic" and "pathology", and "VUX" invokes "vile" and "yuck" and "suck", so we have to find equivalent made-up words that correspond to the same set of *Russian* words... argh.)

My recommendation would be just to do a straight transliteration. (If you want to find a way to wrangle VUX so that it becomes an acronym in Russian, be my guest, but I'd rather just rewrite that story -- which isn't ever in the game's actual dialogue anyway.)

Thanks for your in-depth reply, I'm impressed :)

I think I'll mostly go by transliteration, except some slight modifications where I can add a little expression by just adding/moving a letter and such. Sure, it's impossible to translate the feeling and the associative row, I just hope the translated names will have their own associations (and here's where a careful choice would help a little).

So what, are there any volunteers? I'm not in a urgent need, I can finish the whole translation task myself, but if anybody wants to feel himself -- even marginally -- involved in this task, I'll be glad to hear their opinions on #uqm, irc.freenode.net from 12:00 to 24:00 GMT+4 every day :) So far only Meep-Eep visited me there, just to tell me there's the #sc2 channel there too where The Real Guys are sleeping online :)

Hi all,

I was thinking about a spanish translation. I can manage pretty well with the translation itself, but I'm still a rookie in coding and don't know in depth the source of UQM yet.

The spanish case turns to be quite simple -at least compared to cyrilic languages-, as the special characters (á,é,í,ó,ú,ü and the one and only "ñ") are easy to obtain editing regular alphabet PNGs. So I was trying to translate the first dialogue scene -for easier playtesting- between Fwiffo and the Player. Already edited part of the txt file, and added to the spathi.fon folder new PNGs named according to the ASCII codes of the special characters above. However, there's something I'm missing, maybe related to the txt files coding, because the special characters still do not appear.

It may be a silly question, but I'd appreciate some indications on what's going wrong  ???

Thanks in advance.

You can get a tool that I wrote for the purpose of making editing sc2 fonts easier here: http://cs.ozerki.net/zap/sc2-rus/, I've described it above. Also before converting I made a script that copied the glyphs with similar shape to the respective Cyrillic codes, you may try this too.

I already translated about half of the starbas.txt file and it works okay (Cyrillic characters have codes 192-255). The only problem I've encountered is that unpacked files (the whole share/uqm/content directory) has precedence over addons, so you have either to use a zipped main content (for some reason uqm-content-0.3-0.fdr.1.1.noarch.rpm didn't worked for me :( ) or make a quick hack like me - in options.c line 313 change uio_MOUNT_BELOW to uio_MOUNT_ABOVE, and recompile. This will force addon files have priority over content files.

Thanks to Andrew for the help and the app  :) I'll compile it soon and try to make the necessary special spanish chars with it.

My main problem stayed, though. The translated text appears in the game, but non-standard characters don't, even with the extra PNG files properly named (after each ASCII code)  and placed in the ".fon" folder.

However, I think I've found a tricky workaround to solve it. There are a few regular characters that the game doesn't use - at least a search through all the comm files didn't find any of them in the game text (",#,+,=,[,\ and ]). So I will replace those with the ones I need. Thus, 34.png (by default ") will depict an 'ñ' character - so should I write for instance in my translation 'España', I'd write 'Espa"a' instead. After the proper and readable translation, a 'Replace All' will do the job, just keeping an eye in system text - to make sure it doesn't get corrupted.

It might not be the right solution for western european language packs (if the core team has intended to do so anytime), but it works for a spanish version, and doesn't mess with the english character set. At last I can go ahead with translations without feeling them useless.

If any of you people find it a crazy idea, please wake me up to my senses before it's too late  ::)

That's a ugly hack, I think.

You can try getting the russian pack from cvs here:

cvs -d :pserver:anoncvs@cs.ozerki.net:/mnt/stuff/cvs checkout sc2-ru

There are several already translated things that *work*, so if it works for Russian, it *must* work for Spanish as well (I don't think sc engine is biased towards one or other language ;-))

Actually you were right. That was an ugly solution.
Anyway, I solved my problem - which turned to be a stupid one, as expected. I got inspired by your russian translation files, specifically by the cyrillic character transcription in your txt files. Simply I wasn't using the right character codes, thus the font files were misnamed. That's why I didn't get to see the special chars. I was basing on a wrong ASCII codes table. Apparently, the codes for certain characters in ANSI coding are different... I'm not quite sure about it.
Never mind, anyway. I can carry on translating now, without weird workarounds, and without messing with the standard version files.

Thanks a bunch for your help  ;)

Mac OS X users who want to use this add-on have a couple of extra steps required of them, due to the fact that the Mac Launcher still doesn't do --addon

• After running "make", rename the ru.zip to zzz_ru.zip
  
• Ctrl-click on the UQM game icon and choose "Show Package Contents".  Navigate down to "Content:Resources:content:packages"
  
• Copy the zzz_ru.zip file into this folder.

Then launch the game as normal.

I think there should be an "official" webpage about localizations on this site, where anyone can get info and upload a localization pack.
This would get more people to be interested in making a localization for their country; i would probably volunteer for an italian one.
See http://c-evo.org for example, it's organized very well...

Ehm... sorry, the link doesn't work... how do i make links on this damn forum?  ;D
However the site was http://c-evo.org

I made a small change to sc2 sourcecode in order to support locales.
The problem is that in some places (like MakeReport()) functions from ctype.h (like isalpha() and isgraph()) are used, which won't work for anything but ASCII codes if the locale is not set.

The solution is to place a optional language.txt file in the root content directory (provided by the language addon) which contains the language defines (currently it contains just the locale). This way:

#(Locale)
ru_RU.CP1251

Can any of the developers take a look at the patch, and if it is okay, place in the CVS?
The diff can be downloaded from http://cs.ozerki.net/zap/sc2-rus/language.diff

BTW, just as a side-note:
The german localization team homepage is at
http://sc2de.berlios.de
The last major change was upgrading the branched sourcecode from 0.2 to the current sourceforge cvs-version.
The texts (including landertexts)  have been translated already, but the release still needs some testing, and correcting, especially since a few quirks following the upgrade to 0.3 need some attention by the "code-adjusters".

This message is only here to prevent the start of a second german localization project. One should be enough. Official announcements will follow "when it is ready".

Enjoy,
Krulle

The german translation is complete now.

@UQM-Team
Do you want to integrate it in to UQM ?

Ah, nice.
I suggest you add a new bug report for this, and we'll look at it.
We will probably want the languages as separate packages, which currently isn't possible with all the hard-coded strings, so integration may take a while. But I guess we could just link to your project page from our website for now.

Sounds great  :D

... i add information about where you must do changes to support
chars over 136 from the fonts.

hmm strange, I only had to make one minor change in order to support characters with codes up to 255.

The other Problem is "isprint (ch)" in the files gameopt.c and melee.c

But i see, you working already on the integration of "setlocal".  So  i
dont't post it again...

The third thing is the lander report...  but i search for a solution at the moment.

The lander report is also fixed by setlocale(). According to Meep-Eep, the fix is already in repository, so better get a fresh copy of the sources from svn...

GDP_Spaceman: Where would I download the necessary files?

The project is here:
http://developer.berlios.de/projects/sc2de

The CVS tree is here:
http://cvs.berlios.de/cgi-bin/viewcvs.cgi/sc2de/Sternkontrolle/

Which version of UQM does this correspond to?

Also, I noticed this in your README:

It's version 0.33, but the last update is two weeks ago...
And there are changes in a couple of files, so it's better you look only
for the text in the source files, the text files (race, lander, etc.), the
menu-png files (there are some png with text) and the font-png files.

But if you give me the structure for the language package or how it must
looks like, then i build the german package for you.

I've just committed code for unicode support (output is almost complete; still have to start on input).
The game will now interpret all strings as UTF-8. The file names of the font characters should be the (decimal) number of the character in the unicode table (see http://www.unicode.org/charts/.
For German this is the same as latin-1, so no change is needed there. All the .txt files need to be converted though. On unix this can be done with this command (run in the content dir):


As for how I'd like it to be packaged:
I'd like to have two separate .zip files; one for the fonts (which can be shared with other translations), and one for the other content files.
The .zip files should contain only the files that differ from the main branch and have the same structure as the normal content dir (empty directories may be omited), with at the root the files and directories like comm, lbm, etc (so no directory content/ itself).
If you want to replace .ogg files, don't compress them in the .zip file. With unix zip you can do this by supplying '-n .ogg'.
You can then use them as any other addon (http://uqm.stack.nl/wiki/index.php?title=The_Ur-Quan_Masters_Technical_FAQ#How_do_I_use_add-on_packages.3F) (make sure to give them a file higher in the alphabet than uqm-0.3-content.zip if you put them in packages).
If you package them like this, I'll add them (unzipped) to the cvs repository, in separate directories.

For the strings that are hardcoded in the game, I can unfortunately do nothing at this time. Eventually almost all will be moved to the content (the exception would be strings that need to be available when no content language can be loaded).

That's good but how we will deal with UTF-8 strings in MakeReport (sc2code/planets/report.c). I mean the isgraph() call. It just won't work for UTF-8 characters, and even doing setlocale("ru_RU.UTF-8") (or whatever other language) won't help, because isgraph gets just one character.

Maybe the code could be modified to not depend on the locale in general. In this case the setlocale() code could be removed from InitLanguage() (or just use setlocale (LC_ALL, "") instead of InitLocale().

I haven't got the time at the moment to fix this, but I'll probably replace this by something like

There are a few other places that need still need some attention too.

As for the locale: Setting the locale isn't needed for iswgraph() to work correctly. The only effect on UQM of setting the locale, for as far I can think of, would be that system error messages (the output of strerror()) would be in the correct language. But there's something to be said for that if people want that, they can set the locale themselves.
We could also use the already set locale to determine the default language to use.

I don't see much sense to make sc2 dependent on a completely new layer here (e.g. of the unicode API). This may harm portability for no apparent reason.

The only reason for using isgraph() there seems to skip spaces, tabs and newline chars. So I'd  propose something alot simpler:

while (*pStr <= ' ')
 pStr++;

This won't affect UTF-8 characters because they are always >= 128.

And about setlocale - yes, I think doing setlocale (LC_ALL, "") would be the right thing to do now. Setting locale to anything different from the current locale (as it is done right now -- reading the locale from locale.txt) may cause wrong strings coming out of strerror etc. If you remember, initialy my patch used LC_CTYPE rather than LC_ALL :)

A start to introducing unicode to UQM was already in the code by the time it was handed off to us (which was disabled, and used wide chars everywhere, while I chose to use UTF-8 so that all the standard string functions could be used). As a consequence I never considered making unicode support optional. Perhaps I should have. I may still do it one day.
But it's pretty portable as it is. You only need wchar_t ('typedef unsigned short wchar_t' will do), and now then perhaps iswgraph() ('static inline int iswgraph(wint_t ch) { return ch > 0x7f || isgraph((int) ch); }' ought to do the trick) if you don't have wchar and iswgraph() natively (I'll add checks).