Download
You can download the testing package with gettext support, some directories have been removed in order to reduce the file size.It should be installed like the normal Wikka, the only additional option in the setup process is the language selection. There are 2 languages available right now: Catalan and English. You can try to install it in catalan to see that the final result is a Wikka site, but in catalan (there are some missing strings).
wikka-1.1.5.3-gettext.tar.gz (140k) Updated on October 6, 2004
Tasks
- gettext support -- done
- different domains (wikka and wikka-setup) -- done
- locale selection from the installer -- done
- script to update po files -- done
- make strings translatable -- 142 strings so far
- wikka.pot: 101 strings
- wikka-setup.pot: 41 strings
Known problems
Hardcoded WikiPagesactions/textsearch.php:22 TextSearchExpand actions/footer.php:3 TextSearch actions/usersettings.php:118 MyPages actions/usersettings.php:119 MyChanges actions/category.php:8 Category Category
If we want to let users choose their preferred language from usersettings, we can't translate hardcoded WikiPages. These are always the same, since they are on the database and were translated in the setup process. Possible solutions:
- Do not translate page tags.
- Add these hardcoded pages to the config file with their respective name, so that we can use $config["MyPages"] instead of MyPages.
Php Gettext, a class that can read directly mo files
At http://savannah.nongnu.org/projects/php-gettext/ , you can find a class consisting of 2 files that can read mo files even if you don't have gettext support installed. It doesn't require anything other than PHP, and its size is absolutely small. I (-- DotMG --) was not a fan of Gettext until I found this class. It is GPL-licensed.FAQ
How to make strings translatable with gettext?It is simple, just use the _() or gettext() function.
Example (simple):
<?php
print(_("Hello Wikka!"));
?>
print(_("Hello Wikka!"));
?>
Example (formatted):
<?php
$here = $this->Format('[[PasswordForgotten | '._("here").']]');
printf(_("If you need a temporary password, click %s."), $here);
?>
$here = $this->Format('[[PasswordForgotten | '._("here").']]');
printf(_("If you need a temporary password, click %s."), $here);
?>
What do we gain with gettext?
- automatic detection of outdated translation strings
- standard framework so that existing translation teams can work on wikka at no cost
I've downloaded and installed your distribution, but it speaks... plain english, even if the locale is set to Ca_ES. Am I missing something?
BTW it would be nice to have the user be able to choose his/her preferred language from usersettings.
It should be ca_ES, not Ca_ES.
Can you try to add the following line in wikka.php, under "set gettext configuration":
putenv("LANG=ca_ES");
Or perhaps using "./locale" instead of "locale" ("locale_path" value, wikka.config.php).
If this does not work, can you tell me something about your configuration? Operating System?
My question is: will this gettext version be integrated in the standard wikka version?
I have already mailed Jason... no response yet.
1) What is the appropriate charset for string-translation files?
2) How is your approach dealing (if it does) with translated strings encoded in different charsets? If one day Wikka is internationalized so as to accept content in different charsets, it would be nice to have l10n strings in the same charsets without needing to reinvent l10n from scratch. This might require being able to detect if a file is using ISO-8859-1, ISO-8859-8 etc. Or maybe translated strings should be saved as unicode?
But I think migrating to UTF8 would be the best way to handle a proper internationalization. It is also the easiest way: iconv .php files and change the charset (though that would probably require to drop the support for MySQL 3).
And what about a wikka used in an intranet on windows machines not connected to internet?
AFAIK, gettext supports almost any language, see KDE i18n statistics, there are many minority languages.
http://i18n.kde.org/stats/gui/HEAD/index.php
Oh, and catalan (my main language) is only spoken by 10 million people ;)
There should be a way in the program to get around that limitation in MySQL 3.x - for instance by encoding the text. URL-encode would work (I'm using this in a site of my own); you do the encoding then only depending on MySQL version.
Sorry, I'm getting confused now.
"But we're speaking about encoding wikka in UTF-8, not the content." - What's "wikka" and what's "content" in that statement? For me, what I'm looking at in my browser (or listening to, as the case may be) is the "content" (while the HTML head section is not content, and this not rendered). (IOW, the text on that button that reeads "Delete Comment" is content - no matter where it comes from.)
Anyhow:
- I wholeheartedly support using UTF-8 (Unicode is the only solution for multi-language pages)
- but *only* if this (in Wikka) does *not* require MySQL 4+ (or the latest-and-greatest version of PHP for that matter - we already have dependencies now that shouldn't be there IMO).
1) the content, what the user wrote, which goes to the database, and can be encoded in ASCII + html entities (this covers the UTF8 charset, right?) which does not require more than MySQL 3.0
2) the wikka text, aka text of the buttons, header, footer... which comes from the gettext files (actually the php files) and which can be encoded in UTF8 with no influence on the "content" (1)
About the other points, like you I'm for the UTF8 solution (because I often use characters which are not in latin-1).
JordaPolo, I think we should go for UTF8 now, because it's the only we to do I18N. Uniwakka could be a good source of inspiration. I had no time to check their current state, but they claim that they support UTF8 (I think...).
Thanks, that's a little clearer now - something along the lines of "user-written content" vs. "system-generated user interface" coming from PHP programs.
I see a number of gray areas here though:
- pages that are "created" on system installation: they have content like "user-written content", but form a part of the user interface of the system as a whole
- actions used in "user-written content" that may have parameters (is the parameter content translatable?)
Looking at the archive I downloaded I see that the "key" texts can have variables - that's good. But there are other issues with i18n, for instance:
- homonyms (English "key" means "sleutel" or "toets" in Dutch, depending on context/semantics; or verb forms that are teh sma ein English for infinitive and command, but ddifferent in other languages)
- synomyms
- capitalization rules that are different for various languages
- punctuation rules that are different for various languages (for instance, if you have prompts in a form that in English would end with a colon, the colon should be part of the translatable text, since English would place the colon immediately after the text, French would precede it with a space, and some languages might not normally use a colon at all)
How are such issues handled? Homonyms and capitalization can be major issues; punctuation will be unless punctuation is taken as part of translatable content from the start, and consistently. Consistency of wording used in different functions is also a major concern (especially in combination with the homonyms problem).
Finally, can a user choose (configure) a run-time language?
If so, obviously, this should apply to the "system-generated user interface" (which should not be a problem) as well as the *content* of system-generated pages. The latter would be a problem unless page "content" can also make use of translatable strings. Can it?
I would prefer that there would be a user-configurable run-time language; but this would ideally require an end user interface to translatable strings, or (at least) creation of the system-generated pages in all available langauges and easy extensibility when new languages become available.
Platform support: I note two scripts, translate.sh and translate-setup.sh; what about Windows support?
I'm all for i18n, but having been intimately involved with localization of a (commercial) software application, I would caution against being too hasty in implementing a particular solution. Let's look carefully at _all_ issues and tick them off as either "taken care of", "can be done later" (how?) or "not supported".
It's important. But it's *not* easy, however easy a tool may make it seem.
Pages that are "created" on system installation are already translatable. We have 2 domains: wikka and wikka-setup. The first one contains user interface, errors... all the strings that appear on actions/*.php, handlers/*.php... The second one, wikka-setup, contains the pages that are created on system installation.
You can change the language, but only the strings on wikka domain will be changed since wikka-setup strings are already stored in the database.
"You can change the language, but only the strings on wikka domain will be changed since wikka-setup strings are already stored in the database."
The system pages that are created on setup are, as I indicated, actually part of the system's user interface. So you'll have a user interface that's only partly in the user's language? In fact, in only one single language, instead of as many languages as are available?
Not good.
There should be an mechanism then to include *dynamically* translatable text into pages' content. Perhaps a special formatter action.
The idea behind WikkaGettext is to create internationalized versions of Wikka, not a multilingual version. Yes, it is possible to partially change the user's language, but I'm not interested in that. 1 wikka -> 1 language, but you can choose it.
Actually, the idea behind (GNU) *gettext* is internationalization; it seems what you're doing with WikkaGettext is not actually internationalization (which means supporting multiple languages and locales), but *localization*.
The idea is nice, but while we're at it, why not internationalize it?
Apart from that, all of the other issues I've raised haven't been addressed yet...
And then there is the fact that you need a version of PHP that actually supports gettext - something that may not be the case for all hosted versions of PHP.
About the other issues:
- punctuation: is taken as part of translatable content
- capitalization: I don't see the problem here, translators can choose how to capitalize every string...
- homonyms, synonims: it could be possible to use some kind of contextualization on problematic strings, for example: context|string
- scripts: only needed by translator coordinator, translators use tools like poedit, gtranslator, kbabel, vim...
- php gettext: it is possible to create our own _() function if gettext is not installed (as it is done in phpwiki)
I have been doing both internationalization (adding support for multiple languages) and localization (operation by which an internationalized program is adapted) to catalan. The only problem (as you see it) is that you can't change the language on the fly. And that could be fixed but...
Well, if you want to change the language of wikka on the fly, then I think gettext is not the way to go. Why? It would require the server to have all the locales, and that could be a real problem on some servers.
1) I'm all in favor of enabling Wikka for l10n and/or i18n. And enabling UTF-8 as well.
2) I'm definitely opposed to doing it *now*.
Not to detract from JordaPolo's work in creating this demo (which nicely shows that it /can/ work - thanks for all the work involved!) but in my opnion the code base of Wikka (as it is now) is simply not ready for proper i18n (and this demo actually illustrates that, if you look closely enough). The problem is not with JordaPolo's work, but more with the lack of coding standards that should _enable_ i18n.
The issues I've raised in my comments on this page actually only scratch the surface. I'll start a separate page to present my ideas more clearly than I can do in comments here - but have a little patience, I'm still working on my email stuff and that takes an amount of concentration. ;-)
In short: yes, let's do it. But PLEASE let's not do it *now*.
- scripts: only needed by translator coordinator, translators use tools like poedit, gtranslator, kbabel, vim...
- php gettext: it is possible to create our own _() function if gettext is not installed (as it is done in phpwiki)
<<
If that means gettext can be used in non-Unix environment, I'm OK. Actually, I have this error message on Windows platform : "Fatal error: Call to undefined function: bindtextdomain() in path_to\wikka.php on line 910"
Creating our own _() function is a good idea, as I will use Wikka on an Intranet machines running under Windows. We (I) also need instructions on how to perform the translator's tasks.
I absolutely agree with JavaWoman : yes, let's do it. But PLEASE let's not do it *now*
Try a Google with ["Call to undefined function: bindtextdomain()"] (where [] indicates the textbox); it's a common problem. Put simply, you need to 1) install the windows version of gettext (runtime _and_ tools) which you can get from the GNU ftp site; and 2) enable it for PHP (php.ini).
If you're running on an Intranet that should not be a problem - for hosted sites it might be.
At the very least Wikka with gettext support should come with a clear manual on how to enable it on various platforms, as well as what to do when it isn't (can't be) supported. There are many more gotchas than just this "little" one as I found in my researches last weekend; the documentation at http://php.net/manual/en/ref.gettext.php (and all the functions!) - will give you a taste - in particular the comments on those pages; and that's for starters.
Again, depending on what kind of I18N do you want to have on Wikka, gettext might be (or not) a good solution.
- it requires php to be compiled with a non-default option,
- AFAIK it is far from being simple to setup.
Well, I should not rise the first objection, since in UniWakka, the wakka fork I'm developing, I'm using mbstring to handle utf-8 (there is a test version that does not require it, but I did not have time to test in depth and so I did not include it in the released version - check it out in http://wikka.jsnx.com/HandlingUTF8).
The second objection could be due to the fact that I'm not familiar with gettex...;-)
Anyway, the approach I would like to take is the one adopted by CooCooWakka:
http://www.hsfz.net.cn/coo/wiki/HomePage
Get the source and have a look. They support chinese and english (They pretend to support utf-8 but I did not find any bit related to it and did not test it).
I18n without proper utf-8 support is quite useless, IMHO. I18n makes sense as far as the application can support a wide variety of languages, and multi-language too. With the CooCooWakka approach every user can choose the UI locale. With utf-8 support you do not need to take into account charset definition and stupid user will not mess everything up. Have a look here to understand what I mean: http://wiki.cs.cityu.edu.hk/CitiWiki/Comment
CitiWiki defined utf-8 as the default charset, but wakka (CitiWiki is a wakka fork too) does not support it! If you let user define the charset this kind of issues will show up, that's for sure. Users will blame the developer. And they are right, even if stupid...;-)
Now, I've tried to demonstrate that real multi-language support without having to drop mysql 3.x.x support is possible. AFAIK DarTar ported my API to wikka, and he said it is working just fine. I refined it so that the memory consumption is not so burdensome, and, without requiring mbstring php support, it can be easily adopted. Further improvements are always possible.
As usual, my 2 cents.
That's a trade off and I decided that multi-language support with mysql 3.x.x was worth the price. In the far east there are many guys involved in wiki technologies. They need multi-byte support. Even though mysql and php support for multi-byte sucks. Letting them use UniWakka was one of my goal.
When mysql-4.1.x will be common enough this approach can be changed.
Computing in Zulu -
http://itre.cis.upenn.edu/~myl/languagelog/archives/001596.html