Andrey A. Chernov Contributed by Michael C. Wu Rewritten by FreeBSD is a very distributed project with users and contributors located all over the world. This chapter discusses the internationalization and localization features of FreeBSD that allow non-English speaking users to get real work done. There are many aspects of the i18n implementation in both the system and application levels, so where applicable we refer the reader to more specific sources of documentation. After reading this chapter, you will know: How different languages and locales are encoded on modern operating systems. How to set the locale for your login shell. How to configure your console for non-English languages. How to use X Windows effectively with different languages. Where to find more information about writing i18n-compliant applications. Before reading this chapter, you should: Know how to install additional third-party applications (). internationalization localization Developers shortened internationalization into the term I18N, counting the number of letters between the first and the last letters of internationalization. L10N uses the same naming scheme, coming from "localization". Combined together, I18N/L10N methods, protocols, and applications allow users to use languages of their choice. I18N applications are programmed using I18N kits under libraries. It allows for developers to write a simple file and translate displayed menus and texts to each language. We strongly encourage programmers to follow this convention. I18N/L10N is used whenever you wish to either view, input, or process data in non-English languages. I18N and L10N are not FreeBSD specific. Currently, one can choose from most of the major languages of the World, including but not limited to: Chinese, German, Japanese, Korean, French, Russian, Vietnamese and others. In all its splendor, I18N is not FreeBSD-specific and is a convention. We encourage you to help FreeBSD in following this convention. locale Localization settings are based on three main terms: Language Code, Country Code, and Encoding. Locale names are constructed from these parts as follows: LanguageCode_CountryCode.Encoding language codes country codes In order to localize a FreeBSD system to a specific language (or any other I18N-supporting Unixes), the user needs to find out the codes for the specify country and language (country codes tell applications what variation of given language to use). In addition, web browsers, SMTP/POP servers, web servers, etc. make decisions based on them. The following are examples of language/country codes: Language/Country Code Description en_US English - United States ru_RU Russian for Russia zh_TW Traditional Chinese for Taiwan encodings ASCII Some languages use non-ASCII encodings that are 8-bit, wide or multibyte characters, see &man.multibyte.3; for more details. Older applications do not recognize them and mistake them for control characters. Newer applications usually do recognize 8-bit characters. Depending on the implementation, users may be required to compile an application with wide or multibyte characters support, or configure it correctly. To be able to input and process wide or multibyte characters, the FreeBSD Ports collection has provided each language with different programs. Refer to the I18N documentation in the respective FreeBSD Port. Specifically, the user needs to look at the application documentation to decide on how to configure it correctly or to pass correct values into the configure/Makefile/compiler. Some things to keep in mind are: Language specific single C chars character sets (see &man.multibyte.3;), i.e., ISO-8859-1, ISO-8859-15, KOI8-R, CP437. Wide or multibyte encodings, f.e. EUC, Big5. You can check the active list of character sets at the IANA Registry. FreeBSD versions 4.5 and up use X11-compatible locale encodings instead. In the FreeBSD Ports and Package system, I18N applications have been named with I18N in their names for easy identification. However, they do not always support the language needed. Theoretically, one only needs to export the value of his/her locale name as LANG in the login shell and is usually done through the user's ~/.login_conf or the user login shell configuration (~/.profile, ~/.bashrc, ~/.cshrc). This should set all of the locale subsets (such as LC_CTYPE, LC_CTIME, etc.). Please refer to language-specific FreeBSD documentation for more information. You should set the following two values in your configuration files: POSIX LANG for POSIX &man.setlocale.3; family functions MIME MM_CHARSET for applications' MIME character set This includes the user shell config, the specific application config, and the X11 config. locale login class There are two methods for setting locale, and both are described below. The first (recommended one) is by assigning the environment variables in login class, and the second is by adding the environment variable assignments to the system's shell startup file. This method allows environment variables needed for locale name and MIME character sets to be assigned once for every possible shell instead of adding specific shell assignments to each shell's startup file. User Level Setup can be done by an user himself and Administrator Level Setup require superuser privileges. Here is a minimal example of a .login_conf file in user's home directory which has both variables set for Latin-1 encoding: german:German User:\ :charset=ISO-8859-1:\ :lang=de_DE.ISO_8859-1: de_DE.ISO8859-1 for FreeBSD versions 4.5 and up. Traditional Chinese / BIG-5 encoding Here is an example of a .login_conf that sets the variables for Traditional Chinese in BIG-5 encoding. Notice the many more variables set because some software does not respect locale variables correctly for Chinese, Japanese, and Korean. #Users who do not wish to use monetary units or time formats #of Taiwan can manually change each variable taiwan:Taiwanese User:\ lang=zh_TW.Big5:\ lc_all=zh_TW.Big:\ lc_collate=zh_TW.Big5:\ lc_ctype=zh_TW.Big5:\ lc_messages=zh_TW.Big5:\ lc_monetary=zh_TW.Big5:\ lc_numeric=zh_TW.Big5:\ lc_time=zh_TW.Big5:\ charset=big5:\ xmodifiers="@im=xcin": #Setting the XIM Input Server See Administrator Level Setup and &man.login.conf.5; for more details. Check that /etc/login.conf have the correct language user's class. Make sure these settings appear in /etc/login.conf: language_name:accounts_title:\ :charset=MIME_charset:\ :lang=locale_name:\ :tc=default: So sticking with our previous example using Latin-1, it would look like this: german:German Users Accounts:\ :charset=ISO-8859-1:\ :lang=de_DE.ISO_8859-1:\ :tc=default: de_DE.ISO8859-1 for FreeBSD versions 4.5 and up. Changing Login Classes with &man.vipw.8; vipw Use vipw to add new users, and make the entry look like this: user:password:1111:11:language:0:0:User Name:/home/user:/bin/sh Changing Login Classes with &man.adduser.8; adduser login class Use adduser to add new users, and do the following: Set defaultclass = language in /etc/adduser.conf. Keep in mind you must enter a default class for all users of other languages in this case. An alternative variant is answering the specified language each time that Enter login class: default []: appears from &man.adduser.8; Another alternative is to use the following for each user of a different language that you wish to add: &prompt.root; adduser -class language Changing Login Classes with &man.pw.8; pw If you use &man.pw.8; for adding new users, call it in this form: &prompt.root; pw useradd user_name -L language This method is not recommended because it requires a different setup for each possible login program chosen. Use the Login Class Method instead. MIME locale To add the locale name and MIME character set, just set the two environment variables shown below in the /etc/profile and/or /etc/csh.login shell startup files. We will use the German language as an example below: In /etc/profile: LANG=de_DE.ISO_8859-1; export LANG MM_CHARSET=ISO-8859-1; export MM_CHARSET Or in /etc/csh.login: setenv LANG de_DE.ISO_8859-1 setenv MM_CHARSET ISO-8859-1 de_DE.ISO8859-1 for FreeBSD versions 4.5 and up. Alternatively, you can add the above instructions to /usr/share/skel/dot.profile (similar to what was used in /etc/profile above), or /usr/share/skel/dot.login (similar to what was used in /etc/csh.login above). For X11: In $HOME/.xinitrc: LANG=de_DE.ISO_8859-1; export LANG Or: setenv LANG de_DE.ISO_8859-1 Depending on your shell (see above). de_DE.ISO8859-1 for FreeBSD versions 4.5 and up. For all single C chars character sets, set the correct console fonts in /etc/rc.conf for the language in question with: font8x16=font_name font8x14=font_name font8x8=font_name The font_name here is taken from the /usr/share/syscons/fonts directory, without the .fnt suffix. sysinstall keymap screenmap Also be sure to set the correct keymap and screenmap for your single C chars character set through /stand/sysinstall. Once inside sysinstall, choose Configure, then Console. Alternatively, you can add the following to /etc/rc.conf: scrnmap=screenmap_name keymap=keymap_name keychange="fkey_number sequence" The screenmap_name here is taken from the /usr/share/syscons/scrnmaps directory, without the .scm suffix. A screenmap with a corresponding mapped font is usually needed as a workaround for expanding bit 8 to bit 9 on a VGA adapter's font character matrix in pseudographics area, i.e., to move letters out of that area if screen font uses a bit 8 column. If you have the moused daemon enabled by setting the following in your /etc/rc.conf: moused_enable="YES" then examine the mouse cursor information in the next paragraph. moused By default the mouse cursor of the syscons driver occupies the 0xd0-0xd3 range in the character set. If your language uses this range, you need to move the cursor's range outside of it. To enable the workaround for FreeBSD versions before 5.0, insert the following line into your kernel config: options SC_MOUSE_CHAR=0x03 For the FreeBSD versions 4.4 and up insert the following line into /etc/rc.conf: mousechar_start=3 The keymap_name here is taken from the /usr/share/syscons/keymaps directory, without the .kbd suffix. If you're uncertain which keymap to use, you use &man.kbdmap.1 to test keymaps without rebooting. The keychange is usually needed to program function keys to match the selected terminal type because function key sequences cannot be defined in the key map. Also be sure to set the correct console terminal type in /etc/ttys for all ttyv* entries. Current pre-defined correspondences are: Character Set Terminal Type ISO-8859-1 or ISO-8859-15 cons25l1 ISO-8859-2 cons25l2 ISO-8859-7 cons25l7 KOI8-R cons25r KOI8-U cons25u CP437 (VGA default) cons25 US-ASCII cons25w For wide or multibyte characters languages, use the correct FreeBSD port in your /usr/ports/language directory. Some ports appear as console while the system sees it as serial vtty's, hence you must reserve enough vtty's for both X11 and the pseudo-serial console. Here is a partial list of applications for using other languages in console: Language Location Traditional Chinese (BIG-5) chinese/big5con Japanese japanese/ja-kon2-* or japanese/Mule_Wnn Korean korean/ko-han Although X11 is not part of the FreeBSD Project, we have included some information here for FreeBSD users. For more details, refer to the XFree86 web site or whichever X11 Server you use. In ~/.Xresources, you can additionally tune application specific I18N settings (e.g., fonts, menus, etc.). X11 True Type font server Install the X11 True Type-Common server (XTT-common) and install the language truetype fonts. Setting the correct locale should allow you to view your selected language in menus and such. X11 Input Method (XIM) The X11 Input Method (XIM) Protocol is a new standard for all X11 clients. All X11 applications should be written as XIM clients that take input from XIM Input servers. There are several XIM servers available for different languages. Some single C chars character sets are usually hardware coded into printers. Wide or multibyte character sets require special setup and we recommend using apsfilter. You may also convert the document to PostScript or PDF formats using language specific converters. The FreeBSD FFS filesystem is 8-bit clean, so it can be used with any single C chars character set (see &man.multibyte.3;), but there is no character set name stored in the filesystem; i.e., it is raw 8-bit and does not know anything about encoding order. Officially, FFS does not support any form of wide or multibyte character sets yet. However, some wide or multibyte character sets have independent patches for FFS enabling such support. They are only temporary unportable solutions or hacks and we have decided to not include them in the source tree. Refer to respective languages' web sites for more informations and the patch files. DOS Unicode The FreeBSD MS-DOS filesystem has the configurable ability to convert between MS-DOS, Unicode character sets and chosen FreeBSD filesystem character sets. See &man.mount.msdos.8; for details. Many FreeBSD Ports have been ported with I18N support. Some of them are marked with -I18N in the port name. These and many other programs have built in support for I18N and need no special consideration. MySQL However, some applications such as MySQL need to be have the Makefile configured with the specific charset. This is usually done in the Makefile or done by passing a value to configure in the source. Andrey A. Chernov Originally contributed by Russian localization For more information about KOI8-R encoding, see the KOI8-R References (Russian Net Character Set). Put the following lines into your ~/.login_conf file: me:My Account:\ :charset=KOI8-R:\ :lang=ru_RU.KOI8-R: See earlier in this chapter for examples of setting up the locale. For the FreeBSD versions before 5.0 add the following line to your kernel configuration file: options SC_MOUSE_CHAR=0x03 For the FreeBSD versions 4.4 and up insert the following line into /etc/rc.conf: mousechar_start=3 Use following settings in /etc/rc.conf: keymap="ru.koi8-r" scrnmap="koi8-r2cp866" font8x16="cp866b-8x16" font8x14="cp866-8x14" font8x8="cp866-8x8" For each ttyv* entry in /etc/ttys, use cons25r as the terminal type. See earlier in this chapter for examples of setting up the console. printers Since most printers with Russian characters come with hardware code page CP866, a special output filter is needed for KOI8-R -> CP866 conversion. Such a filter is installed by default as /usr/libexec/lpr/ru/koi2alt. A Russian printer /etc/printcap entry should look like: lp|Russian local line printer:\ :sh:of=/usr/libexec/lpr/ru/koi2alt:\ :lp=/dev/lpt0:sd=/var/spool/output/lpd:lf=/var/log/lpd-errs: See &man.printcap.5; for a detailed description. The following example &man.fstab.5; entry enables support for Russian filenames in mounted MS-DOS filesystems: /dev/ad0s2 /dos/c msdos rw,-Wkoi2dos,-Lru_RU.KOI8-R 0 0 See &man.mount.msdos.8; for a detailed description of the -W and -L options. Do non-X locale setup first as described. The Russian KOI8-R locale may not work with old XFree86 releases (lower than 3.3). The XFree86 port from x11/XFree86 already is the most recent XFree86 version, so it will work if you install XFree86 from the port. This should not be an issue unless you are using an old version of FreeBSD. Go to the russian/X.language directory and issue the following command: &prompt.root; make install The above port installs the latest version of the KOI8-R fonts. XFree86 3.3 already has some KOI8-R fonts, but these are scaled better. Check the "Files" section in your /etc/XF86Config file. The following lines must be added before any other FontPath entries: FontPath "/usr/X11R6/lib/X11/fonts/cyrillic/misc" FontPath "/usr/X11R6/lib/X11/fonts/cyrillic/75dpi" FontPath "/usr/X11R6/lib/X11/fonts/cyrillic/100dpi" If you use a high resolution video mode, swap the 75 dpi and 100 dpi lines. To activate a Russian keyboard, add the following to the "Keyboard" section of your XF86Config file. For XFree86 v3.*: XkbLayout "ru" XkbOptions "grp:caps_toggle" For XFree86 v4.*: Option "XkbLayout" "ru" Option "XkbOptions" "grp:caps_toggle" Also make sure that XkbDisable is turned off (commented out) there. The RUS/LAT switch will be CapsLock. The old CapsLock function is still available via Shift+CapsLock (in LAT mode only). If you have Windows keys on your keyboard, and notice that some non-alphabetical keys are mapped incorrectly in RUS mode, add the following line in your XF86Config file. For XFree86 v3.*: XkbVariant "winkeys" For XFree86 v4.*: Option "XkbVariant" "winkeys" The Russian XKB keyboard may not work with old XFree86 versions, see the above note for more information. The Russian XKB keyboard may also not work with non-localized applications as well. Minimally localized applications should call a XtSetLanguageProc (NULL, NULL, NULL); function early in the program. See KOI8-R for X-Window for more instructions on localizing X11 applications. Traditional Chinese localization The FreeBSD-Taiwan Project has an I18N/L10N tutorial for FreeBSD at http://freebsd.sinica.edu.tw/~ncvs/zh-l10n-tut/ using many chinese/* applications. The editor for the zh-L10N-tut is Clive Lin Clive@CirX.org. You can also cvsup the following collections at freebsd.sinica.edu.tw: Collection Description outta-port tag=. Beta-quality ports collection for Chinese zh-L10N-tut tag=. Localizing FreeBSD Tutorial in BIG-5 Traditional Chinese zh-doc tag=. FreeBSD Documentation Translation to BIG-5 Traditional Chinese Chuan-Hsing Shen s874070@mail.yzu.edu.tw has created the Chinese FreeBSD Collection (CFC) using FreeBSD-Taiwan's zh-L10N-tut. The packages and the script files are available at ftp://ftp.csie.ncu.edu.tw/OS/FreeBSD/taiwan/CFC/. German localization Slaven Rezic eserte@cs.tu-berlin.de wrote a tutorial how to use umlauts on a FreeBSD machine. The tutorial is written in German and available at http://www.de.FreeBSD.org/de/umlaute/. Japanese localization Korean localization For Japanese, refer to http://www.jp.FreeBSD.org/, and for Korean, refer to http://www.kr.FreeBSD.org/. Some FreeBSD contributors have translated parts of FreeBSD to other languages. They are available through links on the main site or in /usr/share/doc.