Internationalisation
********************

Internationalisation in pspp is complicated. The most annoying aspect is that of character-encoding. Currently, pspp does not fully deal with the issues. This chapter attempts to describe the problems and current ways in which they are addressed.

The working locales
===================

Pspp has three ``working'' locales:

  * The local of the user interface.

  * The local of the output.

  * The local of the data.

Each of these locales may, at different times take separate (or identical) values. So for example, a French statistician can use pspp to prepare a report in the English language, using a datafile which has been created by a Japanese researcher hence uses a Japanese character set.

It's rarely, if ever, necessary to interrogate the system to find out the values of the 3 locales. However it's important to be aware of the source (destination) locale when reading (writing) string data. When transfering data between a source and a destination, the appropriate recoding must be performed.

The user interface locale
-------------------------

This is the locale which is visible to the person using pspp. Error messages and confidence indications  are written in this locale. For example ``Cannot open file'' will be written in the user interface locale.

This locale is set from the environment of the user who starts pspp{ire} or from the system locale if not set.

The output locale
-----------------

This locale is the one that should be visible to the person reading a report generated by pspp.  Non-data related strings (Eg: ``Page number'', ``Standard Deviation'' etc.) will appear in this locale.

The data locale
---------------

This locale is the one associated with the data being analysed with pspp. The only important aspect of this locale is the character encoding.
It might also be desirable for the LC_COLLATE category to be used for the purposes of sorting data.
Any string data stored in a value will be encoded in the character set of the data locale.

The data locale defaults to the locale of the user who starts pspp{ire}. Spss has a SET LOCALE command (not currently supported in pspp) which can be used to specify the character encoding of the data locale.

System files
============

*.sav files contain a field which is supposed to identify the encoding of the data they contain (see Machine Integer Info Record). This field is currently unused by Pspp. Probably, would be appropriate to set the data locale from this field when reading a new data file, and set it back to the default value upon a NEW FILE command. However, many files produced by early versions of spss set this to ``2'' (ASCII) regardless of the encoding of the data.

GUI
===

The psppire graphic user interface is written using the Gtk+ api, for which all strings must be encoded in UTF8. All strings passed to the Gtk+/Glib library functions must be UTF-8 encoded otherwise errors will occur. Thus, for the purposes of the programming psppire, the user interface locale should be assumed to be UTF8, even if setlocale and/or nl_langinfo indicates otherwise.

Existing locale handling functions
==================================

The major aspect of locale handling which the programmer has to consider is that of character encoding.

The following function is used to recode strings:

 -- Function: char * recode_string (enum conv_id HOW, const char *TEXT, int LEN);
     Converts the string TEXT to a new encoding according to HOW. HOW can (currently) take the values CONV_PSPP_TO_UTF8, CONV_SYSTEM_TO_PSPP or CONV_UTF8_TO_PSPP  (The label ``_PSPP'' ought to be changed to ``_DATA''). If LEN is not -1, then it must be the number of bytes in TEXT. It is the caller's responsibility to free the returned string when no longer required.

For example, in order to display a string variable's value in a label widget in the psppire gui one would use code similar to

     struct variable *var = /* assigned from somewhere */
     struct case c = /* from somewhere else */

     const union value *val = case_data (&c, var);

     char *utf8string = recode_string (CONV_PSPP_TO_UTF8, val->s,
           var_get_width (var));

     GtkWidget *entry = gtk_entry_new();
     gtk_entry_set_text (entry, utf8string);
     gtk_widget_show (entry);

     free (utf8string);

Quirks
======

For historical reasons, not all locale handling follows posix conventions. This makes it difficult (impossible?) to elegantly handle the issues. For example, it would make sense for the gui's datasheet to display numbers formatted according to the LC_NUMERIC category of the data locale. Instead however there is the data_out function (see Obtaining Properties of Format Types) which uses the settings_get_decimal_char function instead of the decimal separator of the locale.  Similarly, formatting of monetary values is displayed in a pspp/spss specific fashion instead of using the LC_MONETARY category.