lexer: Reimplement for better testability and internationalization.

[pspp-builds.git] / doc / utilities.texi

diff --git a/doc/utilities.texi b/doc/utilities.texi
index fabec248b6aed6f894ea3afdd11ddf4243fe22a0..2cf95a3107839634d4b360ea1ce4431b236bf5ea 100644 (file)
--- a/doc/utilities.texi
+++ b/doc/utilities.texi
@@ -10,6 +10,7 @@ encountered in the input.
 
 @menu
 * ADD DOCUMENT::                Add documentary text to the active file.
 
 @menu
 * ADD DOCUMENT::                Add documentary text to the active file.

+* CACHE::                       Ignored for compatibility.

 * CD::                          Change the current directory.
 * COMMENT::                     Document your syntax file.
 * DOCUMENT::                    Document the active file.
 * CD::                          Change the current directory.
 * COMMENT::                     Document your syntax file.
 * DOCUMENT::                    Document the active file.


@@ -25,6 +26,7 @@ encountered in the input.
 * INCLUDE::                     Include a file within the current one.
 * INSERT::                      Insert a file within the current one.
 * PERMISSIONS::                 Change permissions on a file.
 * INCLUDE::                     Include a file within the current one.
 * INSERT::                      Insert a file within the current one.
 * PERMISSIONS::                 Change permissions on a file.

+* PRESERVE and RESTORE::        Saving settings and restoring them later.

 * SET::                         Adjust PSPP runtime parameters.
 * SHOW::                        Display runtime parameters.
 * SUBTITLE::                    Provide a document subtitle.
 * SET::                         Adjust PSPP runtime parameters.
 * SHOW::                        Display runtime parameters.
 * SUBTITLE::                    Provide a document subtitle.


@@ -51,6 +53,16 @@ DOCUMENTS}.
 Each line of documentary text must be enclosed in quotation marks, and 
 may not be more than 80 bytes long. @xref{DOCUMENT}.
 
 Each line of documentary text must be enclosed in quotation marks, and 
 may not be more than 80 bytes long. @xref{DOCUMENT}.
 

+@node CACHE
+@section CACHE
+@vindex CACHE
+
+@display
+CACHE.
+@end display
+
+This command is accepted, for compatibility, but it has no effect.
+

 @node CD
 @section CD
 @vindex CD
 @node CD
 @section CD
 @vindex CD


@@ -211,19 +223,26 @@ control to the operating system.
 
 @display
 HOST.
 
 @display
 HOST.

+HOST COMMAND=['command'...].

 @end display
 
 @cmd{HOST} suspends the current PSPP session and temporarily returns control 
 to the operating system.
 This command cannot be used if the SAFER setting is active.
 
 @end display
 
 @cmd{HOST} suspends the current PSPP session and temporarily returns control 
 to the operating system.
 This command cannot be used if the SAFER setting is active.
 

+If the COMMAND subcommand is specified, as a sequence of shell
+commands as quoted strings within square brackets, then PSPP executes
+them together in a single subshell.
+
+If no subcommands are specified, then PSPP invokes an interactive
+subshell.

 
 @node INCLUDE
 @section INCLUDE
 @vindex INCLUDE
 
 @display
 
 @node INCLUDE
 @section INCLUDE
 @vindex INCLUDE
 
 @display

-        INCLUDE [FILE=]'file-name'.
+        INCLUDE [FILE=]'file-name' [ENCODING='encoding'].

 @end display
 
 @cmd{INCLUDE} causes the PSPP command processor to read an
 @end display
 
 @cmd{INCLUDE} causes the PSPP command processor to read an


@@ -234,19 +253,11 @@ stop and no more commands will be processed.
 Include files may be nested to any depth, up to the limit of available
 memory.
 
 Include files may be nested to any depth, up to the limit of available
 memory.
 

+The @cmd{INSERT} command (@pxref{INSERT}) is a more flexible
+alternative to @cmd{INCLUDE}.  An INCLUDE command acts the same as
+INSERT with ERROR=STOP CD=NO SYNTAX=BATCH specified.

 
 

-The @cmd{INSERT} command (@pxref{INSERT}) may be used instead of
-@cmd{INCLUDE} if you require more flexible options.
-The syntax 
-@example
-INCLUDE FILE=@var{file-name}.
-@end example
-@noindent 
-functions identically to 
-@example
-INSERT FILE=@var{file-name} ERROR=STOP CD=NO SYNTAX=BATCH.
-@end example
-
+The optional ENCODING subcommand has the same meaning as on INSERT.

 
 @node INSERT
 @section INSERT
 
 @node INSERT
 @section INSERT


@@ -256,7 +267,8 @@ INSERT FILE=@var{file-name} ERROR=STOP CD=NO SYNTAX=BATCH.
      INSERT [FILE=]'file-name'
         [CD=@{NO,YES@}]
         [ERROR=@{CONTINUE,STOP@}]
      INSERT [FILE=]'file-name'
         [CD=@{NO,YES@}]
         [ERROR=@{CONTINUE,STOP@}]

-        [SYNTAX=@{BATCH,INTERACTIVE@}].
+        [SYNTAX=@{BATCH,INTERACTIVE@}]
+        [ENCODING='encoding'].

 @end display
 
 @cmd{INSERT} is similar to @cmd{INCLUDE} (@pxref{INCLUDE}) 
 @end display
 
 @cmd{INSERT} is similar to @cmd{INCLUDE} (@pxref{INCLUDE}) 


@@ -284,6 +296,37 @@ the included file must conform to interactive syntax
 conventions. @xref{Syntax Variants}.
 The default setting is @samp{SYNTAX=BATCH}.
 
 conventions. @xref{Syntax Variants}.
 The default setting is @samp{SYNTAX=BATCH}.
 

+ENCODING optionally specifies the character set used by the included
+file.  Its argument, which is not case-sensitive, must be in one of
+the following forms:
+
+@table @asis
+@item @code{Locale}
+The encoding used by the system locale, or as overridden by the SET
+LOCALE command (@pxref{SET}).  On Unix systems, environment variables,
+e.g.@: @env{LANG} or @env{LC_ALL}, determine the system locale.
+
+@item IANA character set name
+One of the character set names listed by IANA at
+@uref{http://www.iana.org/assignments/character-sets}.  Some examples
+are @code{ASCII} (United States), @code{ISO-8859-1} (western Europe),
+@code{EUC-JP} (Japan), and @code{windows-1252} (Windows).  Not all
+systems support all character sets.
+
+@item @code{Auto}
+@item @code{Auto,@var{encoding}}
+Automatically detects whether a syntax file is encoded in
+@var{encoding} or in a Unicode encoding such as UTF-8, UTF-16, or
+UTF-32.  The @var{encoding} may be an IANA character set name or
+@code{Locale} (the default).  Only ASCII compatible encodings can
+automatically be distinguished from UTF-8 (the most common locale
+encodings are all ASCII-compatible).
+@end table
+
+When ENCODING is not specified, the default is taken from the
+@option{--syntax-encoding} command option, if it was specified, and
+otherwise it is @code{Auto}.
+

 @node PERMISSIONS
 @section PERMISSIONS
 @vindex PERMISSIONS
 @node PERMISSIONS
 @section PERMISSIONS
 @vindex PERMISSIONS


@@ -308,6 +351,22 @@ unchanged.
 This command cannot be used if the SAFER setting is active.
 
 
 This command cannot be used if the SAFER setting is active.
 
 

+@node PRESERVE and RESTORE
+@section PRESERVE and RESTORE
+@vindex PRESERVE
+@vindex RESTORE
+
+@display
+PRESERVE.
+@dots{}
+RESTORE.
+@end display
+
+@cmd{PRESERVE} saves all of the settings that @cmd{SET} (@pxref{SET})
+can adjust.  A later @cmd{RESTORE} command restores those settings.
+
+@cmd{PRESERVE} can be nested up to five levels deep.
+

 @node SET
 @section SET
 @vindex SET
 @node SET
 @section SET
 @vindex SET


@@ -323,19 +382,13 @@ SET
         /RIB=@{NATIVE,MSBFIRST,LSBFIRST,VAX@}
         /RRB=@{NATIVE,ISL,ISB,IDL,IDB,VF,VD,VG,ZS,ZL@}
 
         /RIB=@{NATIVE,MSBFIRST,LSBFIRST,VAX@}
         /RRB=@{NATIVE,ISL,ISB,IDL,IDB,VF,VD,VG,ZS,ZL@}
 

-(program input)
-        /ENDCMD='.'
-        /NULLINE=@{ON,OFF@}
-

 (interaction)
 (interaction)

-        /CPROMPT='cprompt_string'
-        /DPROMPT='dprompt_string'

         /MXERRS=max_errs
         /MXWARNS=max_warnings
         /MXERRS=max_errs
         /MXWARNS=max_warnings

-        /PROMPT='prompt'

         /WORKSPACE=workspace_size
 
         /WORKSPACE=workspace_size
 

-(program execution)
+(syntax execution)
+        /LOCALE='locale'

         /MEXPAND=@{ON,OFF@}
         /MITERATE=max_iterations
         /MNEST=max_nest
         /MEXPAND=@{ON,OFF@}
         /MITERATE=max_iterations
         /MNEST=max_nest


@@ -495,46 +548,37 @@ formats are only for use with very old input files.
 The default is NATIVE.
 @end table
 
 The default is NATIVE.
 @end table
 

-Program input subcommands affect the way that programs are parsed when
-they are typed interactively or run from a command file.  They are
-
-@table @asis
-@item ENDCMD
-This is a single character indicating the end of a command.  The default
-is @samp{.}.  Don't change this.
-
-@item NULLINE
-Whether a blank line is interpreted as ending the current command.  The
-default is ON.
-@end table
-

 Interaction subcommands affect the way that PSPP interacts with an
 online user.  The interaction subcommands are
 
 @table @asis
 Interaction subcommands affect the way that PSPP interacts with an
 online user.  The interaction subcommands are
 
 @table @asis

-@item CPROMPT
-The command continuation prompt.  The default is @samp{    > }.
-
-@item DPROMPT
-Prompt used when expecting data input within @cmd{BEGIN DATA} (@pxref{BEGIN
-DATA}).  The default is @samp{data> }.
-

 @item MXERRS
 The maximum number of errors before PSPP halts processing of the current
 command file.  The default is 50.
 
 @item MXWARNS
 The maximum number of warnings + errors before PSPP halts processing the
 @item MXERRS
 The maximum number of errors before PSPP halts processing of the current
 command file.  The default is 50.
 
 @item MXWARNS
 The maximum number of warnings + errors before PSPP halts processing the

-current command file.  The default is 100.
-
-@item PROMPT
-The command prompt.  The default is @samp{PSPP> }.
+current command file.  
+The special value of zero means that all warning situations should be ignored.
+No warnings will be issued, except a single initial warning advising the user
+that warnings will not be given.
+The default value is 100.

 @end table
 
 @end table
 

-Program execution subcommands control the way that PSPP commands
-execute.  The program execution subcommands are
+Syntax execution subcommands control the way that PSPP commands
+execute.  The syntax execution subcommands are

 
 @table @asis
 
 @table @asis

+@item LOCALE
+Overrides the system locale for the purpose of reading and writing
+syntax and data files.  The argument should be a locale name in the
+general form @code{language_country.encoding}, where @code{language}
+and @code{country} are 2-character language and country abbreviations,
+respectively, and @code{encoding} is an IANA character set name.
+Example locales are @code{en_US.UTF-8} (UTF-8 encoded English as
+spoken in the United States) and @code{ja_JP.EUC-JP} (EUC-JP encoded
+Japanese as spoken in Japan).
+

 @item MEXPAND
 @itemx MITERATE
 @itemx MNEST
 @item MEXPAND
 @itemx MITERATE
 @itemx MNEST


@@ -768,7 +812,6 @@ SHOW
         [CCE]
         [COPYING]
         [DECIMALS]
         [CCE]
         [COPYING]
         [DECIMALS]

-        [ENDCMD]

         [FORMAT]
         [LENGTH]
         [MXERRS]
         [FORMAT]
         [LENGTH]
         [MXERRS]