Fix bugs deleting temporary files and their containing directory.

[pspp-builds.git] / doc / dev / system-file-format.texi

diff --git a/doc/dev/system-file-format.texi b/doc/dev/system-file-format.texi
index 757867cab230f5a7d6b11936d5b00951120f7332..7b0eff9f79823f0bb623e10d2f1da8c22922eff7 100644 (file)
--- a/doc/dev/system-file-format.texi
+++ b/doc/dev/system-file-format.texi
@@ -60,7 +60,8 @@ if present.
 Document record, if present.
 
 @item
 Document record, if present.
 
 @item

-Any records not explicitly included in this list, in any order.
+Extension (type 7) records, in ascending numerical order of their
+subtypes.

 
 @item
 Dictionary termination record.
 
 @item
 Dictionary termination record.


@@ -390,6 +391,11 @@ Format types are defined as follows:
 @end multitable
 @end quotation
 
 @end multitable
 @end quotation
 

+A few system files have been observed in the wild with invalid
+@code{write} fields, in particular with value 0.  Readers should
+probably treat invalid @code{print} or @code{write} fields as some
+default format.
+

 @node Value Labels Records
 @section Value Labels Records
 
 @node Value Labels Records
 @section Value Labels Records
 


@@ -549,14 +555,45 @@ Compression code.  Always set to 1.
 Machine endianness.  1 indicates big-endian, 2 indicates little-endian.
 
 @item int32 character_code;
 Machine endianness.  1 indicates big-endian, 2 indicates little-endian.
 
 @item int32 character_code;

-@anchor{character-code}
-Character code.  1 indicates EBCDIC, 2 indicates 7-bit ASCII, 3
-indicates 8-bit ASCII, 4 indicates DEC Kanji.
-Windows code page numbers are also valid.
-
-Experience has shown that in many files, this field is ignored or incorrect.
-For a more reliable indication of the file's character encoding
-see @ref{Character Encoding Record}.
+@anchor{character-code} Character code.  The following values have
+been actually observed in system files:
+
+@table @asis
+@item 2
+7-bit ASCII.
+
+@item 1250
+The @code{windows-1250} code page for Central European and Eastern
+European languages.
+
+@item 1252
+The @code{windows-1252} code page for Western European languages.
+
+@item 28591
+ISO 8859-1.
+
+@item 65001
+UTF-8.
+@end table
+
+The following additional values are known to be defined:
+
+@table @asis
+@item 1
+EBCDIC.
+
+@item 3
+8-bit ``ASCII''.
+
+@item 4
+DEC Kanji.
+@end table
+
+Other Windows code page numbers are known to be generally valid.
+
+Old versions of SPSS always wrote value 2 in this field, regardless of
+the encoding in use.  Newer versions also write the character encoding
+as a string (see @ref{Character Encoding Record}).

 @end table
 
 @node Machine Floating-Point Info Record
 @end table
 
 @node Machine Floating-Point Info Record


@@ -644,7 +681,8 @@ following:
 
 @itemize @bullet
 @item
 
 @itemize @bullet
 @item

-The set's name (an identifier that begins with @samp{$}).
+The set's name (an identifier that begins with @samp{$}), in mixed
+upper and lower case.

 
 @item
 An equals sign (@samp{=}).
 
 @item
 An equals sign (@samp{=}).


@@ -685,8 +723,8 @@ written if LABELSOURCE=VARLABEL was specified.
 A space.
 
 @item
 A space.
 
 @item

-The names of the variables in the set, each separated from the
-previous by a single space.
+The short names of the variables in the set, converted to lowercase,
+each separated from the previous by a single space.

 
 @item
 A line feed (byte 0x0a).
 
 @item
 A line feed (byte 0x0a).


@@ -774,8 +812,8 @@ Ordinal Scale
 Continuous Scale
 @end table
 
 Continuous Scale
 @end table
 

-SPSS 14 sometimes writes a @code{measure} of 0.  PSPP interprets this
-as nominal scale.
+SPSS 14 sometimes writes a @code{measure} of 0 for string variables.
+PSPP interprets this as nominal scale.

 
 @item int32 width;
 The width of the display column for the variable in characters.
 
 @item int32 width;
 The width of the display column for the variable in characters.


@@ -955,12 +993,29 @@ The size of each element in the @code{encoding} member. Always set to 1.
 The total number of bytes in @code{encoding}.
 
 @item char encoding[];
 The total number of bytes in @code{encoding}.
 
 @item char encoding[];

-The name of the character encoding.  Normally this will be an official IANA characterset name or alias.
+The name of the character encoding.  Normally this will be an official
+IANA character set name or alias.

 See @url{http://www.iana.org/assignments/character-sets}.
 See @url{http://www.iana.org/assignments/character-sets}.

+Character set names are not case-sensitive, but SPSS appears to write
+them in all-uppercase.

 @end table
 
 @end table
 

-This record is not present in files generated by older software.
-See also @ref{character-code}.
+This record is not present in files generated by older software.  See
+also the @code{character_code} field in the machine integer info
+record (@pxref{character-code}).
+
+When the character encoding record and the machine integer info record
+are both present, all system files observed in practice indicate the
+same character encoding, e.g.@: 1252 as @code{character_code} and
+@code{windows-1252} as @code{encoding}, 65001 and @code{UTF-8}, etc.
+
+If, for testing purposes, a file is crafted with different
+@code{character_code} and @code{encoding}, it seems that
+@code{character_code} controls the encoding for all strings in the
+system file before the dictionary termination record, including
+strings in data (e.g.@: string missing values), and @code{encoding}
+controls the encoding for strings following the dictionary termination
+record.

 
 @node Long String Value Labels Record
 @section Long String Value Labels Record
 
 @node Long String Value Labels Record
 @section Long String Value Labels Record


@@ -1087,7 +1142,8 @@ element.
 In record type 18, this field contains a sequence of one or more
 variable attribute sets.  If more than one variable attribute set is
 present, each one after the first is delimited from the previous by
 In record type 18, this field contains a sequence of one or more
 variable attribute sets.  If more than one variable attribute set is
 present, each one after the first is delimited from the previous by

-@code{/}.  Each variable attribute set consists of a variable name,
+@code{/}.  Each variable attribute set consists of a long
+variable name,

 followed by @code{:}, followed by an attribute set with the same
 syntax as on record type 17.
 
 followed by @code{:}, followed by an attribute set with the same
 syntax as on record type 17.