floating-point format in use, as well as the endianness of IEEE 754
floating-point numbers, and translates as needed. However, only IEEE
754 numbers with the same endianness as integer data in the same file
-has actually been observed in system files, and it is likely that
+have actually been observed in system files, and it is likely that
other formats are obsolete or were never used.
System files use a few floating point values for special purposes:
possible to artificially synthesize files that use different encodings
(@pxref{Character Encoding Record}).
-System files are divided into records, each of which begins with a
-4-byte record type, usually regarded as an @code{int32}.
+@menu
+* System File Record Structure::
+* File Header Record::
+* Variable Record::
+* Value Labels Records::
+* Document Record::
+* Machine Integer Info Record::
+* Machine Floating-Point Info Record::
+* Multiple Response Sets Records::
+* Extra Product Info Record::
+* Variable Display Parameter Record::
+* Long Variable Names Record::
+* Very Long String Record::
+* Character Encoding Record::
+* Long String Value Labels Record::
+* Long String Missing Values Record::
+* Data File and Variable Attributes Records::
+* Extended Number of Cases Record::
+* Other Informational Records::
+* Dictionary Termination Record::
+* Data Record::
+* Encrypted System Files::
+@end menu
+
+@node System File Record Structure
+@section System File Record Structure
+
+System files are divided into records with the following format:
+
+@example
+int32 type;
+char data[];
+@end example
+
+This header does not identify the length of the @code{data} or any
+information about what it contains, so the system file reader must
+understand the format of @code{data} based on @code{type}. However,
+records with type 7, called @dfn{extension records}, have a stricter
+format:
+
+@example
+int32 type;
+int32 subtype;
+int32 size;
+int32 count;
+char data[size * count];
+@end example
+
+@table @code
+@item int32 rec_type;
+Record type. Always set to 7.
+
+@item int32 subtype;
+Record subtype. This value identifies a particular kind of extension
+record.
+
+@item int32 size;
+The size of each piece of data that follows the header, in bytes.
+Known extension records use 1, 4, or 8, for @code{char}, @code{int32},
+and @code{flt64} format data, respectively.
+
+@item int32 count;
+The number of pieces of data that follow the header.
-The records must appear in the following order:
+@item char data[size * count];
+Data, whose format and interpretation depend on the subtype.
+@end table
+
+An extension record contains exactly @code{size * count} bytes of
+data, which allows a reader that does not understand an extension
+record to skip it. Extension records provide only nonessential
+information, so this allows for files written by newer software to
+preserve backward compatibility with older or less capable readers.
+
+Records in a system file must appear in the following order:
@itemize @bullet
@item
Data record.
@end itemize
-Each type of record is described separately below.
+We advise authors of programs that read system files to tolerate
+format variations. Various kinds of misformatting and corruption have
+been observed in system files written by SPSS and other software
+alike. In particular, because extension records provide nonessential
+information, it is generally better to ignore an extension record
+entirely than to refuse to read a system file.
-@menu
-* File Header Record::
-* Variable Record::
-* Value Labels Records::
-* Document Record::
-* Machine Integer Info Record::
-* Machine Floating-Point Info Record::
-* Multiple Response Sets Records::
-* Extra Product Info Record::
-* Variable Display Parameter Record::
-* Long Variable Names Record::
-* Very Long String Record::
-* Character Encoding Record::
-* Long String Value Labels Record::
-* Long String Missing Values Record::
-* Data File and Variable Attributes Records::
-* Extended Number of Cases Record::
-* Miscellaneous Informational Records::
-* Dictionary Termination Record::
-* Data Record::
-* Encrypted System Files::
-@end menu
+The following sections describe the known kinds of records.
@node File Header Record
@section File Header Record
-The file header is always the first record in the file. It has the
-following format:
+A system file begins with the file header, with the following format:
@example
char rec_type[4];
by a number of narrower string variables. @xref{Very Long String
Record}, for details.
+A system file should contain at least one variable and thus at least
+one variable record, but system files have been observed in the wild
+without any variables (thus, no data either).
+
@example
int32 rec_type;
int32 type;
Number of pieces of data in the data part. Always set to 3.
@item flt64 sysmis;
-The system missing value.
-
-@item flt64 highest;
-The value used for HIGHEST in missing values.
-
-@item flt64 lowest;
-The value used for LOWEST in missing values.
+@itemx flt64 highest;
+@itemx flt64 lowest;
+The system missing value, the value used for HIGHEST in missing
+values, and the value used for LOWEST in missing values, respectively.
+@xref{System File Format}, for more information.
+
+The SPSSWriter library in PHP, which identifies itself as @code{FOM
+SPSS 1.0.0} in the file header record @code{prod_name} field, writes
+unexpected values to these fields, but it uses the same values
+consistently throughout the rest of the file.
@end table
@node Multiple Response Sets Records
each separated from the previous by a single space.
Even though a multiple response set must have at least two variables,
-some system files contain multiple response sets with no variables at
-all. The source and meaning of these multiple response sets is
+some system files contain multiple response sets with no variables or
+one variable. The source and meaning of these multiple response sets is
unknown. (Perhaps they arise from creating a multiple response set
then deleting all the variables that it contains?)
not been observed in the wild.
@end table
-@node Miscellaneous Informational Records
-@section Miscellaneous Informational Records
+@node Other Informational Records
+@section Other Informational Records
-Some specific types of miscellaneous informational records are
+This chapter documents many specific types of extension records are
documented here, but others are known to exist. PSPP ignores unknown
-miscellaneous informational records when reading system files.
-
-@example
-/* @r{Header.} */
-int32 rec_type;
-int32 subtype;
-int32 size;
-int32 count;
-
-/* @r{Exactly @code{size * count} bytes of data.} */
-char data[];
-@end example
+extension records when reading system files.
-@table @code
-@item int32 rec_type;
-Record type. Always set to 7.
+The following extension record subtypes have also been observed, with
+the following believed meanings:
-@item int32 subtype;
-Record subtype. May take any value. According to Aapi
-H@"am@"al@"ainen, value 5 indicates a set of grouped variables and 6
-indicates date info (probably related to USE). Subtype 24 appears to
-contain XML that describes how data in the file should be displayed
-on-screen.
+@table @asis
+@item 5
+A set of grouped variables (according to Aapi H@"am@"al@"ainen).
-@item int32 size;
-Size of each piece of data in the data part. Should have the value 1,
-4, or 8, for @code{char}, @code{int32}, and @code{flt64} format data,
-respectively.
+@item 6
+Date info, probably related to USE (according to Aapi H@"am@"al@"ainen).
-@item int32 count;
-Number of pieces of data in the data part.
+@item 12
+A UUID in the format described in RFC 4122. Only two examples
+observed, both written by SPSS 13, and in each case the UUID contained
+both upper and lower case.
-@item char data[];
-Arbitrary data. There must be @code{size} times @code{count} bytes of
-data.
+@item 24
+XML that describes how data in the file should be displayed on-screen.
@end table
@node Dictionary Termination Record
projects / pspp / blobdiff