Automatic inference of measeurement level works.

[pspp] / doc / dev / system-file-format.texi

diff --git a/doc/dev/system-file-format.texi b/doc/dev/system-file-format.texi
index 484fbb43f1eca08fc089bd0f5ec171215f279403..476aec405a3e909a36f29547eee0879be3d2d607 100644 (file)
--- a/doc/dev/system-file-format.texi
+++ b/doc/dev/system-file-format.texi
@@ -1,3 +1,13 @@
+@c PSPP - a program for statistical analysis.
+@c Copyright (C) 2019 Free Software Foundation, Inc.
+@c Permission is granted to copy, distribute and/or modify this document
+@c under the terms of the GNU Free Documentation License, Version 1.3
+@c or any later version published by the Free Software Foundation;
+@c with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
+@c A copy of the license is included in the section entitled "GNU
+@c Free Documentation License".
+@c
+
 @node System File Format
 @appendix System File Format
 
@@ -6,7 +16,7 @@ that describes how they may be interpreted.  This chapter describes
 the format of a system file.
 
 System files use four data types: 8-bit characters, 32-bit integers,
-64-bit integers, 
+64-bit integers,
 and 64-bit floating points, called here @code{char}, @code{int32},
 @code{int64}, and
 @code{flt64}, respectively.  Data is not necessarily aligned on a word
@@ -316,10 +326,10 @@ so readers should take care to parse dummy variable records in the
 same way as other variable records.
 
 @anchor{Dictionary Index}
-The @dfn{dictionary index} of a variable is its offset in the set of
+The @dfn{dictionary index} of a variable is a 1-based offset in the set of
 variable records, including dummy variable records for long string
-variables.  The first variable record has a dictionary index of 0, the
-second has a dictionary index of 1, and so on.
+variables.  The first variable record has a dictionary index of 1, the
+second has a dictionary index of 2, and so on.
 
 The system file format does not directly support string variables
 wider than 255 bytes.  Such very long string variables are represented
@@ -514,6 +524,10 @@ Format types are defined as follows:
 @tab @code{EDATE}
 @item 39
 @tab @code{SDATE}
+@item 40
+@tab @code{MTIME}
+@item 41
+@tab @code{YMDHMS}
 @end multitable
 @end quotation
 
@@ -544,7 +558,7 @@ The value label record has the following format:
 int32               rec_type;
 int32               label_count;
 
-/* @r{Repeated @code{label_cnt} times}. */
+/* @r{Repeated @code{n_label} times}. */
 char                value[8];
 char                label_len;
 char                label[];
@@ -596,7 +610,7 @@ Number of variables that the associated value labels from the value
 label record are to be applied.
 
 @item int32 vars[];
-A list of dictionary indexes of variables to which to apply the value
+A list of 1-based dictionary indexes of variables to which to apply the value
 labels (@pxref{Dictionary Index}).  There are @code{var_count}
 elements.
 
@@ -987,18 +1001,24 @@ members are as follows:
 
 @table @code
 @item int32 measure;
-The measurement type of the variable:
+The measurement level of the variable:
 @table @asis
+@item 0
+Unknown
 @item 1
-Nominal Scale
+Nominal
 @item 2
-Ordinal Scale
+Ordinal
 @item 3
-Continuous Scale
+Scale
 @end table
 
-SPSS sometimes writes a @code{measure} of 0.  PSPP interprets this as
-nominal scale.
+An ``unknown'' @code{measure} of 0 means that the variable was created
+in some way that doesn't make the measurement level clear, e.g.@: with
+a @code{COMPUTE} transformation.  PSPP sets the measurement level the
+first time it reads the data using the rules documented in
+@ref{Measurement Level,,,pspp, PSPP Users Guide}, so this should
+rarely appear.
 
 @item int32 width;
 The width of the display column for the variable in characters.
@@ -1572,9 +1592,10 @@ value @var{code} - @var{bias}, where
 variable @code{bias} from the file header.  For example,
 code 105 with bias 100.0 (the normal value) indicates a numeric variable
 of value 5.
-One file has been seen written by SPSS 14 that contained such a code
-in a @emph{string} field with the value 0 (after the bias is
-subtracted) as a way of encoding null bytes.
+
+A code of 0 (after subtracting the bias) in a string field encodes
+null bytes.  This is unusual, since a string field normally encodes
+text data, but it exists in real system files.
 
 @item 252
 End of file.  This code may or may not appear at the end of the data
@@ -1636,7 +1657,7 @@ The number of bytes in the ZLIB data trailer.  This and the previous
 field sum to the size of the system file in bytes.
 @end table
 
-The data header is followed by @code{(ztrailer_ofs - 24) / 24} ZLIB
+The data header is followed by @code{(ztrailer_len - 24) / 24} ZLIB
 compressed data blocks.  Each ZLIB compressed data block begins with a
 ZLIB header as specified in RFC@tie{}1950, e.g.@: hex bytes @code{78
 01} (the only header yet observed in practice).  Each block
@@ -1673,7 +1694,7 @@ been observed so far.
 
 @item int32 n_blocks;
 The number of ZLIB compressed data blocks, always exactly
-@code{(ztrailer_ofs - 24) / 24}.
+@code{(ztrailer_len - 24) / 24}.
 @end table
 
 The fixed header is followed by @code{n_blocks} 24-byte ZLIB data