1 @node System File Format
2 @appendix System File Format
4 A system file encapsulates a set of cases and dictionary information
5 that describes how they may be interpreted. This chapter describes
6 the format of a system file.
8 System files use four data types: 8-bit characters, 32-bit integers,
10 and 64-bit floating points, called here @code{char}, @code{int32},
12 @code{flt64}, respectively. Data is not necessarily aligned on a word
13 or double-word boundary: the long variable name record (@pxref{Long
14 Variable Names Record}) and very long string records (@pxref{Very Long
15 String Record}) have arbitrary byte length and can therefore cause all
16 data coming after them in the file to be misaligned.
18 Integer data in system files may be big-endian or little-endian. A
19 reader may detect the endianness of a system file by examining
20 @code{layout_code} in the file header record
21 (@pxref{layout_code,,@code{layout_code}}).
23 Floating-point data in system files may nominally be in IEEE 754, IBM,
24 or VAX formats. A reader may detect the floating-point format in use
25 by examining @code{bias} in the file header record
26 (@pxref{bias,,@code{bias}}).
28 PSPP detects big-endian and little-endian integer formats in system
29 files and translates as necessary. PSPP also detects the
30 floating-point format in use, as well as the endianness of IEEE 754
31 floating-point numbers, and translates as needed. However, only IEEE
32 754 numbers with the same endianness as integer data in the same file
33 has actually been observed in system files, and it is likely that
34 other formats are obsolete or were never used.
36 The PSPP system-missing value is represented by the largest possible
37 negative number in the floating point format (@code{-DBL_MAX}). Two
38 other values are important for use as missing values: @code{HIGHEST},
39 represented by the largest possible positive number (@code{DBL_MAX}),
40 and @code{LOWEST}, represented by the second-largest negative number
41 (in IEEE 754 format, @code{0xffeffffffffffffe}).
43 System files are divided into records, each of which begins with a
44 4-byte record type, usually regarded as an @code{int32}.
46 The records must appear in the following order:
56 All pairs of value labels records and value label variables records,
60 Document record, if present.
63 Any records not explicitly included in this list, in any order.
66 Dictionary termination record.
72 Each type of record is described separately below.
75 * File Header Record::
77 * Value Labels Records::
79 * Machine Integer Info Record::
80 * Machine Floating-Point Info Record::
81 * Variable Display Parameter Record::
82 * Long Variable Names Record::
83 * Very Long String Record::
84 * Character Encoding Record::
85 * Long String Value Labels Record::
86 * Data File and Variable Attributes Records::
87 * Extended Number of Cases Record::
88 * Miscellaneous Informational Records::
89 * Dictionary Termination Record::
93 @node File Header Record
94 @section File Header Record
96 The file header is always the first record in the file. It has the
103 int32 nominal_case_size;
108 char creation_date[9];
109 char creation_time[8];
115 @item char rec_type[4];
116 Record type code, set to @samp{$FL2}.
118 @item char prod_name[60];
119 Product identification string. This always begins with the characters
120 @samp{@@(#) SPSS DATA FILE}. PSPP uses the remaining characters to
121 give its version and the operating system name; for example, @samp{GNU
122 pspp 0.1.4 - sparc-sun-solaris2.5.2}. The string is truncated if it
123 would be longer than 60 characters; otherwise it is padded on the right
127 @item int32 layout_code;
128 Normally set to 2, although a few system files have been spotted in
129 the wild with a value of 3 here. PSPP use this value to determine the
130 file's integer endianness (@pxref{System File Format}).
132 @item int32 nominal_case_size;
133 Number of data elements per case. This is the number of variables,
134 except that long string variables add extra data elements (one for every
135 8 characters after the first 8). However, string variables do not
136 contribute to this value beyond the first 255 bytes. Further, system
137 files written by some systems set this value to -1. In general, it is
138 unsafe for systems reading system files to rely upon this value.
140 @item int32 compressed;
141 Set to 1 if the data in the file is compressed, 0 otherwise.
143 @item int32 weight_index;
144 If one of the variables in the data set is used as a weighting
145 variable, set to the dictionary index of that variable, plus 1
146 (@pxref{Dictionary Index}). Otherwise, set to 0.
149 Set to the number of cases in the file if it is known, or -1 otherwise.
151 In the general case it is not possible to determine the number of cases
152 that will be output to a system file at the time that the header is
153 written. The way that this is dealt with is by writing the entire
154 system file, including the header, then seeking back to the beginning of
155 the file and writing just the @code{ncases} field. For files in which
156 this is not valid, the seek operation fails. In this case,
157 @code{ncases} remains -1.
161 Compression bias, ordinarily set to 100. Only integers between
162 @code{1 - bias} and @code{251 - bias} can be compressed.
164 By assuming that its value is 100, PSPP uses @code{bias} to determine
165 the file's floating-point format and endianness (@pxref{System File
166 Format}). If the compression bias is not 100, PSPP cannot auto-detect
167 the floating-point format and assumes that it is IEEE 754 format with
168 the same endianness as the system file's integers, which is correct
169 for all known system files.
171 @item char creation_date[9];
172 Date of creation of the system file, in @samp{dd mmm yy}
173 format, with the month as standard English abbreviations, using an
174 initial capital letter and following with lowercase. If the date is not
175 available then this field is arbitrarily set to @samp{01 Jan 70}.
177 @item char creation_time[8];
178 Time of creation of the system file, in @samp{hh:mm:ss}
179 format and using 24-hour time. If the time is not available then this
180 field is arbitrarily set to @samp{00:00:00}.
182 @item char file_label[64];
183 File label declared by the user, if any (@pxref{FILE LABEL,,,pspp,
184 PSPP Users Guide}). Padded on the right with spaces.
186 @item char padding[3];
187 Ignored padding bytes to make the structure a multiple of 32 bits in
188 length. Set to zeros.
191 @node Variable Record
192 @section Variable Record
194 There must be one variable record for each numeric variable and each
195 string variable with width 8 bytes or less. String variables wider
196 than 8 bytes have one variable record for each 8 bytes, rounding up.
197 The first variable record for a long string specifies the variable's
198 correct dictionary information. Subsequent variable records for a
199 long string are filled with dummy information: a type of -1, no
200 variable label or missing values, print and write formats that are
201 ignored, and an empty string as name. A few system files have been
202 encountered that include a variable label on dummy variable records,
203 so readers should take care to parse dummy variable records in the
204 same way as other variable records.
206 @anchor{Dictionary Index}
207 The @dfn{dictionary index} of a variable is its offset in the set of
208 variable records, including dummy variable records for long string
209 variables. The first variable record has a dictionary index of 0, the
210 second has a dictionary index of 1, and so on.
212 The system file format does not directly support string variables
213 wider than 255 bytes. Such very long string variables are represented
214 by a number of narrower string variables. @xref{Very Long String
215 Record}, for details.
221 int32 n_missing_values;
226 /* @r{Present only if @code{has_var_label} is 1.} */
230 /* @r{Present only if @code{n_missing_values} is nonzero}. */
231 flt64 missing_values[];
235 @item int32 rec_type;
236 Record type code. Always set to 2.
239 Variable type code. Set to 0 for a numeric variable. For a short
240 string variable or the first part of a long string variable, this is set
241 to the width of the string. For the second and subsequent parts of a
242 long string variable, set to -1, and the remaining fields in the
243 structure are ignored.
245 @item int32 has_var_label;
246 If this variable has a variable label, set to 1; otherwise, set to 0.
248 @item int32 n_missing_values;
249 If the variable has no missing values, set to 0. If the variable has
250 one, two, or three discrete missing values, set to 1, 2, or 3,
251 respectively. If the variable has a range for missing variables, set to
252 -2; if the variable has a range for missing variables plus a single
253 discrete value, set to -3.
256 Print format for this variable. See below.
259 Write format for this variable. See below.
262 Variable name. The variable name must begin with a capital letter or
263 the at-sign (@samp{@@}). Subsequent characters may also be digits, octothorpes
264 (@samp{#}), dollar signs (@samp{$}), underscores (@samp{_}), or full
265 stops (@samp{.}). The variable name is padded on the right with spaces.
267 @item int32 label_len;
268 This field is present only if @code{has_var_label} is set to 1. It is
269 set to the length, in characters, of the variable label, which must be a
270 number between 0 and 120.
273 This field is present only if @code{has_var_label} is set to 1. It has
274 length @code{label_len}, rounded up to the nearest multiple of 32 bits.
275 The first @code{label_len} characters are the variable's variable label.
277 @item flt64 missing_values[];
278 This field is present only if @code{n_missing_values} is nonzero. It
279 has the same number of 8-byte elements as the absolute value of
280 @code{n_missing_values}. Each element is interpreted as a number for
281 numeric variables (with HIGHEST and LOWEST indicated as described in
282 the chapter introduction). For string variables of width less than 8
283 bytes, elements are right-padded with spaces; for string variables
284 wider than 8 bytes, only the first 8 bytes of each missing value are
285 specified, with the remainder implicitly all spaces.
287 For discrete missing values, each element represents one missing
288 value. When a range is present, the first element denotes the minimum
289 value in the range, and the second element denotes the maximum value
290 in the range. When a range plus a value are present, the third
291 element denotes the additional discrete missing value.
294 The @code{print} and @code{write} members of sysfile_variable are output
295 formats coded into @code{int32} types. The least-significant byte
296 of the @code{int32} represents the number of decimal places, and the
297 next two bytes in order of increasing significance represent field width
298 and format type, respectively. The most-significant byte is not
299 used and should be set to zero.
301 Format types are defined as follows:
304 @multitable {Value} {@code{DATETIME}}
390 @node Value Labels Records
391 @section Value Labels Records
393 The value label records documented in this section are used for
394 numeric and short string variables only. Long string variables may
395 have value labels, but their value labels are recorded using a
396 different record type (@pxref{Long String Value Labels Record}).
398 The value label record has the following format:
404 /* @r{Repeated @code{label_cnt} times}. */
411 @item int32 rec_type;
412 Record type. Always set to 3.
414 @item int32 label_count;
415 Number of value labels present in this record.
418 The remaining fields are repeated @code{count} times. Each
419 repetition specifies one value label.
423 A numeric value or a short string value padded as necessary to 8 bytes
424 in length. Its type and width cannot be determined until the
425 following value label variables record (see below) is read.
427 @item char label_len;
428 The label's length, in bytes.
431 @code{label_len} bytes of the actual label, followed by up to 7 bytes
432 of padding to bring @code{label} and @code{label_len} together to a
433 multiple of 8 bytes in length.
436 The value label record is always immediately followed by a value label
437 variables record with the following format:
446 @item int32 rec_type;
447 Record type. Always set to 4.
449 @item int32 var_count;
450 Number of variables that the associated value labels from the value
451 label record are to be applied.
454 A list of dictionary indexes of variables to which to apply the value
455 labels (@pxref{Dictionary Index}). There are @code{var_count}
458 String variables wider than 8 bytes may not be specified in this list.
461 @node Document Record
462 @section Document Record
464 The document record, if present, has the following format:
473 @item int32 rec_type;
474 Record type. Always set to 6.
477 Number of lines of documents present.
479 @item char lines[][80];
480 Document lines. The number of elements is defined by @code{n_lines}.
481 Lines shorter than 80 characters are padded on the right with spaces.
484 @node Machine Integer Info Record
485 @section Machine Integer Info Record
487 The integer info record, if present, has the following format:
499 int32 version_revision;
501 int32 floating_point_rep;
502 int32 compression_code;
504 int32 character_code;
508 @item int32 rec_type;
509 Record type. Always set to 7.
512 Record subtype. Always set to 3.
515 Size of each piece of data in the data part, in bytes. Always set to 4.
518 Number of pieces of data in the data part. Always set to 8.
520 @item int32 version_major;
521 PSPP major version number. In version @var{x}.@var{y}.@var{z}, this
524 @item int32 version_minor;
525 PSPP minor version number. In version @var{x}.@var{y}.@var{z}, this
528 @item int32 version_revision;
529 PSPP version revision number. In version @var{x}.@var{y}.@var{z},
532 @item int32 machine_code;
533 Machine code. PSPP always set this field to value to -1, but other
536 @item int32 floating_point_rep;
537 Floating point representation code. For IEEE 754 systems this is 1.
538 IBM 370 sets this to 2, and DEC VAX E to 3.
540 @item int32 compression_code;
541 Compression code. Always set to 1.
543 @item int32 endianness;
544 Machine endianness. 1 indicates big-endian, 2 indicates little-endian.
546 @item int32 character_code;
547 @anchor{character-code}
548 Character code. 1 indicates EBCDIC, 2 indicates 7-bit ASCII, 3
549 indicates 8-bit ASCII, 4 indicates DEC Kanji.
550 Windows code page numbers are also valid.
552 Experience has shown that in many files, this field is ignored or incorrect.
553 For a more reliable indication of the file's character encoding
554 see @ref{Character Encoding Record}.
557 @node Machine Floating-Point Info Record
558 @section Machine Floating-Point Info Record
560 The floating-point info record, if present, has the following format:
576 @item int32 rec_type;
577 Record type. Always set to 7.
580 Record subtype. Always set to 4.
583 Size of each piece of data in the data part, in bytes. Always set to 8.
586 Number of pieces of data in the data part. Always set to 3.
589 The system missing value.
592 The value used for HIGHEST in missing values.
595 The value used for LOWEST in missing values.
598 @node Variable Display Parameter Record
599 @section Variable Display Parameter Record
601 The variable display parameter record, if present, has the following
611 /* @r{Repeated @code{count} times}. */
613 int32 width; /* @r{Not always present.} */
618 @item int32 rec_type;
619 Record type. Always set to 7.
622 Record subtype. Always set to 11.
625 The size of @code{int32}. Always set to 4.
628 The number of sets of variable display parameters (ordinarily the
629 number of variables in the dictionary), times 2 or 3.
632 The remaining members are repeated @code{count} times, in the same
633 order as the variable records. No element corresponds to variable
634 records that continue long string variables. The meanings of these
635 members are as follows:
639 The measurement type of the variable:
649 SPSS 14 sometimes writes a @code{measure} of 0. PSPP interprets this
653 The width of the display column for the variable in characters.
655 This field is present if @var{count} is 3 times the number of
656 variables in the dictionary. It is omitted if @var{count} is 2 times
657 the number of variables.
659 @item int32 alignment;
660 The alignment of the variable for display purposes:
672 @node Long Variable Names Record
673 @section Long Variable Names Record
675 If present, the long variable names record has the following format:
684 /* @r{Exactly @code{count} bytes of data.} */
685 char var_name_pairs[];
689 @item int32 rec_type;
690 Record type. Always set to 7.
693 Record subtype. Always set to 13.
696 The size of each element in the @code{var_name_pairs} member. Always set to 1.
699 The total number of bytes in @code{var_name_pairs}.
701 @item char var_name_pairs[];
702 A list of @var{key}--@var{value} tuples, where @var{key} is the name
703 of a variable, and @var{value} is its long variable name.
704 The @var{key} field is at most 8 bytes long and must match the
705 name of a variable which appears in the variable record (@pxref{Variable
707 The @var{value} field is at most 64 bytes long.
708 The @var{key} and @var{value} fields are separated by a @samp{=} byte.
709 Each tuple is separated by a byte whose value is 09. There is no
710 trailing separator following the last tuple.
711 The total length is @code{count} bytes.
714 @node Very Long String Record
715 @section Very Long String Record
717 Old versions of SPSS limited string variables to a width of 255 bytes.
718 For backward compatibility with these older versions, the system file
719 format represents a string longer than 255 bytes, called a @dfn{very
720 long string}, as a collection of strings no longer than 255 bytes
721 each. The strings concatenated to make a very long string are called
722 its @dfn{segments}; for consistency, variables other than very long
723 strings are considered to have a single segment.
725 A very long string with a width of @var{w} has @var{n} =
726 (@var{w} + 251) / 252 segments, that is, one segment for every
727 252 bytes of width, rounding up. It would be logical, then, for each
728 of the segments except the last to have a width of 252 and the last
729 segment to have the remainder, but this is not the case. In fact,
730 each segment except the last has a width of 255 bytes. The last
731 segment has width @var{w} - (@var{n} - 1) * 252; some versions
732 of SPSS make it slightly wider, but not wide enough to make the last
733 segment require another 8 bytes of data.
735 Data is packed tightly into segments of a very long string, 255 bytes
736 per segment. Because 255 bytes of segment data are allocated for
737 every 252 bytes of the very long string's width (approximately), some
738 unused space is left over at the end of the allocated segments. Data
739 in unused space is ignored.
741 Example: Consider a very long string of width 20,000. Such a very
742 long string has 20,000 / 252 = 80 (rounding up) segments. The first
743 79 segments have width 255; the last segment has width 20,000 - 79 *
744 252 = 92 or slightly wider (up to 96 bytes, the next multiple of 8).
745 The very long string's data is actually stored in the 19,890 bytes in
746 the first 78 segments, plus the first 110 bytes of the 79th segment
747 (19,890 + 110 = 20,000). The remaining 145 bytes of the 79th segment
748 and all 92 bytes of the 80th segment are unused.
750 The very long string record explains how to stitch together segments
751 to obtain very long string data. For each of the very long string
752 variables in the dictionary, it specifies the name of its first
753 segment's variable and the very long string variable's actual width.
754 The remaining segments immediately follow the named variable in the
755 system file's dictionary.
757 The very long string record, which is present only if the system file
758 contains very long string variables, has the following format:
767 /* @r{Exactly @code{count} bytes of data.} */
768 char string_lengths[];
772 @item int32 rec_type;
773 Record type. Always set to 7.
776 Record subtype. Always set to 14.
779 The size of each element in the @code{string_lengths} member. Always set to 1.
782 The total number of bytes in @code{string_lengths}.
784 @item char string_lengths[];
785 A list of @var{key}--@var{value} tuples, where @var{key} is the name
786 of a variable, and @var{value} is its length.
787 The @var{key} field is at most 8 bytes long and must match the
788 name of a variable which appears in the variable record (@pxref{Variable
790 The @var{value} field is exactly 5 bytes long. It is a zero-padded,
791 ASCII-encoded string that is the length of the variable.
792 The @var{key} and @var{value} fields are separated by a @samp{=} byte.
793 Tuples are delimited by a two-byte sequence @{00, 09@}.
794 After the last tuple, there may be a single byte 00, or @{00, 09@}.
795 The total length is @code{count} bytes.
798 @node Character Encoding Record
799 @section Character Encoding Record
801 This record, if present, indicates the character encoding for string data,
802 long variable names, variable labels, value labels and other strings in the
812 /* @r{Exactly @code{count} bytes of data.} */
817 @item int32 rec_type;
818 Record type. Always set to 7.
821 Record subtype. Always set to 20.
824 The size of each element in the @code{encoding} member. Always set to 1.
827 The total number of bytes in @code{encoding}.
829 @item char encoding[];
830 The name of the character encoding. Normally this will be an official IANA characterset name or alias.
831 See @url{http://www.iana.org/assignments/character-sets}.
834 This record is not present in files generated by older software.
835 See also @ref{character-code}.
837 @node Long String Value Labels Record
838 @section Long String Value Labels Record
840 This record, if present, specifies value labels for long string
850 /* @r{Repeated up to exactly @code{count} bytes.} */
855 long_string_label labels[];
859 @item int32 rec_type;
860 Record type. Always set to 7.
863 Record subtype. Always set to 21.
869 The number of bytes following the header until the next header.
871 @item int32 var_name_len;
872 @itemx char var_name[];
873 The number of bytes in the name of the variable that has long string
874 value labels, plus the variable name itself, which consists of exactly
875 @code{var_name_len} bytes. The variable name is not padded to any
876 particular boundary, nor is it null-terminated.
878 @item int32 var_width;
879 The width of the variable, in bytes, which will be between 9 and
882 @item int32 n_labels;
883 @itemx long_string_label labels[];
884 The long string labels themselves. The @code{labels} array contains
885 exactly @code{n_labels} elements, each of which has the following
896 @item int32 value_len;
898 The string value being labeled. @code{value_len} is the number of
899 bytes in @code{value}; it is equal to @code{var_width}. The
900 @code{value} array is not padded or null-terminated.
902 @item int32 label_len;
904 The label for the string value. @code{label_len}, which must be
905 between 0 and 120, is the number of bytes in @code{label}. The
906 @code{label} array is not padded or null-terminated.
910 @node Data File and Variable Attributes Records
911 @section Data File and Variable Attributes Records
913 The data file and variable attributes records represent custom
914 attributes for the system file or for individual variables in the
915 system file, as defined on the DATAFILE ATTRIBUTE (@pxref{DATAFILE
916 ATTRIBUTE,,,pspp, PSPP Users Guide}) and VARIABLE ATTRIBUTE commands
917 (@pxref{VARIABLE ATTRIBUTE,,,pspp, PSPP Users Guide}), respectively.
926 /* @r{Exactly @code{count} bytes of data.} */
931 @item int32 rec_type;
932 Record type. Always set to 7.
935 Record subtype. Always set to 17 for a data file attribute record or
936 to 18 for a variable attributes record.
939 The size of each element in the @code{attributes} member. Always set to 1.
942 The total number of bytes in @code{attributes}.
944 @item char attributes[];
945 The attributes, in a text-based format.
947 In record type 17, this field contains a single attribute set. An
948 attribute set is a sequence of one or more attributes concatenated
949 together. Each attribute consists of a name, which has the same
950 syntax as a variable name, followed by, inside parentheses, a sequence
951 of one or more values. Each value consists of a string enclosed in
952 single quotes (@code{'}) followed by a line feed (byte 0x0a). A value
953 may contain single quote characters, which are not themselves escaped
954 or quoted or required to be present in pairs. There is no apparent
955 way to embed a line feed in a value. There is no distinction between
956 an attribute with a single value and an attribute array with one
959 In record type 18, this field contains a sequence of one or more
960 variable attribute sets. If more than one variable attribute set is
961 present, each one after the first is delimited from the previous by
962 @code{/}. Each variable attribute set consists of a variable name,
963 followed by @code{:}, followed by an attribute set with the same
964 syntax as on record type 17.
966 The total length is @code{count} bytes.
971 A system file produced with the following VARIABLE ATTRIBUTE commands
975 VARIABLE ATTRIBUTE VARIABLES=dummy ATTRIBUTE=fred[1]('23') fred[2]('34').
976 VARIABLE ATTRIBUTE VARIABLES=dummy ATTRIBUTE=bert('123').
980 will contain a variable attribute record with the following contents:
983 00000000 07 00 00 00 12 00 00 00 01 00 00 00 22 00 00 00 |............"...|
984 00000010 64 75 6d 6d 79 3a 66 72 65 64 28 27 32 33 27 0a |dummy:fred('23'.|
985 00000020 27 33 34 27 0a 29 62 65 72 74 28 27 31 32 33 27 |'34'.)bert('123'|
989 @node Extended Number of Cases Record
990 @section Extended Number of Cases Record
992 The file header record expresses the number of cases in the system
993 file as an int32 (@pxref{File Header Record}). This record allows the
994 number of cases in the system file to be expressed as a 64-bit number.
1006 @item int32 rec_type;
1007 Record type. Always set to 7.
1009 @item int32 subtype;
1010 Record subtype. Always set to 16.
1013 Size of each element. Always set to 8.
1016 Number of pieces of data in the data part. Alway set to 2.
1018 @item int64 unknown;
1019 Meaning unknown. Always set to 1.
1021 @item int64 ncases64;
1022 Number of cases in the file as a 64-bit integer. Presumably this
1023 could be -1 to indicate that the number of cases is unknown, for the
1024 same reason as @code{ncases} in the file header record, but this has
1025 not been observed in the wild.
1028 @node Miscellaneous Informational Records
1029 @section Miscellaneous Informational Records
1031 Some specific types of miscellaneous informational records are
1032 documented here, but others are known to exist. PSPP ignores unknown
1033 miscellaneous informational records when reading system files.
1042 /* @r{Exactly @code{size * count} bytes of data.} */
1047 @item int32 rec_type;
1048 Record type. Always set to 7.
1050 @item int32 subtype;
1051 Record subtype. May take any value. According to Aapi
1052 H@"am@"al@"ainen, value 5 indicates a set of grouped variables and 6
1053 indicates date info (probably related to USE).
1056 Size of each piece of data in the data part. Should have the value 1,
1057 4, or 8, for @code{char}, @code{int32}, and @code{flt64} format data,
1061 Number of pieces of data in the data part.
1064 Arbitrary data. There must be @code{size} times @code{count} bytes of
1068 @node Dictionary Termination Record
1069 @section Dictionary Termination Record
1071 The dictionary termination record separates all other records from the
1080 @item int32 rec_type;
1081 Record type. Always set to 999.
1084 Ignored padding. Should be set to 0.
1088 @section Data Record
1090 Data records must follow all other records in the system file. There must
1091 be at least one data record in every system file.
1093 The format of data records varies depending on whether the data is
1094 compressed. Regardless, the data is arranged in a series of 8-byte
1097 When data is not compressed,
1098 each element corresponds to
1099 the variable declared in the respective variable record (@pxref{Variable
1100 Record}). Numeric values are given in @code{flt64} format; string
1101 values are literal characters string, padded on the right when
1102 necessary to fill out 8-byte units.
1104 Compressed data is arranged in the following manner: the first 8 bytes
1105 in the data section is divided into a series of 1-byte command
1106 codes. These codes have meanings as described below:
1110 Ignored. If the program writing the system file accumulates compressed
1111 data in blocks of fixed length, 0 bytes can be used to pad out extra
1112 bytes remaining at the end of a fixed-size block.
1116 value @var{code} - @var{bias}, where
1117 @var{code} is the value of the compression code and @var{bias} is the
1118 variable @code{bias} from the file header. For example,
1119 code 105 with bias 100.0 (the normal value) indicates a numeric variable
1121 One file has been seen written by SPSS 14 that contained such a code
1122 in a @emph{string} field with the value 0 (after the bias is
1123 subtracted) as a way of encoding null bytes.
1126 End of file. This code may or may not appear at the end of the data
1127 stream. PSPP always outputs this code but its use is not required.
1130 A numeric or string value that is not
1131 compressible. The value is stored in the 8 bytes following the
1132 current block of command bytes. If this value appears twice in a block
1133 of command bytes, then it indicates the second group of 8 bytes following the
1134 command bytes, and so on.
1137 An 8-byte string value that is all spaces.
1140 The system-missing value.
1143 When the end of the an 8-byte group of command bytes is reached, any
1144 blocks of non-compressible values indicated by code 253 are skipped,
1145 and the next element of command bytes is read and interpreted, until
1146 the end of the file or a code with value 252 is reached.
1147 @setfilename ignored