1 @node System File Format
2 @appendix System File Format
4 A system file encapsulates a set of cases and dictionary information
5 that describes how they may be interpreted. This chapter describes
6 the format of a system file.
8 System files use four data types: 8-bit characters, 32-bit integers,
10 and 64-bit floating points, called here @code{char}, @code{int32},
12 @code{flt64}, respectively. Data is not necessarily aligned on a word
13 or double-word boundary: the long variable name record (@pxref{Long
14 Variable Names Record}) and very long string records (@pxref{Very Long
15 String Record}) have arbitrary byte length and can therefore cause all
16 data coming after them in the file to be misaligned.
18 Integer data in system files may be big-endian or little-endian. A
19 reader may detect the endianness of a system file by examining
20 @code{layout_code} in the file header record
21 (@pxref{layout_code,,@code{layout_code}}).
23 Floating-point data in system files may nominally be in IEEE 754, IBM,
24 or VAX formats. A reader may detect the floating-point format in use
25 by examining @code{bias} in the file header record
26 (@pxref{bias,,@code{bias}}).
28 PSPP detects big-endian and little-endian integer formats in system
29 files and translates as necessary. PSPP also detects the
30 floating-point format in use, as well as the endianness of IEEE 754
31 floating-point numbers, and translates as needed. However, only IEEE
32 754 numbers with the same endianness as integer data in the same file
33 has actually been observed in system files, and it is likely that
34 other formats are obsolete or were never used.
36 The PSPP system-missing value is represented by the largest possible
37 negative number in the floating point format (@code{-DBL_MAX}). Two
38 other values are important for use as missing values: @code{HIGHEST},
39 represented by the largest possible positive number (@code{DBL_MAX}),
40 and @code{LOWEST}, represented by the second-largest negative number
41 (in IEEE 754 format, @code{0xffeffffffffffffe}).
43 System files are divided into records, each of which begins with a
44 4-byte record type, usually regarded as an @code{int32}.
46 The records must appear in the following order:
56 All pairs of value labels records and value label variables records,
60 Document record, if present.
63 Extension (type 7) records, in ascending numerical order of their
67 Dictionary termination record.
73 Each type of record is described separately below.
76 * File Header Record::
78 * Value Labels Records::
80 * Machine Integer Info Record::
81 * Machine Floating-Point Info Record::
82 * Multiple Response Sets Records::
83 * Variable Display Parameter Record::
84 * Long Variable Names Record::
85 * Very Long String Record::
86 * Character Encoding Record::
87 * Long String Value Labels Record::
88 * Data File and Variable Attributes Records::
89 * Extended Number of Cases Record::
90 * Miscellaneous Informational Records::
91 * Dictionary Termination Record::
95 @node File Header Record
96 @section File Header Record
98 The file header is always the first record in the file. It has the
105 int32 nominal_case_size;
110 char creation_date[9];
111 char creation_time[8];
117 @item char rec_type[4];
118 Record type code, set to @samp{$FL2}.
120 @item char prod_name[60];
121 Product identification string. This always begins with the characters
122 @samp{@@(#) SPSS DATA FILE}. PSPP uses the remaining characters to
123 give its version and the operating system name; for example, @samp{GNU
124 pspp 0.1.4 - sparc-sun-solaris2.5.2}. The string is truncated if it
125 would be longer than 60 characters; otherwise it is padded on the right
129 @item int32 layout_code;
130 Normally set to 2, although a few system files have been spotted in
131 the wild with a value of 3 here. PSPP use this value to determine the
132 file's integer endianness (@pxref{System File Format}).
134 @item int32 nominal_case_size;
135 Number of data elements per case. This is the number of variables,
136 except that long string variables add extra data elements (one for every
137 8 characters after the first 8). However, string variables do not
138 contribute to this value beyond the first 255 bytes. Further, system
139 files written by some systems set this value to -1. In general, it is
140 unsafe for systems reading system files to rely upon this value.
142 @item int32 compressed;
143 Set to 1 if the data in the file is compressed, 0 otherwise.
145 @item int32 weight_index;
146 If one of the variables in the data set is used as a weighting
147 variable, set to the dictionary index of that variable, plus 1
148 (@pxref{Dictionary Index}). Otherwise, set to 0.
151 Set to the number of cases in the file if it is known, or -1 otherwise.
153 In the general case it is not possible to determine the number of cases
154 that will be output to a system file at the time that the header is
155 written. The way that this is dealt with is by writing the entire
156 system file, including the header, then seeking back to the beginning of
157 the file and writing just the @code{ncases} field. For files in which
158 this is not valid, the seek operation fails. In this case,
159 @code{ncases} remains -1.
163 Compression bias, ordinarily set to 100. Only integers between
164 @code{1 - bias} and @code{251 - bias} can be compressed.
166 By assuming that its value is 100, PSPP uses @code{bias} to determine
167 the file's floating-point format and endianness (@pxref{System File
168 Format}). If the compression bias is not 100, PSPP cannot auto-detect
169 the floating-point format and assumes that it is IEEE 754 format with
170 the same endianness as the system file's integers, which is correct
171 for all known system files.
173 @item char creation_date[9];
174 Date of creation of the system file, in @samp{dd mmm yy}
175 format, with the month as standard English abbreviations, using an
176 initial capital letter and following with lowercase. If the date is not
177 available then this field is arbitrarily set to @samp{01 Jan 70}.
179 @item char creation_time[8];
180 Time of creation of the system file, in @samp{hh:mm:ss}
181 format and using 24-hour time. If the time is not available then this
182 field is arbitrarily set to @samp{00:00:00}.
184 @item char file_label[64];
185 File label declared by the user, if any (@pxref{FILE LABEL,,,pspp,
186 PSPP Users Guide}). Padded on the right with spaces.
188 @item char padding[3];
189 Ignored padding bytes to make the structure a multiple of 32 bits in
190 length. Set to zeros.
193 @node Variable Record
194 @section Variable Record
196 There must be one variable record for each numeric variable and each
197 string variable with width 8 bytes or less. String variables wider
198 than 8 bytes have one variable record for each 8 bytes, rounding up.
199 The first variable record for a long string specifies the variable's
200 correct dictionary information. Subsequent variable records for a
201 long string are filled with dummy information: a type of -1, no
202 variable label or missing values, print and write formats that are
203 ignored, and an empty string as name. A few system files have been
204 encountered that include a variable label on dummy variable records,
205 so readers should take care to parse dummy variable records in the
206 same way as other variable records.
208 @anchor{Dictionary Index}
209 The @dfn{dictionary index} of a variable is its offset in the set of
210 variable records, including dummy variable records for long string
211 variables. The first variable record has a dictionary index of 0, the
212 second has a dictionary index of 1, and so on.
214 The system file format does not directly support string variables
215 wider than 255 bytes. Such very long string variables are represented
216 by a number of narrower string variables. @xref{Very Long String
217 Record}, for details.
223 int32 n_missing_values;
228 /* @r{Present only if @code{has_var_label} is 1.} */
232 /* @r{Present only if @code{n_missing_values} is nonzero}. */
233 flt64 missing_values[];
237 @item int32 rec_type;
238 Record type code. Always set to 2.
241 Variable type code. Set to 0 for a numeric variable. For a short
242 string variable or the first part of a long string variable, this is set
243 to the width of the string. For the second and subsequent parts of a
244 long string variable, set to -1, and the remaining fields in the
245 structure are ignored.
247 @item int32 has_var_label;
248 If this variable has a variable label, set to 1; otherwise, set to 0.
250 @item int32 n_missing_values;
251 If the variable has no missing values, set to 0. If the variable has
252 one, two, or three discrete missing values, set to 1, 2, or 3,
253 respectively. If the variable has a range for missing variables, set to
254 -2; if the variable has a range for missing variables plus a single
255 discrete value, set to -3.
258 Print format for this variable. See below.
261 Write format for this variable. See below.
264 Variable name. The variable name must begin with a capital letter or
265 the at-sign (@samp{@@}). Subsequent characters may also be digits, octothorpes
266 (@samp{#}), dollar signs (@samp{$}), underscores (@samp{_}), or full
267 stops (@samp{.}). The variable name is padded on the right with spaces.
269 @item int32 label_len;
270 This field is present only if @code{has_var_label} is set to 1. It is
271 set to the length, in characters, of the variable label. The
272 documented maximum length varies from 120 to 255 based on SPSS
273 version, but some files have been seen with longer labels. PSPP
274 accepts longer labels and truncates them to 255 bytes on input.
277 This field is present only if @code{has_var_label} is set to 1. It has
278 length @code{label_len}, rounded up to the nearest multiple of 32 bits.
279 The first @code{label_len} characters are the variable's variable label.
281 @item flt64 missing_values[];
282 This field is present only if @code{n_missing_values} is nonzero. It
283 has the same number of 8-byte elements as the absolute value of
284 @code{n_missing_values}. Each element is interpreted as a number for
285 numeric variables (with HIGHEST and LOWEST indicated as described in
286 the chapter introduction). For string variables of width less than 8
287 bytes, elements are right-padded with spaces; for string variables
288 wider than 8 bytes, only the first 8 bytes of each missing value are
289 specified, with the remainder implicitly all spaces.
291 For discrete missing values, each element represents one missing
292 value. When a range is present, the first element denotes the minimum
293 value in the range, and the second element denotes the maximum value
294 in the range. When a range plus a value are present, the third
295 element denotes the additional discrete missing value.
298 The @code{print} and @code{write} members of sysfile_variable are output
299 formats coded into @code{int32} types. The least-significant byte
300 of the @code{int32} represents the number of decimal places, and the
301 next two bytes in order of increasing significance represent field width
302 and format type, respectively. The most-significant byte is not
303 used and should be set to zero.
305 Format types are defined as follows:
308 @multitable {Value} {@code{DATETIME}}
394 A few system files have been observed in the wild with invalid
395 @code{write} fields, in particular with value 0. Readers should
396 probably treat invalid @code{print} or @code{write} fields as some
399 @node Value Labels Records
400 @section Value Labels Records
402 The value label records documented in this section are used for
403 numeric and short string variables only. Long string variables may
404 have value labels, but their value labels are recorded using a
405 different record type (@pxref{Long String Value Labels Record}).
407 The value label record has the following format:
413 /* @r{Repeated @code{label_cnt} times}. */
420 @item int32 rec_type;
421 Record type. Always set to 3.
423 @item int32 label_count;
424 Number of value labels present in this record.
427 The remaining fields are repeated @code{count} times. Each
428 repetition specifies one value label.
432 A numeric value or a short string value padded as necessary to 8 bytes
433 in length. Its type and width cannot be determined until the
434 following value label variables record (see below) is read.
436 @item char label_len;
437 The label's length, in bytes. The documented maximum length varies
438 from 60 to 120 based on SPSS version. PSPP supports value labels up
442 @code{label_len} bytes of the actual label, followed by up to 7 bytes
443 of padding to bring @code{label} and @code{label_len} together to a
444 multiple of 8 bytes in length.
447 The value label record is always immediately followed by a value label
448 variables record with the following format:
457 @item int32 rec_type;
458 Record type. Always set to 4.
460 @item int32 var_count;
461 Number of variables that the associated value labels from the value
462 label record are to be applied.
465 A list of dictionary indexes of variables to which to apply the value
466 labels (@pxref{Dictionary Index}). There are @code{var_count}
469 String variables wider than 8 bytes may not be specified in this list.
472 @node Document Record
473 @section Document Record
475 The document record, if present, has the following format:
484 @item int32 rec_type;
485 Record type. Always set to 6.
488 Number of lines of documents present.
490 @item char lines[][80];
491 Document lines. The number of elements is defined by @code{n_lines}.
492 Lines shorter than 80 characters are padded on the right with spaces.
495 @node Machine Integer Info Record
496 @section Machine Integer Info Record
498 The integer info record, if present, has the following format:
510 int32 version_revision;
512 int32 floating_point_rep;
513 int32 compression_code;
515 int32 character_code;
519 @item int32 rec_type;
520 Record type. Always set to 7.
523 Record subtype. Always set to 3.
526 Size of each piece of data in the data part, in bytes. Always set to 4.
529 Number of pieces of data in the data part. Always set to 8.
531 @item int32 version_major;
532 PSPP major version number. In version @var{x}.@var{y}.@var{z}, this
535 @item int32 version_minor;
536 PSPP minor version number. In version @var{x}.@var{y}.@var{z}, this
539 @item int32 version_revision;
540 PSPP version revision number. In version @var{x}.@var{y}.@var{z},
543 @item int32 machine_code;
544 Machine code. PSPP always set this field to value to -1, but other
547 @item int32 floating_point_rep;
548 Floating point representation code. For IEEE 754 systems this is 1.
549 IBM 370 sets this to 2, and DEC VAX E to 3.
551 @item int32 compression_code;
552 Compression code. Always set to 1.
554 @item int32 endianness;
555 Machine endianness. 1 indicates big-endian, 2 indicates little-endian.
557 @item int32 character_code;
558 @anchor{character-code} Character code. The following values have
559 been actually observed in system files:
566 The @code{windows-1250} code page for Central European and Eastern
570 The @code{windows-1252} code page for Western European languages.
579 The following additional values are known to be defined:
592 Other Windows code page numbers are known to be generally valid.
594 Old versions of SPSS always wrote value 2 in this field, regardless of
595 the encoding in use. Newer versions also write the character encoding
596 as a string (see @ref{Character Encoding Record}).
599 @node Machine Floating-Point Info Record
600 @section Machine Floating-Point Info Record
602 The floating-point info record, if present, has the following format:
618 @item int32 rec_type;
619 Record type. Always set to 7.
622 Record subtype. Always set to 4.
625 Size of each piece of data in the data part, in bytes. Always set to 8.
628 Number of pieces of data in the data part. Always set to 3.
631 The system missing value.
634 The value used for HIGHEST in missing values.
637 The value used for LOWEST in missing values.
640 @node Multiple Response Sets Records
641 @section Multiple Response Sets Records
643 The system file format has two different types of records that
644 represent multiple response sets (@pxref{MRSETS,,,pspp, PSPP Users
645 Guide}). The first type of record describes multiple response sets
646 that can be understood by SPSS before version 14. The second type of
647 record, with a closely related format, is used for multiple dichotomy
648 sets that use the CATEGORYLABELS=COUNTEDVALUES feature added in
658 /* @r{Exactly @code{count} bytes of data.} */
663 @item int32 rec_type;
664 Record type. Always set to 7.
667 Record subtype. Set to 7 for records that describe multiple response
668 sets understood by SPSS before version 14, or to 19 for records that
669 describe dichotomy sets that use the CATEGORYLABELS=COUNTEDVALUES
670 feature added in version 14.
673 The size of each element in the @code{mrsets} member. Always set to 1.
676 The total number of bytes in @code{mrsets}.
679 A series of multiple response sets, each of which consists of the
684 The set's name (an identifier that begins with @samp{$}), in mixed
685 upper and lower case.
688 An equals sign (@samp{=}).
691 @samp{C} for a multiple category set, @samp{D} for a multiple
692 dichotomy set with CATEGORYLABELS=VARLABELS, or @samp{E} for a
693 multiple dichotomy set with CATEGORYLABELS=COUNTEDVALUES.
696 For a multiple dichotomy set with CATEGORYLABELS=COUNTEDVALUES, a
697 space, followed by a number expressed as decimal digits, followed by a
698 space. If LABELSOURCE=VARLABEL was specified on MRSETS, then the
699 number is 11; otherwise it is 1.@footnote{This part of the format may
700 not be fully understood, because only a single example of each
701 possibility has been examined.}
704 For either kind of multiple dichotomy set, the counted value, as a
705 positive integer count specified as decimal digits, followed by a
706 space, followed by as many string bytes as specified in the count. If
707 the set contains numeric variables, the string consists of the counted
708 integer value expressed as decimal digits. If the set contains string
709 variables, the string contains the counted string value. Either way,
710 the string may be padded on the right with spaces (older versions of
711 SPSS seem to always pad to a width of 8 bytes; newer versions don't).
717 The multiple response set's label, using the same format as for the
718 counted value for multiple dichotomy sets. A string of length 0 means
719 that the set does not have a label. A string of length 0 is also
720 written if LABELSOURCE=VARLABEL was specified.
726 The short names of the variables in the set, converted to lowercase,
727 each separated from the previous by a single space.
730 A line feed (byte 0x0a).
734 Example: Given appropriate variable definitions, consider the
735 following MRSETS command:
738 MRSETS /MCGROUP NAME=$a LABEL='my mcgroup' VARIABLES=a b c
739 /MDGROUP NAME=$b VARIABLES=g e f d VALUE=55
740 /MDGROUP NAME=$c LABEL='mdgroup #2' VARIABLES=h i j VALUE='Yes'
741 /MDGROUP NAME=$d LABEL='third mdgroup' CATEGORYLABELS=COUNTEDVALUES
742 VARIABLES=k l m VALUE=34
743 /MDGROUP NAME=$e CATEGORYLABELS=COUNTEDVALUES LABELSOURCE=VARLABEL
744 VARIABLES=n o p VALUE='choice'.
747 The above would generate the following multiple response set record of
751 $a=C 10 my mcgroup a b c
753 $c=D3 Yes 10 mdgroup #2 h i j
756 It would also generate the following multiple response set record with
760 $d=E 1 2 34 13 third mdgroup k l m
761 $e=E 11 6 choice 0 n o p
764 @node Variable Display Parameter Record
765 @section Variable Display Parameter Record
767 The variable display parameter record, if present, has the following
777 /* @r{Repeated @code{count} times}. */
779 int32 width; /* @r{Not always present.} */
784 @item int32 rec_type;
785 Record type. Always set to 7.
788 Record subtype. Always set to 11.
791 The size of @code{int32}. Always set to 4.
794 The number of sets of variable display parameters (ordinarily the
795 number of variables in the dictionary), times 2 or 3.
798 The remaining members are repeated @code{count} times, in the same
799 order as the variable records. No element corresponds to variable
800 records that continue long string variables. The meanings of these
801 members are as follows:
805 The measurement type of the variable:
815 SPSS 14 sometimes writes a @code{measure} of 0 for string variables.
816 PSPP interprets this as nominal scale.
819 The width of the display column for the variable in characters.
821 This field is present if @var{count} is 3 times the number of
822 variables in the dictionary. It is omitted if @var{count} is 2 times
823 the number of variables.
825 @item int32 alignment;
826 The alignment of the variable for display purposes:
838 @node Long Variable Names Record
839 @section Long Variable Names Record
841 If present, the long variable names record has the following format:
850 /* @r{Exactly @code{count} bytes of data.} */
851 char var_name_pairs[];
855 @item int32 rec_type;
856 Record type. Always set to 7.
859 Record subtype. Always set to 13.
862 The size of each element in the @code{var_name_pairs} member. Always set to 1.
865 The total number of bytes in @code{var_name_pairs}.
867 @item char var_name_pairs[];
868 A list of @var{key}--@var{value} tuples, where @var{key} is the name
869 of a variable, and @var{value} is its long variable name.
870 The @var{key} field is at most 8 bytes long and must match the
871 name of a variable which appears in the variable record (@pxref{Variable
873 The @var{value} field is at most 64 bytes long.
874 The @var{key} and @var{value} fields are separated by a @samp{=} byte.
875 Each tuple is separated by a byte whose value is 09. There is no
876 trailing separator following the last tuple.
877 The total length is @code{count} bytes.
880 @node Very Long String Record
881 @section Very Long String Record
883 Old versions of SPSS limited string variables to a width of 255 bytes.
884 For backward compatibility with these older versions, the system file
885 format represents a string longer than 255 bytes, called a @dfn{very
886 long string}, as a collection of strings no longer than 255 bytes
887 each. The strings concatenated to make a very long string are called
888 its @dfn{segments}; for consistency, variables other than very long
889 strings are considered to have a single segment.
891 A very long string with a width of @var{w} has @var{n} =
892 (@var{w} + 251) / 252 segments, that is, one segment for every
893 252 bytes of width, rounding up. It would be logical, then, for each
894 of the segments except the last to have a width of 252 and the last
895 segment to have the remainder, but this is not the case. In fact,
896 each segment except the last has a width of 255 bytes. The last
897 segment has width @var{w} - (@var{n} - 1) * 252; some versions
898 of SPSS make it slightly wider, but not wide enough to make the last
899 segment require another 8 bytes of data.
901 Data is packed tightly into segments of a very long string, 255 bytes
902 per segment. Because 255 bytes of segment data are allocated for
903 every 252 bytes of the very long string's width (approximately), some
904 unused space is left over at the end of the allocated segments. Data
905 in unused space is ignored.
907 Example: Consider a very long string of width 20,000. Such a very
908 long string has 20,000 / 252 = 80 (rounding up) segments. The first
909 79 segments have width 255; the last segment has width 20,000 - 79 *
910 252 = 92 or slightly wider (up to 96 bytes, the next multiple of 8).
911 The very long string's data is actually stored in the 19,890 bytes in
912 the first 78 segments, plus the first 110 bytes of the 79th segment
913 (19,890 + 110 = 20,000). The remaining 145 bytes of the 79th segment
914 and all 92 bytes of the 80th segment are unused.
916 The very long string record explains how to stitch together segments
917 to obtain very long string data. For each of the very long string
918 variables in the dictionary, it specifies the name of its first
919 segment's variable and the very long string variable's actual width.
920 The remaining segments immediately follow the named variable in the
921 system file's dictionary.
923 The very long string record, which is present only if the system file
924 contains very long string variables, has the following format:
933 /* @r{Exactly @code{count} bytes of data.} */
934 char string_lengths[];
938 @item int32 rec_type;
939 Record type. Always set to 7.
942 Record subtype. Always set to 14.
945 The size of each element in the @code{string_lengths} member. Always set to 1.
948 The total number of bytes in @code{string_lengths}.
950 @item char string_lengths[];
951 A list of @var{key}--@var{value} tuples, where @var{key} is the name
952 of a variable, and @var{value} is its length.
953 The @var{key} field is at most 8 bytes long and must match the
954 name of a variable which appears in the variable record (@pxref{Variable
956 The @var{value} field is exactly 5 bytes long. It is a zero-padded,
957 ASCII-encoded string that is the length of the variable.
958 The @var{key} and @var{value} fields are separated by a @samp{=} byte.
959 Tuples are delimited by a two-byte sequence @{00, 09@}.
960 After the last tuple, there may be a single byte 00, or @{00, 09@}.
961 The total length is @code{count} bytes.
964 @node Character Encoding Record
965 @section Character Encoding Record
967 This record, if present, indicates the character encoding for string data,
968 long variable names, variable labels, value labels and other strings in the
978 /* @r{Exactly @code{count} bytes of data.} */
983 @item int32 rec_type;
984 Record type. Always set to 7.
987 Record subtype. Always set to 20.
990 The size of each element in the @code{encoding} member. Always set to 1.
993 The total number of bytes in @code{encoding}.
995 @item char encoding[];
996 The name of the character encoding. Normally this will be an official
997 IANA character set name or alias.
998 See @url{http://www.iana.org/assignments/character-sets}.
999 Character set names are not case-sensitive, but SPSS appears to write
1000 them in all-uppercase.
1003 This record is not present in files generated by older software. See
1004 also the @code{character_code} field in the machine integer info
1005 record (@pxref{character-code}).
1007 When the character encoding record and the machine integer info record
1008 are both present, all system files observed in practice indicate the
1009 same character encoding, e.g.@: 1252 as @code{character_code} and
1010 @code{windows-1252} as @code{encoding}, 65001 and @code{UTF-8}, etc.
1012 If, for testing purposes, a file is crafted with different
1013 @code{character_code} and @code{encoding}, it seems that
1014 @code{character_code} controls the encoding for all strings in the
1015 system file before the dictionary termination record, including
1016 strings in data (e.g.@: string missing values), and @code{encoding}
1017 controls the encoding for strings following the dictionary termination
1020 @node Long String Value Labels Record
1021 @section Long String Value Labels Record
1023 This record, if present, specifies value labels for long string
1033 /* @r{Repeated up to exactly @code{count} bytes.} */
1038 long_string_label labels[];
1042 @item int32 rec_type;
1043 Record type. Always set to 7.
1045 @item int32 subtype;
1046 Record subtype. Always set to 21.
1052 The number of bytes following the header until the next header.
1054 @item int32 var_name_len;
1055 @itemx char var_name[];
1056 The number of bytes in the name of the variable that has long string
1057 value labels, plus the variable name itself, which consists of exactly
1058 @code{var_name_len} bytes. The variable name is not padded to any
1059 particular boundary, nor is it null-terminated.
1061 @item int32 var_width;
1062 The width of the variable, in bytes, which will be between 9 and
1065 @item int32 n_labels;
1066 @itemx long_string_label labels[];
1067 The long string labels themselves. The @code{labels} array contains
1068 exactly @code{n_labels} elements, each of which has the following
1079 @item int32 value_len;
1080 @itemx char value[];
1081 The string value being labeled. @code{value_len} is the number of
1082 bytes in @code{value}; it is equal to @code{var_width}. The
1083 @code{value} array is not padded or null-terminated.
1085 @item int32 label_len;
1086 @itemx char label[];
1087 The label for the string value. @code{label_len}, which must be
1088 between 0 and 120, is the number of bytes in @code{label}. The
1089 @code{label} array is not padded or null-terminated.
1093 @node Data File and Variable Attributes Records
1094 @section Data File and Variable Attributes Records
1096 The data file and variable attributes records represent custom
1097 attributes for the system file or for individual variables in the
1098 system file, as defined on the DATAFILE ATTRIBUTE (@pxref{DATAFILE
1099 ATTRIBUTE,,,pspp, PSPP Users Guide}) and VARIABLE ATTRIBUTE commands
1100 (@pxref{VARIABLE ATTRIBUTE,,,pspp, PSPP Users Guide}), respectively.
1109 /* @r{Exactly @code{count} bytes of data.} */
1114 @item int32 rec_type;
1115 Record type. Always set to 7.
1117 @item int32 subtype;
1118 Record subtype. Always set to 17 for a data file attribute record or
1119 to 18 for a variable attributes record.
1122 The size of each element in the @code{attributes} member. Always set to 1.
1125 The total number of bytes in @code{attributes}.
1127 @item char attributes[];
1128 The attributes, in a text-based format.
1130 In record type 17, this field contains a single attribute set. An
1131 attribute set is a sequence of one or more attributes concatenated
1132 together. Each attribute consists of a name, which has the same
1133 syntax as a variable name, followed by, inside parentheses, a sequence
1134 of one or more values. Each value consists of a string enclosed in
1135 single quotes (@code{'}) followed by a line feed (byte 0x0a). A value
1136 may contain single quote characters, which are not themselves escaped
1137 or quoted or required to be present in pairs. There is no apparent
1138 way to embed a line feed in a value. There is no distinction between
1139 an attribute with a single value and an attribute array with one
1142 In record type 18, this field contains a sequence of one or more
1143 variable attribute sets. If more than one variable attribute set is
1144 present, each one after the first is delimited from the previous by
1145 @code{/}. Each variable attribute set consists of a long
1147 followed by @code{:}, followed by an attribute set with the same
1148 syntax as on record type 17.
1150 The total length is @code{count} bytes.
1155 A system file produced with the following VARIABLE ATTRIBUTE commands
1159 VARIABLE ATTRIBUTE VARIABLES=dummy ATTRIBUTE=fred[1]('23') fred[2]('34').
1160 VARIABLE ATTRIBUTE VARIABLES=dummy ATTRIBUTE=bert('123').
1164 will contain a variable attribute record with the following contents:
1167 00000000 07 00 00 00 12 00 00 00 01 00 00 00 22 00 00 00 |............"...|
1168 00000010 64 75 6d 6d 79 3a 66 72 65 64 28 27 32 33 27 0a |dummy:fred('23'.|
1169 00000020 27 33 34 27 0a 29 62 65 72 74 28 27 31 32 33 27 |'34'.)bert('123'|
1170 00000030 0a 29 |.) |
1173 @node Extended Number of Cases Record
1174 @section Extended Number of Cases Record
1176 The file header record expresses the number of cases in the system
1177 file as an int32 (@pxref{File Header Record}). This record allows the
1178 number of cases in the system file to be expressed as a 64-bit number.
1190 @item int32 rec_type;
1191 Record type. Always set to 7.
1193 @item int32 subtype;
1194 Record subtype. Always set to 16.
1197 Size of each element. Always set to 8.
1200 Number of pieces of data in the data part. Alway set to 2.
1202 @item int64 unknown;
1203 Meaning unknown. Always set to 1.
1205 @item int64 ncases64;
1206 Number of cases in the file as a 64-bit integer. Presumably this
1207 could be -1 to indicate that the number of cases is unknown, for the
1208 same reason as @code{ncases} in the file header record, but this has
1209 not been observed in the wild.
1212 @node Miscellaneous Informational Records
1213 @section Miscellaneous Informational Records
1215 Some specific types of miscellaneous informational records are
1216 documented here, but others are known to exist. PSPP ignores unknown
1217 miscellaneous informational records when reading system files.
1226 /* @r{Exactly @code{size * count} bytes of data.} */
1231 @item int32 rec_type;
1232 Record type. Always set to 7.
1234 @item int32 subtype;
1235 Record subtype. May take any value. According to Aapi
1236 H@"am@"al@"ainen, value 5 indicates a set of grouped variables and 6
1237 indicates date info (probably related to USE).
1240 Size of each piece of data in the data part. Should have the value 1,
1241 4, or 8, for @code{char}, @code{int32}, and @code{flt64} format data,
1245 Number of pieces of data in the data part.
1248 Arbitrary data. There must be @code{size} times @code{count} bytes of
1252 @node Dictionary Termination Record
1253 @section Dictionary Termination Record
1255 The dictionary termination record separates all other records from the
1264 @item int32 rec_type;
1265 Record type. Always set to 999.
1268 Ignored padding. Should be set to 0.
1272 @section Data Record
1274 Data records must follow all other records in the system file. There must
1275 be at least one data record in every system file.
1277 The format of data records varies depending on whether the data is
1278 compressed. Regardless, the data is arranged in a series of 8-byte
1281 When data is not compressed,
1282 each element corresponds to
1283 the variable declared in the respective variable record (@pxref{Variable
1284 Record}). Numeric values are given in @code{flt64} format; string
1285 values are literal characters string, padded on the right when
1286 necessary to fill out 8-byte units.
1288 Compressed data is arranged in the following manner: the first 8 bytes
1289 in the data section is divided into a series of 1-byte command
1290 codes. These codes have meanings as described below:
1294 Ignored. If the program writing the system file accumulates compressed
1295 data in blocks of fixed length, 0 bytes can be used to pad out extra
1296 bytes remaining at the end of a fixed-size block.
1300 value @var{code} - @var{bias}, where
1301 @var{code} is the value of the compression code and @var{bias} is the
1302 variable @code{bias} from the file header. For example,
1303 code 105 with bias 100.0 (the normal value) indicates a numeric variable
1305 One file has been seen written by SPSS 14 that contained such a code
1306 in a @emph{string} field with the value 0 (after the bias is
1307 subtracted) as a way of encoding null bytes.
1310 End of file. This code may or may not appear at the end of the data
1311 stream. PSPP always outputs this code but its use is not required.
1314 A numeric or string value that is not
1315 compressible. The value is stored in the 8 bytes following the
1316 current block of command bytes. If this value appears twice in a block
1317 of command bytes, then it indicates the second group of 8 bytes following the
1318 command bytes, and so on.
1321 An 8-byte string value that is all spaces.
1324 The system-missing value.
1327 When the end of the an 8-byte group of command bytes is reached, any
1328 blocks of non-compressible values indicated by code 253 are skipped,
1329 and the next element of command bytes is read and interpreted, until
1330 the end of the file or a code with value 252 is reached.
1331 @setfilename ignored