1 @node Portable File Format
2 @appendix Portable File Format
4 These days, most computers use the same internal data formats for
5 integer and floating-point data, if one ignores little differences like
6 big- versus little-endian byte ordering. However, occasionally it is
7 necessary to exchange data between systems with incompatible data
8 formats. This is what portable files are designed to do.
10 @strong{Please note:} Although all of the following information is
11 correct, as far as the author has been able to ascertain, it is gleaned
12 from examination of ASCII-formatted portable files only, so some of it
13 may be incorrect in the general case.
16 * Portable File Characters::
17 * Portable File Structure::
18 * Portable File Header::
19 * Version and Date Info Record::
20 * Identification Records::
21 * Variable Count Record::
22 * Case Weight Variable Record::
24 * Value Label Records::
25 * Portable File Document Record::
26 * Portable File Data::
29 @node Portable File Characters
30 @section Portable File Characters
32 Portable files are arranged as a series of lines of 80
33 characters each. Each line is terminated by a carriage-return,
34 line-feed sequence (``new-lines''). New-lines are only used to avoid
35 line length limits imposed by some OSes; they are not meaningful.
37 Most lines in portable files are exactly 80 characters long. The only
38 exception is a line that ends in one or more spaces, in which the
39 spaces may optionally be omitted. Thus, a portable file reader must
40 act as though a line shorter than 80 characters is padded to that
43 The file must be terminated with a @samp{Z} character. In addition, if
44 the final line in the file does not have exactly 80 characters, then it
45 is padded on the right with @samp{Z} characters. (The file contents may
46 be in any character set; the file contains a description of its own
47 character set, as explained in the next section. Therefore, the
48 @samp{Z} character is not necessarily an ASCII @samp{Z}.)
50 For the rest of the description of the portable file format, new-lines
51 and the trailing @samp{Z}s will be ignored, as if they did not exist,
52 because they are not an important part of understanding the file
55 @node Portable File Structure
56 @section Portable File Structure
58 Every portable file consists of the following records, in sequence:
66 Version and date info.
69 Product identification.
72 Author identification (optional).
75 Subproduct identification (optional).
81 Case weight variable (optional).
84 Variables. Each variable record may optionally be followed by a
85 missing value record and a variable label record.
88 Value labels (optional).
97 Most records are identified by a single-character tag code. The file
98 header and version info record do not have a tag.
100 Other than these single-character codes, there are three types of fields
101 in a portable file: floating-point, integer, and string. Floating-point
102 fields have the following format:
107 Zero or more leading spaces.
110 Optional asterisk (@samp{*}), which indicates a missing value. The
111 asterisk must be followed by a single character, generally a period
112 (@samp{.}), but it appears that other characters may also be possible.
113 This completes the specification of a missing value.
116 Optional minus sign (@samp{-}) to indicate a negative number.
119 A whole number, consisting of one or more base-30 digits: @samp{0}
120 through @samp{9} plus capital letters @samp{A} through @samp{T}.
123 Optional fraction, consisting of a radix point (@samp{.}) followed by
124 one or more base-30 digits.
127 Optional exponent, consisting of a plus or minus sign (@samp{+} or
128 @samp{-}) followed by one or more base-30 digits.
131 A forward slash (@samp{/}).
134 Integer fields take a form identical to floating-point fields, but they
135 may not contain a fraction.
137 String fields take the form of a integer field having value @var{n},
138 followed by exactly @var{n} characters, which are the string content.
140 @node Portable File Header
141 @section Portable File Header
143 Every portable file begins with a 464-byte header, consisting of a
144 200-byte collection of vanity splash strings, followed by a 256-byte
145 character set translation table, followed by an 8-byte tag string.
147 The 200-byte segment is divided into five 40-byte sections, each of
148 which represents the string @code{@var{charset} SPSS PORT FILE} in a
149 different character set encoding, where @var{charset} is the name of
150 the character set used in the file, e.g.@: @code{ASCII} or
151 @code{EBCDIC}. Each string is padded on the right with spaces in its
152 respective character set.
154 It appears that these strings exist only to inform those who might view
155 the file on a screen, and that they are not parsed by SPSS products.
156 Thus, they can be safely ignored. For those interested, the strings are
157 supposed to be in the following character sets, in the specified order:
158 EBCDIC, 7-bit ASCII, CDC 6-bit ASCII, 6-bit ASCII, Honeywell 6-bit
161 The 256-byte segment describes a mapping from the character set used in
162 the portable file to an arbitrary character set having characters at the
168 Control characters. Not important enough to describe in full here.
176 Digits @samp{0} through @samp{9}.
180 Capital letters @samp{A} through @samp{Z}.
184 Lowercase letters @samp{a} through @samp{z}.
200 Symbols @code{&[]!$*);^-/}
204 Broken vertical pipe.
208 Symbols @code{,%_>}?@code{`:} @c @code{?} is an inverted question mark
212 British pound symbol.
216 Symbols @code{@@'="}.
220 Less than or equal symbol.
252 Lower left corner box draw.
256 Upper left corner box draw.
260 Greater than or equal symbol.
264 Superscript @samp{0} through @samp{9}.
268 Lower right corner box draw.
272 Upper right corner box draw.
284 Superscript @samp{(}.
288 Superscript @samp{)}.
292 Horizontal dagger (?).
296 Symbols @samp{@{@}\}.
303 Centered dot, or bullet.
310 Symbols that are not defined in a particular character set are set to
311 the same value as symbol 64; i.e., to @samp{0}.
313 The 8-byte tag string consists of the exact characters @code{SPSSPORT}
314 in the portable file's character set, which can be used to verify that
315 the file is indeed a portable file.
317 @node Version and Date Info Record
318 @section Version and Date Info Record
320 This record does not have a tag code. It has the following structure:
324 A single character identifying the file format version. The letter A
325 represents version 0, and so on.
328 An 8-character string field giving the file creation date in the format
332 A 6-character string field giving the file creation time in the format
336 @node Identification Records
337 @section Identification Records
339 The product identification record has tag code @samp{1}. It consists of
340 a single string field giving the name of the product that wrote the
343 The author identification record has tag code @samp{2}. It is
344 optional. If present, it consists of a single string field giving the
345 name of the person who caused the portable file to be written.
347 The subproduct identification record has tag code @samp{3}. It is
348 optional. If present, it consists of a single string field giving
349 additional information on the product that wrote the portable file.
351 @node Variable Count Record
352 @section Variable Count Record
354 The variable count record has tag code @samp{4}. It consists of two
355 integer fields. The first contains the number of variables in the file
356 dictionary. The purpose of the second is unknown; it contains the value
357 161 in all portable files examined so far.
359 @node Case Weight Variable Record
360 @section Case Weight Variable Record
362 The case weight variable record is optional. If it is present, it
363 indicates the variable used for weighting cases; if it is absent,
364 cases are unweighted. It has tag code @samp{6}. It consists of a
365 single string field that names the weighting variable.
367 @node Variable Records
368 @section Variable Records
370 Each variable record represents a single variable. Variable records
371 have tag code @samp{7}. They have the following structure:
376 Width (integer). This is 0 for a numeric variable, and a number between 1
377 and 255 for a string variable.
380 Name (string). 1--8 characters long. Must be in all capitals.
382 A few portable files that contain duplicate variable names have been
383 spotted in the wild. PSPP handles these by renaming the duplicates
384 with numeric extensions: @code{@var{var}_1}, @code{@var{var}_2}, and
388 Print format. This is a set of three integer fields:
393 Format type (@pxref{Variable Record}).
399 Number of decimal places. 1--40.
402 A few portable files with invalid format types or formats that are not
403 of the appropriate width for their variables have been spotted in the
404 wild. PSPP assigns a default F or A format to a variable with an
408 Write format. Same structure as the print format described above.
411 Each variable record can optionally be followed by a missing value
412 record, which has tag code @samp{8}. A missing value record has one
413 field, the missing value itself (a floating-point or string, as
414 appropriate). Up to three of these missing value records can be used.
416 There is also a record for missing value ranges, which has tag code
417 @samp{B}. It is followed by two fields representing the range, which
418 are floating-point or string as appropriate. If a missing value range
419 is present, it may be followed by a single missing value record.
421 Tag codes @samp{9} and @samp{A} represent @code{LO THRU @var{x}} and
422 @code{@var{x} THRU HI} ranges, respectively. Each is followed by a
423 single field representing @var{x}. If one of the ranges is present, it
424 may be followed by a single missing value record.
426 In addition, each variable record can optionally be followed by a
427 variable label record, which has tag code @samp{C}. A variable label
428 record has one field, the variable label itself (string).
430 @node Value Label Records
431 @section Value Label Records
433 Value label records have tag code @samp{D}. They have the following
438 Variable count (integer).
441 List of variables (strings). The variable count specifies the number in
442 the list. Variables are specified by their names. All variables must
443 be of the same type (numeric or string), but string variables do not
444 necessarily have the same width.
447 Label count (integer).
450 List of (value, label) tuples. The label count specifies the number of
451 tuples. Each tuple consists of a value, which is numeric or string as
452 appropriate to the variables, followed by a label (string).
455 A few portable files that specify duplicate value labels, that is, two
456 different labels for a single value of a single variable, have been
457 spotted in the wild. PSPP uses the last value label specified in
460 @node Portable File Document Record
461 @section Document Record
463 One document record may optionally follow the value label record. The
464 document record consists of tag code @samp{E}, following by the number
465 of document lines as an integer, followed by that number of strings,
466 each of which represents one document line. Document lines must be 80
467 bytes long or shorter.
469 @node Portable File Data
470 @section Portable File Data
472 The data record has tag code @samp{F}. There is only one tag for all
473 the data; thus, all the data must follow the dictionary. The data is
474 terminated by the end-of-file marker @samp{Z}, which is not valid as the
475 beginning of a data element.
477 Data elements are output in the same order as the variable records
478 describing them. String variables are output as string fields, and
479 numeric variables are output as floating-point fields.