X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=spv-file-format.texi;h=0b9ad8cf6e7c076f5db984f908738581043668e9;hb=00357578f9bdbe00e5237bdb82dc80ec2214cbef;hp=c1d5d0712393cc301562180c60ac43dfbc6ff6a7;hpb=0ed91de68e9149be640b996c21c8121296ded298;p=pspp diff --git a/spv-file-format.texi b/spv-file-format.texi index c1d5d07123..0b9ad8cf6e 100644 --- a/spv-file-format.texi +++ b/spv-file-format.texi @@ -9,7 +9,7 @@ write them. An an aside, SPSS 15 and earlier versions use a completely different output format based on the Microsoft Compound Document Format. This -format is not documented. +format is not documented here. An SPV file is a Zip archive that can be read with @command{zipinfo} and @command{unzip} and similar programs. The final member in the Zip @@ -83,9 +83,9 @@ that are commonly found in the corpus. Structure members use a different XML namespace for each schema, but these namespaces are not entirely consistent: in some SPV files, for example, the @code{viewer-tree} schema is associated with namespace -@indicateurl{http://xml.spss.com/spss/viewer-tree} and in other with +@indicateurl{http://xml.spss.com/spss/viewer-tree} and in others with @indicateurl{http://xml.spss.com/spss/viewer/viewer-tree} (note the -additional @file{viewer/} directory. In any case, the schema URIs are +additional @file{viewer/}). In any case, the schema URIs are not resolvable to obtain the schemas themselves. One may ignore all of the above in interpreting a structure member. @@ -174,7 +174,7 @@ describes what it labels, often by naming the statistical procedure that was executed, e.g.@: ``Frequencies'' or ``T-Test''. Labels are often very generic, especially within a @code{container}, e.g.@: ``Title'' or ``Warnings'' or ``Notes''. Label text is localized -according to the output language, e.g. in Italian a frequency table +according to the output language, e.g.@: in Italian a frequency table procedure is labeled ``Frequenze''. The corpus contains one example of an empty label, one that contains @@ -622,17 +622,17 @@ index @math{5 \times (4 \times (3 \times 0 + 1) + 2) + 3 = 33}. @example value := 00? 00? 00? 00? raw-value raw-value := - 01 value-mod int32[format] double[x] - | 02 value-mod int32[format] double[x] + 01 value-mod int[format] double[x] + | 02 value-mod int[format] double[x] string[varname] string[vallab] (01 | 02 | 03) | 03 string[local] value-mod string[id] string[c] (00 | 01)[type] - | 04 value-mod int32[format] string[vallab] string[varname] + | 04 value-mod int[format] string[vallab] string[varname] (01 | 02 | 03) string[s] | 05 value-mod string[varname] string[varlabel] (01 | 02 | 03) - | value-mod string[format] int32[n-args] arg*[n-args] + | value-mod string[format] int[n-args] arg*[n-args] arg := i0 value - | int32[x] i0 value*[x + 1] /* @r{x > 0} */ + | int[x] i0 value*[x + 1] /* @r{x > 0} */ @end example A @code{value} boils down to a number or a string. There are several @@ -774,11 +774,11 @@ The format string is localized to the user's locale. value-mod := 31 i0 (i0 | i1 string[subscript]) value-mod-i0-v1 /* @r{version 1} */ | 31 i0 (i0 | i1 string[subscript]) value-mod-i0-v3 /* @r{version 3} */ - | 31 i1 int32[footnote-number] format + | 31 i1 int[footnote-number] format | 31 i2 (00 | 01 | 02) 00 (i1 | i2 | i3) format | 31 i3 00 00 01 00 i2 format | 58 -value-mod-i0-v1 := 00 (i1 | i2) 00 00 int32 00 00 +value-mod-i0-v1 := 00 (i1 | i2) 00 00 int 00 00 value-mod-i0-v3 := count(format-string (58 | 31 style) (58 @@ -817,3 +817,103 @@ English-language version of the localized format string in the The @code{style}, if present, changes the style for this individual @code{value}. @end itemize + +@node SPV Legacy Detail Member Binary Format +@subsection SPV Legacy Detail Member Binary Format + +Whereas the light binary format represents everything about a given +pivot table, the legacy binary format conceptually consists of a +number of named sources, each of which consists of a number of named +series, each of which is a 1-dimensional array of numbers or strings +or a mix. Thus, the legacy binary file format is quite simple. + +@example +legacy-binary := 00 byte[version] int16[n-sources] int[file-size] + metadata*[n-sources] data*[n-sources] +@end example + +@code{version} is a version number that affects the interpretation of +some of the other data in the member. Versions 0xaf and 0xb0 are +known. We will refer to ``version 0xaf'' and ``version 0xb0'' members +later on. + +A legacy member consists of @code{n-sources} data sources, each of +which has @code{metadata} and @code{data}. + +@code{file-size} is the size of the file, in bytes. + +@example +/* @r{version 0xaf} */ +metadata := int[per-series] int[n-series] int[ofs] byte*32[source-name] + +/* @r{version 0xb0} */ +metadata := int[per-series] int[n-series] int[ofs] byte*64[source-name] int[x] +@end example + +A data source consists of @code{n-series} series of data, with +@code{per-series} data values per series. + +@code{source-name} is a 32- or 64-byte string padded on the right with +zero bytes. The names that appear in the corpus are very generic, +usually @code{tableData} or @code{source0}. + +The @code{ofs} is the offset, in bytes, from the beginning of the file +to the start of this data source's @code{data}. This allows programs +to skip to the beginning of the data for a particular source; it is +also important to determine whether a source includes any string data +(see below). + +The meaning of @code{x} in version 0xb0 is unknown. + +@example +data := numeric-data string-data? +numeric-data := numeric-series*[n-series] +numeric-series := byte*288[series-name] double*[per-series] +@end example + +Data follow the metadata in the legacy binary format, with sources in +the same order. Each series begins with a @code{series-name}, which +generally indicates its role in the pivot table, e.g.@: ``cell'', +``cellFormat'', ``dimension0categories'', ``dimension0group0''. The +name is followed by the data, one double per element in the series. A +double with the maximum negative double @code{-DBL_MAX} represents the +system-missing value SYSMIS. + +@example +string-data := i1 string[source-name] pairs labels + +pairs := int[n-string-series] pair-series*[n-string-series] +pair-series := string[pair-series-name] int[n-pairs] pair*[n-pairs] +pair := int[i] int[j] + +labels := int[n-labels] label*[n-labels] +label := int[frequency] int[s] +@end example + +A source may include a mix of numeric and string data values. When a +source includes any string data, the data values that are strings are +set to SYSMIS in the @code{numeric-series}, and @code{string-data} +follows the @code{numeric-data}. To reliably determine whether a +source includes @code{string-data}, the reader should check whether +the offset following the @code{numeric-data} is the offset of the next +series, as indicated by its @code{metadata} (or end of file, in the +case of the last source in a file). + +@code{string-data} repeats the name of the source. + +The string data overlays the numeric data. @code{n-string-series} is +the number of series within the source that include string data. More +precisely, it is the 1-based index of the last series in the source +that includes any string data; thus, it would be 4 if there are 5 +series and only the fourth one includes string data. + +Each @code{pair-series} consists a sequence of 0 or more pairs, each +of which maps from a 0-based index within the series @code{i} to a +0-based label index @code{j}. The pair @code{i} = 2, @code{j} = 3, +for example, would mean that the third data value (with value SYSMIS) +is to be replaced by the string of the fourth label. + +The labels themselves follow the pairs. The valuable part of each +label is the string @code{s}. Each label also includes a +@code{frequency} that reports the number of pairs that reference it +(although this is not useful).