X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=spv-file-format.texi;h=f9d698706e50acf0ba48e577f2f1647a2737a118;hb=8af10bb39253b97589c5f4b455b708c8fb9e233b;hp=987a06dcf8200fb702dd4bf618780da4ecea472f;hpb=b07cc00facf9d44f590fa9b2ef5b2b6ab9b08911;p=pspp diff --git a/spv-file-format.texi b/spv-file-format.texi index 987a06dcf8..f9d698706e 100644 --- a/spv-file-format.texi +++ b/spv-file-format.texi @@ -93,12 +93,14 @@ require a reader to take schemas or namespaces into account. @table @code @item heading Parent: Document root or @code{heading} @* -Contents: @code{label} [@code{container} | @code{heading}]* +Contents: [@code{pageSetup}] @code{label} [@code{container} | @code{heading}]* The root of a structure member is a @code{heading}, which represents a section of output beginning with a title (the @code{label}) and -ordinarily followed by a container for content and possibly further -nested (sub)-sections of output. +ordinarily followed by content containers or further nested +(sub)-sections of output. + +The document root heading may also contain a @code{pageSetup} element. The following attributes have been observed on both document root and nested @code{heading} elements: @@ -143,13 +145,11 @@ The following attributes have been observed only on nested @table @asis @item Required attribute: @code{commandName} The locale-invariant name of the command that produced the output, -e.g.@: @code{Frequencies} or @code{T-Test}. For output not specific -to a command, this is simply @code{log}. +e.g.@: @code{Frequencies}, @code{T-Test}, @code{Non Par Corr}. @item Optional attribute: @code{visibility} To what degree the output represented by the element is visible. The -possible values are @code{visible}, @code{hidden}, and -@code{collapsed}. +only observed value is @code{collapsed}. @item Optional attribute: @code{locale} The locale used for output, in Windows format, which is similar to the @@ -185,18 +185,85 @@ Contents: @code{label} [@code{table} | @code{text}] A @code{container} serves to label a @code{table} or a @code{text} item. +@table @asis +@item Required attribute: @code{visibility} +Either @code{visible} or @code{hidden}, this indicates whether the +container's content is displayed. + +@item Optional attribute: @code{text-align} +Presumably indicates the alignment of text within the container. The +only observed value is @code{left}. Observed with nested @code{table} +and @code{text} elements. + +@item Optional attribute: @code{width} +The width of the container in the form @code{@var{n}px}, e.g.@: +@code{1097px}. +@end table + @item text Parent: @code{container} @* Contents: @code{html} +This @code{text} element is nested inside a @code{container}. There +is a different @code{text} element that is nested inside a +@code{pageParagraph}. + +@table @asis +@item Required attribute: @code{type} +One of @code{title}, @code{log}, or @code{text}. + +@item Optional attribute: @code{commandName} +As on the @code{heading} element. For output not specific to a +command, this is simply @code{log}. The corpus contains one example +of where @code{commandName} is present but set to the empty string. + +@item Optional attribute: @code{creator-version} +As on the @code{heading} element. +@end table + @item html Parent: @code{text} @* Contents: cdata +The cdata contains an HTML document. In some cases, the document +starts with @code{} and ends with @code{}. The actual content ranges from trivial to simple: +just discarding the CSS and tags yields readable results. + +@table @asis +@item Required attribute: @code{lang} +This always contains @code{en} in the corpus. +@end table + @item table Parent: @code{container} @* Contents: @code{tableStructure} +@table @asis +@item Required attribute: @code{commandName} +As on the @code{heading} element. + +@item Required attribute: @code{type} +One of @code{table}, @code{note}, or @code{warning}. + +@item Required attribute: @code{subType} +The locale-invariant name for the particular kind of output that this +table represents in the procedure. This can be the same as +@code{commandName} e.g.@: @code{Frequencies}, or different, e.g.@: +@code{Case Processing Summary}. Generic subtypes @code{Notes} and +@code{Warnings} are often used. + +@item Required attribute: @code{tableId} +A number that uniquely identifies the table within the SPV file, +typically a large negative number such as @code{-4147135649387905023}. + +@item Optional attribute: @code{creator-version} +As on the @code{heading} element. In the corpus, this is only present +for version 21 and up and always includes all 8 digits. +@end table + @item tableStructure Parent: @code{table} Contents: @code{dataPath} @@ -204,4 +271,364 @@ Contents: @code{dataPath} @item dataPath Parent: @code{tableStructure} Contents: text + +Contains the name of the Zip member that holds the table details, +e.g.@: @code{0000000001437_lightTableData.bin}. + +@item pageSetup +Parent: @code{heading} @* +Contents: @code{pageHeader} @code{pageFooter} + +@table @asis +@item Required attribute: @code{initial-page-number} +Always @code{1}. + +@item Optional attribute: @code{chart-size} +Always @code{as-is} or a localization (!) of it (e.g.@: @code{dimensione +attuale}, @code{Wie vorgegeben}). + +@item Optional attribute: @code{margin-left} +@itemx Optional attribute: @code{margin-right} +@itemx Optional attribute: @code{margin-top} +@itemx Optional attribute: @code{margin-bottom} +Margin sizes in the form @code{@var{size}in}, e.g.@: @code{0.25in}. + +@item Optional attribute: @code{paper-height} +@itemx Optional attribute: @code{paper-width} +Paper sizes in the form @code{@var{size}in}, e.g.@: @code{8.5in} by +@code{11in} for letter paper or @code{8.267in} by @code{11.692in} for +A4 paper. + +@item Optional attribute: @code{reference-orientation} +Always @code{0deg}. + +@item Optional attribute: @code{space-after} +Always @code{12pt}. +@end table + +@item pageHeader +@itemx pageFooter +Parent: @code{pageSetup} @* +Contents: @code{pageParagraph}* + +No attributes. + +@item pageParagraph +Parent: @code{pageHeader} or @code{pageFooter} @* +Contents: @code{text} + +Text to go at the top or bottom of a page, respectively. + +@item text +Parent: @code{pageParagraph} @* +Contents: [cdata] + +This @code{text} element is nested inside a @code{pageParagraph}. There +is a different @code{text} element that is nested inside a +@code{container}. + +The element is either empty, or contains cdata that holds almost-XHTML +text: in the corpus, either an @code{html} or @code{p} element. It is +@emph{almost}-XHTML because the @code{html} element designates the +default namespace as +@code{http://xml.spss.com/spss/viewer/viewer-tree} instead of an XHTML +namespace. + +The cdata can contain substitution variables: @code{&[Page]} for the +page number and @code{&[PageTitle]} for the page title. + +Typical contents (indented for clarity): + +@example + + + +

Page &[Page]

+ + +@end example + +@table @asis +@item Required attribute: @code{type} +Always @code{text}. @end table +@end table + +@node SPV Light Detail Member Format +@subsection Light Detail Member Format + +A ``light'' detail member @file{.bin} consists of a number of sections +concatenated together, terminated by a byte 01: + +@example +light-member := header title styles dimensions data 01 +@end example + +The first section is a 0x27-byte header: + +@example +header := 01 00 version 01 (00 | 01) byte*21 00 00 table-id byte*4 +version := i1 | i3 +table-id := int +@end example + +@code{header} includes @code{version}, a version number that affects +the interpretation of some of the other data in the member. We will +refer to ``version 1'' and ``version 3'' members later on. It also +@code{table-id} is a binary version of @code{tableId} attribute in the +structure member that refers to the detail member. For example, if +@code{tableId} is @code{-4154297861994971133}, then @code{table-id} +would be 0xdca00003. The meaning of the other variable parts of the +header is not known. + +@example +title := value 01? /* @r{localized title} */ + value 01? 31 /* @r{subtype} */ + value 01? 00? 58 /* @r{locale-invariant title} */ + (31 value | 58) /* @r{caption} */ + int[n] footnote*[n] /* @r{footnotes} */ +footnote := value (31 value | 58) byte*4 +@end example + +@example +styles := 00 font*8 + int[x1] byte*[x1] + int[x2] byte*[x2] + int[x3] byte*[x3] + int[x4] int*[x4] + string[encoding] + (i0 | i-1) (00 | 01) 00 (00 | 01) + int + byte[decimal] byte[grouping] + int[x5] string*[x5] /* @r{custom currency} */ + int[x6] byte*[x6] +@end example + +In every example in the corpus, @code{x1} is 240. The meaning of the +bytes that follow it is unknown. + +In every example in the corpus, @code{x2} is 18 and the bytes that +follow it are @code{00 00 00 01 00 00 00 00 00 00 00 00 00 02 00 00 00 +00}. The meaning of these bytes is unknown. + +In every example in the corpus for version 1, @code{x3} is 16 and the +bytes that follow it are @code{00 00 00 01 00 00 00 01 00 00 00 00 01 +01 01 01}. In version 3, observed @code{x3} varies from 117 to 150 and +the bytes that follow it vary somewhat and often include a readable +text string, e.g. ``Default'' or ``Academic'', which appears to be the +name of a ``TableLook''. + +Observed values of @code{x4} vary from 0 to 17. Out of 7060 examples +in the corpus, it is nonzero only 36 times. + +@code{encoding} is a character encoding, usually a Windows code page +such as @code{en_US.windows-1252} or @code{it_IT.windows-1252}. The +encoding string is itself encoded in US-ASCII. The rest of the +character strings in the file use this encoding. + +@code{decimal} is the decimal point character. The observed values +are @samp{.} and @samp{,}. + +@code{grouping} is the grouping character. The observed values are +@samp{,}, @samp{.}, @samp{'}, @samp{ }, and zero (presumably +indicating that digits should not be grouped). + +@code{x5} is observed as either 0 or 5. When it is 5, the following +strings are CCA through CCE format strings. Most commonly these are +all @code{-,,,} but other strings occur. + +@example +font := byte[index] 31 string[typeface] + 00 00 + (10 | 20 | 40 | 50 | 70 | 80)[f1] + 41 + (i0 | i1 | i2)[f2] + 00 + (i0 | i2 | i64173)[f3] + (i0 | i1 | i2 | i3)[f4] + string[fgcolor] string[bgcolor] + i0 i0 00 + (v3: int[f5] int[f6] int[f7] int[f8]) +@end example + +Each @code{font}, in order, represents the font style for a different +element: title, caption, footnote, row labels, column labels, corner +labels, data, and layers. + +@code{index} is the 1-based index of the @code{font}, i.e. 1 for the +first @code{font}, through 8 for the final @code{font}. + +@code{typeface} is the string name of the font. In the corpus, this +is @code{SansSerif} in over 99% of instances and @code{Times New +Roman} in the rest. + +@code{fgcolor} and @code{bgcolor} are the foreground color and +background color, respectively. In the corpus, these are always +@code{#000000} and @code{#ffffff}, respectively. + +The meaning of the remaining data is unknown. It seems likely to +include font sizes, horizontal and vertical alignment, attributes such +as bold or italic, and margins. @code{f1} is @code{40} most of the +time. @code{f2} is @code{i1} most of the time for the title and +@code{i0} most of the time for other fonts. + +The table below lists the values observed in the corpus. When a cell +contains a single value, then 99+% of the corpus contains that value. +When a cell contains a pair of values, then the first value is seen in +about two-third of the corpus and the second value in about the +remaining one-third. In fonts that include multiple pairs, values are +correlated, that is, for font 3, f5 = 24, f6 = 24, f7 = 2 appears +about two-thirds of the time, as does the combination of f4 = 0, f6 = +10 for font 7. + +@example +font f1 f2 f3 f4 f5 f6 f7 f8 + + 1 40 1 0 0 8 10/11 1 8 + 2 40 0 2 1 8 10/11 1 1 + 3 40 0 2 1 24/11 24/ 8 2/3 4 + 4 40 0 2 3 8 10/11 1 1 + 5 40 0 0 1 8 10/11 1 4 + 6 40 0 2 1 8 10/11 1 4 + 7 40 0 64173 0/1 8 10/11 1 1 + 8 40 0 2 3 8 10/11 1 4 +@end example + +@example +dimensions := int[n-dims] dimension*[n-dims] +dimension := value[name] + byte[d1] + (00 | 01 | 02)[d2] + (i0 | i2)[d3] + (00 | 01)[d4] + (00 | 01)[d5] + 01 + int[d6] + int[n-categories] category*[n-categories] +@end example + +@code{name} is the name of the dimension, e.g. @code{Variables}, +@code{Statistics}, or a variable name. + +@code{d1} is usually 0 but many other values have been observed. + +@code{d3} is 2 over 99% of the time. + +@code{d5} is 0 over 99% of the time. + +@code{d6} is either -1 or the 0-based index of the dimension, e.g.@: 0 +for the first dimension, 1 for the second, and so on. The latter is +the case 98% of the time in the corpus. + +@example +category := value[name] (terminal | group) +terminal-category := 00 00 00 i2 int[index] i0 +@end example + +@code{name} is the name of the category (or group). + +@code{category} can represent a terminal category. In that case, +@code{index} is a nonnegative integer less than @code{n-categories} in +the @code{dimension} in which the @code{category} is nested (directly +or indirectly). + +Alternatively, @code{category} can represent a @code{group} of nested +categories: + +@example +group := (00 | 01)[merge] 00 01 (i0 | i2)[data] + i-1 int[n-subcategories] category*[n-subcategories] +@end example + +Ordinarily a group has some nested content, so that +@code{n-subcategories} is positive, but a few instances of groups with +@code{n-subcategories} 0 has been observed. + +If @code{merge} is 00, the most common value, then the group is really +a distinct group that should be represented as such in the visual +representation and user interface. If @code{merge} is 01, however, +the categories in this group should be shown and treated as if they +were direct children of the group's parent group (or if it has no +parent group, then direct children of the dimension), and this group's +name is irrelevant and should not be displayed. (Merged groups can be +nested!) + +@code{data} appears to be i2 when all of the categories within a group +are terminal categories that directly represent data values for a +variable (e.g. in a frequency table or crosstabulation, a group of +values in a variable being tabulated) and i0 otherwise, but this might +be naive. + +@example +data := int[layers] int[rows] int[columns] int*[n-dimensions] + int[n-data] datum*[n-data] +@end example + +The values of @code{layers}, @code{rows}, and @code{columns} each +specifies the number of dimensions represented in layers or rows or +columns, respectively, and their values sum to the number of +dimensions. + +The @code{n-dimensions} integers are a permutation of the 0-based +dimension numbers. The first @code{layers} of them specify each of +the dimensions represented by layers, the next @code{rows} of them +specify the dimensions represented by rows, and the final +@code{columns} of them specify the dimensions represented by columns. +When there is more than one dimension of a given kind, the inner +dimensions are given first. + +@example +datum := int64[index] 00? value /* @r{version 1} */ +datum := int64[index] value /* @r{version 3} */ +@end example + +The format of a datum varies slightly from version 1 to version 3: in +version 1 it allows for an extra optional 00 byte. + +A datum consists of an index and a value. Suppose there are @math{d} +dimensions and dimension @math{i} for @math{0 \le i < d} has +@math{n_i} categories. Consider the datum at coordinates @math{x_i} +for @math{0 \le i < d}; note that @math{0 \le x_i < n_i}. Then the +index is calculated by the following algorithm: + +@display +let index = 0 +for each @math{i} from 0 to @math{d - 1}: + index = @math{n_i \times} index + @math{x_i} +@end display + +For example, suppose there are 3 dimensions with 3, 4, and 5 +categories, respectively. The datum at coordinates (1, 2, 3) has +index @math{5 \times (4 \times (3 \times 0 + 1) + 2) + 3 = 33}. + +@example +value := 00? 00? 00? 00? raw-value +raw-value := 01 opt-value int32[format] double + | 02 opt-value int32[format] double string[varname] string[vallab] + (01 | 02 | 03) + | 03 string[local] opt-value string[id] string[c] (00 | 01) + | 04 opt-value int32[format] string[vallab] string[varname] + (01 | 02 | 03) string[vallab] + | 05 opt-value string[varname] string[varlabel] (01 | 02 | 03) + | opt-value string[format] int32[n-substs] substitution*[n-substs] +substitution := i0 value + | int32[x] value*[x + 1] /* @r{x > 0} */ +opt-value := 31 i0 (i0 | i1 string) opt-value-i0-v1 /* @r{version 1} */ + | 31 i0 (i0 | i1 string) opt-value-i0-v3 /* @r{version 3} */ + | 31 i1 int32[footnote-number] nested-string + | 31 i2 (00 | 02) 00 (i1 | i2 | i3) nested-string + | 31 i3 00 00 01 00 i2 nested-string + | 58 +opt-value-i0-v1 := 00 (i1 | i2) 00 00 int32 00 00 +opt-value-i0-v3 := count(counted-string + (58 + | 31 01? 00? 00? 00? 01 + string[fgcolor] string[bgcolor] string[typeface] + byte) + (58 + | 31 i0 i0 i0 i0 01 00 (01 | 02 | 08) + 00 08 00 0a 00)) + +nested-string := 00 00 count(counted-string 58 58) +counted-string := count((i0 (58 | 31 string))?) +@end example