new-line. PSPP uses this string to identify an SPV file; it is
invariant across the corpus.@footnote{SPV files always begin with the
7-byte sequence 50 4b 03 04 14 00 08, but this is not a useful magic
-number because most Zip archives start the same way.}
+number because most Zip archives start the same way.}@footnote{SPSS
+writes @file{META-INF/MANIFEST.MF} to every SPV file, but it does not
+read it or even require it to exist, so using different contents,
+e.g.@: as @samp{allowingPivot=false} has no effect.}
The rest of the members in an SPV file's Zip archive fall into two
categories: @dfn{structure} and @dfn{detail} members. Structure
The structure of a chart plus its data. Charts do not have a
``light'' format.
+@item @file{@var{prefix}_Imagegeneric.png}
+@itemx @file{@var{prefix}_PastedObjectgeneric.png}
+@itemx @file{@var{prefix}_imageData.bin}
+A PNG image referenced by an @code{object} element (in the first two
+cases) or an @code{image} element (in the final case). @xref{SPV
+Structure object and image Elements}.
+
@item @file{@var{prefix}_pmml.scf}
@itemx @file{@var{prefix}_stats.scf}
@item @file{@var{prefix}_model.xml}
* SPV Structure table Element::
* SPV Structure graph Element::
* SPV Structure model Element::
+* SPV Structure object and image Elements::
* SPV Structure tree Element::
* SPV Structure Path Elements::
* SPV Structure pageSetup Element::
@end example
The root of a structure member is a @code{heading}, which represents a
-section of output beginning with a title (the @code{label}) and
+section of output beginning with a @code{label} and
ordinarily followed by content containers or further nested
(sub)-sections of output. Unlike heading elements in HTML and other
common document formats, which precede the content that they head,
@end example
Every @code{heading} and @code{container} holds a @code{label} as its
-first child. The root @code{heading} in a structure member always
-contains the string ``Output'' (localized). Otherwise, the text in
-@code{label} describes what it labels, often by naming the statistical
-procedure that was executed, e.g.@: ``Frequencies'' or ``T-Test''.
-Labels are often very generic, especially within a @code{container},
-e.g.@: ``Title'' or ``Warnings'' or ``Notes''. Label text is
-localized according to the output language, e.g.@: in Italian a
-frequency table procedure is labeled ``Frequenze''.
-
-The corpus contains a few examples of empty labels, ones that contain
-no text.
+first child. The label text is what appears in the outline pane of
+the GUI's viewer window. PSPP also puts it into the outline of PDF
+output. The label text doesn't appear in the output itself.
+
+The text in @code{label} describes what it labels, often by naming the
+statistical procedure that was executed, e.g.@: ``Frequencies'' or
+``T-Test''. The root @code{heading} in a structure member is normally
+``Output''. Labels are often very generic, especially within a
+@code{container}, e.g.@: ``Title'' or ``Warnings'' or ``Notes''.
+Label text is localized according to the output language, e.g.@: in
+Italian a frequency table procedure is labeled ``Frequenze''.
+
+The user can edit labels to be anything they want. The corpus
+contains a few examples of empty labels, ones that contain no text,
+probably as a result of user editing.
@node SPV Structure container Element
@subsection The @code{container} Element
The CSS in the corpus is simple. To understand it, a parser only
needs to be able to skip white space, @code{<!--}, and @code{-->}, and
-parse style only for @code{p} elements. Only @code{font-weight},
-@code{font-style}, @code{font-decoration}, @code{font-family}, and
-@code{font-size} matter.
+parse style only for @code{p} elements. Only the following properties
+matter:
+
+@table @code
+@item color
+In the form @code{@var{rr}@var{gg}@var{bb}}, e.g. @code{000000}, with
+no leading @samp{#}.
+
+@item font-weight
+Either @code{bold} or @code{normal}.
+
+@item font-style
+Either @code{italic} or @code{normal}.
+
+@item text-decoration
+Either @code{underline} or @code{normal}.
+
+@item font-family
+A font name, commonly @code{Monospaced} or @code{SansSerif}.
+
+@item font-size
+Values claim to be in points, e.g.@: @code{14pt}, but the values are
+actually in ``device-independent pixels'' (px), at 96/inch.
+@end table
This element has the following attributes.
Alternatively, @code{pmmlContainerPath} and @code{statsContainerPath}
name Zip members with @file{.scf} extension.
+@node SPV Structure object and image Elements
+@subsection The @code{object} and @code{image} Elements
+
+@example
+object :type[object_type]=(unknown)? :uri => EMPTY
+
+image :VDPId :commandName => dataPath
+@end example
+
+These two elements represent an image in PNG format. They are
+equivalent and the corpus contains examples of both. The only
+difference is the syntax: for @code{object}, the @code{uri} attribute
+names the Zip member that contains a PNG file; for @code{image}, the
+text of the inner @code{dataPath} element names the Zip member.
+
+PSPP writes @code{object} in output but there is no strong reason to
+choose this form.
+
+The corpus only contains PNG image files.
+
@node SPV Structure tree Element
@subsection The @code{tree} Element
@item string
@itemx bestring
A 32-bit unsigned integer, in little-endian or big-endian byte order,
-respectively, followed by the specified number of bytes of character
-data. (The encoding is indicated by the Formats nonterminal.)
+respectively, followed by the specified number of bytes of UTF-8
+encoded character data.
@item @var{x}?
@var{x} is optional, e.g.@: 00? is an optional zero byte.
concatenated together, terminated by an optional byte 01:
@example
-LightMember =>
+Table =>
Header Titles Footnotes
Areas Borders PrintSettings TableSettings Formats
Dimensions Axes Cells
version-specific formatting (as described previously).
If @code{rotate-inner-column-labels} is 1, then column labels closest
-to the data are rotated to be vertical; otherwise, they are shown
-in the normal way.
+to the data are rotated 90° counterclockwise; otherwise, they are
+shown in the normal way.
If @code{rotate-outer-row-labels} is 1, then row labels farthest from
-the data are rotated to be vertical; otherwise, they are shown in the
-normal way.
-
-@code{table-id} is a binary version of the @code{tableId} attribute in
-the structure member that refers to the detail member. For example,
-if @code{tableId} is @code{-4122591256483201023}, then @code{table-id}
-would be 0xc6c99d183b300001.
+the data are rotated 90° counterclockwise; otherwise, they are shown
+in the normal way.
@code{min-col-width} is the minimum width that a column will be
assigned automatically. @code{max-col-width} is the maximum width
the width of row labels. All of these measurements are in 1/96 inch
units (called a ``device independent pixel'' unit in Windows).
+@code{table-id} is a binary version of the @code{tableId} attribute in
+the structure member that refers to the detail member. For example,
+if @code{tableId} is @code{-4122591256483201023}, then @code{table-id}
+would be 0xc6c99d183b300001.
+
The meaning of the other variable parts of the header is not known. A
writer may safely use version 3, true for @code{x0}, false for
@code{x1}, true for @code{x2}, and 0x15 for @code{x3}.
The Titles follow the Header and specify the table's title, caption,
and corner text.
-The @code{user-title} is shown above the title and reflects any user
+The @code{user-title} reflects any user
editing of the title text or style. The @code{title} is the title
originally generated by the procedure. Both of these are appropriate
for presentation and localized to the user's language. For example,
The @code{corner-text}, if present, is shown in the upper-left corner
of the table, above the row headings and to the left of the column
-headings. It is usually absent. Corner text prevents row dimension
-labels from being displayed above the dimension's group and category
-labels (see @code{show-row-labels-in-corner}).
+headings. It is usually absent. When row dimension labels are
+displayed in the corner (see @code{show-row-labels-in-corner}), corner
+text is hidden.
The @code{caption}, if present, is shown below the table.
@code{caption} reflects user editing of the caption.
Each footnote has @code{text} and an optional custom @code{marker}
(such as @samp{*}).
+The syntax for Value would allow footnotes (and their markers) to
+reference other footnotes, but in practice this doesn't work.
+
@code{show} is a 32-bit signed integer. It is positive to show the
footnote or negative to hide it. Its magnitude is often 1, and in
other cases tends to be the number of references to the footnote.
the following order: title, caption, footer, corner, column labels,
row labels, data, and layers.
-@code{index} is the 1-based index of the Area, i.e. 1 for the first
+@code{index} is the 1-based index of the Area, i.e.@: 1 for the first
Area, through 8 for the final Area.
@code{typeface} is the string name of the font used in the area. In
@code{Times New Roman} in the rest.
@code{size} is the size of the font, in px (@pxref{SPV Light Detail
-Member Format}) The most common size in the corpus is 12 px. Even
+Member Format}). The most common size in the corpus is 12 px. Even
though @code{size} has a floating-point type, in the corpus its values
are always integers.
The PrintSettings reflect settings for printing. The fixed value of
@code{endian} can be used to validate the endianness.
-@code{all-layers} is 1 to print all layers, 0 to print only the
-visible layers.
+@code{all-layers} is 1 to print all layers, 0 to print only the layer
+designated by @code{current-layer} in TableSettings (@pxref{SPV Light
+Member Table Settings}).
@code{paginate-layers} is 1 to print each layer at the start of a new
page, 0 otherwise. (This setting is honored only @code{all-layers} is
)
bestring[notes]
bestring[table-look]
- 00...))
+ )...)
Breakpoints => be32[n-breaks] be32*[n-breaks]
or columns to pixel or point offsets.
@code{notes} is a text string that contains user-specified notes. It
-is displayed when the user hovers the cursor over the table, like
-``alt text'' on a webpage. It is not printed. It is usually empty.
+is displayed when the user hovers the cursor over the table, like text
+in the @code{title} attribute in HTML@. It is not printed. It is
+usually empty.
@code{table-look} is the name of a SPSS ``TableLook'' table style,
such as ``Default'' or ``Academic''; it is often empty.
int32[n-widths] int32*[n-widths]
string[locale]
int32[current-layer]
- bool bool bool
+ bool[x7] bool[x8] bool[x9]
Y0
CustomCurrency
count(
column widths as manually adjusted by the user.
@code{locale} is a locale including an encoding, such as
-@code{en_US.windows-1252} or @code{it_IT.windows-1252}. The rest of
-the character strings in the member use this encoding. The encoding
-string is itself encoded in US-ASCII.
+@code{en_US.windows-1252} or @code{it_IT.windows-1252}. The encoding
+string (like other strings in the member) is encoded in UTF-8.
@code{epoch} is the year that starts the epoch. A 2-digit year is
interpreted as belonging to the 100 years beginning at the epoch. The
Currency Formats,,, pspp, PSPP}. Most commonly these are all
@code{-,,,} but other strings occur.
+A writer may safely use false for @code{x7}, @code{x8}, and @code{x9}.
+
@subsubheading X0
X0 only appears, optionally, in version 1 members.
the procedure: for example, NPAR TESTS becomes ``Nonparametric
Tests.'' @code{command-local} is the procedure's name, translated
into the output language; it is often empty and, when it is not,
-sometimes the same as @code{command}.
-
-@code{dataset} is the name of the dataset analyzed to produce the
-output, e.g.@: @code{DataSet1}, and @code{datafile} the name of the
-file it was read from, e.g.@: @file{C:\Users\foo\bar.sav}. The latter
-is sometimes the empty string.
+sometimes the same as @code{command}.q
@code{missing} is the character used to indicate that a cell contains
a missing value. It is always observed as @samp{.}.
-X0 repeats @code{decimal}, @code{grouping}, CustomCurrency, and
-@code{missing} already included in Formats.
-
A writer may safely use false for @code{x17}.
@subsubheading X1
@example
X1 =>
- bool byte[x15] bool[x16]
+ bool[x14]
+ byte[show-title]
+ bool[x16]
byte[lang]
byte[show-variables]
byte[show-values]
available, 3 to display both. Again, the most common value is 0,
which probably means to use a global default.
+@code{show-title} is 1 to show the caption, 10 to hide it.
+
@code{show-caption} is true to show the caption, false to hide it.
-A writer may safely use false for @code{x14}, 1 for @code{x15}, false
-for @code{x16}, -1 for @code{x18} and @code{x19}, and false for
+A writer may safely use false for @code{x14}, false for @code{x16}, 0
+for @code{lang}, -1 for @code{x18} and @code{x19}, and false for
@code{x20}.
@subsubheading X2
(int32[x22] i0)?
@end example
+@code{small} is a small real number. In the corpus, it overwhelmingly
+takes the value 0.0001, with zero occasionally seen. Nonzero numbers
+with format 40 (@pxref{SPV Light Member Value}) whose magnitudes are
+smaller than displayed in scientific notation. (Thus, a @code{small}
+of zero prevents scientific notation from being chosen.)
+
+@code{dataset} is the name of the dataset analyzed to produce the
+output, e.g.@: @code{DataSet1}, and @code{datafile} the name of the
+file it was read from, e.g.@: @file{C:\Users\foo\bar.sav}. The latter
+is sometimes the empty string.
+
@code{date} is a date, as seconds since the epoch, i.e.@: since
January 1, 1970. Pivot tables within an SPV file often have dates a
few minutes apart, so this is probably a creation date for the table
rather than for the file.
-X3 repeats @code{decimal}, @code{grouping}, CustomCurrency, and
-@code{missing} already included in Formats. @code{command},
-@code{command-local}, @code{language}, @code{charset}, and
-@code{locale} have the same meaning as in X0.
-
-@code{small} is a small real number, e.g.@: .001. Numbers smaller
-than this in absolute value are displayed in scientific notation.
-
Sometimes @code{dataset}, @code{datafile}, and @code{date} are present
and other times they are absent. The reader can distinguish by
assuming that they are present and then checking whether the
@code{dim-index} is usually the 0-based index of the dimension, e.g.@:
0 for the first dimension, 1 for the second, and so on. Sometimes it
-is -1. There is no visible difference.
+is -1. There is no visible difference. A writer may safely use the
+0-based index.
@node SPV Light Member Categories
@subsection Categories
order of categories in the file reflects that change and
@code{leaf-index} reflects the original order.
-Occasionally a dimension has no leaf categories at all. A table that
+A dimension can have no leaf categories at all. A table that
contains such a dimension necessarily has no data at all.
A Group is a group of nested categories. Usually a Group contains at
-least one Category, so that @code{n-subcategories} is positive, but a
-few Groups with @code{n-subcategories} 0 has been observed.
+least one Category, so that @code{n-subcategories} is positive, but
+Groups with zero subcategories have been observed.
If a Group's @code{merge} is 00, the most common value, then the group
is really a distinct group that should be represented as such in the
name is irrelevant and should not be displayed. (Merged groups can be
nested!)
-(For writing an SPV file, there is no need to use the @code{merge}
-feature unless it is convenient.)
+Writers need not use merged groups.
A Group's @code{x23} appears to be i2 when all of the categories
within a group are leaf categories that directly represent data values
A Cell consists of an @code{index} and a Value. Suppose there are
@math{d} dimensions, numbered 1 through @math{d} in the order given in
-the Dimensions previously, and that dimension @math{i}, has @math{n_i}
+the Dimensions previously, and that dimension @math{i} has @math{n_i}
categories. Consider the cell at coordinates @math{x_i}, @math{1 \le
i \le d}, and note that @math{0 \le x_i < n_i}. Then the index is
calculated by the following algorithm:
@math{|} 04 ValueMod int32[format] string[value-label] string[var-name]
byte[show] string[s]
@math{|} 05 ValueMod string[var-name] string[var-label] byte[show]
+ @math{|} 06 string[local] ValueMod string[id] string[c]
@math{|} ValueMod string[template] int32[n-args] Argument*[n-args]
Argument =>
i0 Value
@table @asis
@item 01
The numeric value @code{x}, intended to be presented to the user
-formatted according to @code{format}, which is in the format described
-for system files, except that format 40 is a synonym for F format
-instead of MTIME. @xref{System File Output Formats}, for details.
+formatted according to @code{format}, which is about the same as the
+format described for system files (@pxref{System File Output
+Formats}). The exception is that format 40 is not MTIME but instead
+approximately a synonym for F format with a different rule for whether
+a value is shown in scientific notation: a value in format 40 is shown
+in scientific notation if and only if it is nonzero and its magnitude
+is less than @code{small} (@pxref{SPV Light Member Formats}).
+
Most commonly, @code{format} has width 40 (the maximum).
An @code{x} with the maximum negative double value @code{-DBL_MAX}
represents the system-missing value SYSMIS. (HIGHEST and LOWEST have
-not been observed.) @xref{System File Format}, for more about these
-special values.
+not been observed.) See @ref{System File Format}, for more about
+these special values.
@item 02
Similar to @code{01}, with the additional information that @code{x} is
The string value @code{s}, intended to be presented to the user
formatted according to @code{format}. The format for a string is not
too interesting, and the corpus contains many clearly invalid formats
-like A16.39 or A255.127 or A134.1, so readers should probably ignore
-the format entirely.
+like A16.39 or A255.127 or A134.1, so readers should probably entirely
+disregard the format. PSPP only checks @code{format} to distinguish
+AHEX format.
@code{s} is a value of variable @code{var-name} and has value label
@code{value-label}. @code{var-name} is never empty but
@code{show} has the same meaning as in the encoding for 02.
@item 05
-Variable @code{var-name}, which is rarely observed as empty in the
-corpus, with variable label @code{var-label}, which is often empty.
+Variable @code{var-name} with variable label @code{var-label}. In the
+corpus, @code{var-name} is rarely empty and @code{var-label} is often
+empty.
@code{show} determines whether to show the variable name or the
variable label. A value of 1 means to show the name, 2 to show the
label, 3 to show both, and 0 means to use the default specified in
@code{show-variables} (@pxref{SPV Light Member Formats}).
+@item 06
+Similar to type 03, with @code{fixed} assumed to be true.
+
@item otherwise
When the first byte of a RawValue is not one of the above, the
RawValue starts with a ValueMod, whose syntax is described in the next
Each of the @code{n-refs} integers is a reference to a Footnote
(@pxref{SPV Light Member Footnotes}) by 0-based index. Footnote
markers are shown appended to the main text of the Value, as
-superscripts.
+superscripts or subscripts.
The @code{subscripts}, if present, are strings to append to the main
text of the Value, as subscripts. Each subscript text is a brief
projects / pspp / blobdiff