The rest of the members in an SPV file's Zip archive fall into two
categories: @dfn{structure} and @dfn{detail} members. Structure
-member names begin with @file{outputViewer@var{nnnnnnnnnn}}, where
-each @var{n} is a decimal digit, and end with @file{.xml}, and often
-include the string @file{_heading} in between. Each of these members
-represents some kind of output item (a table, a heading, a block of
-text, etc.) or a group of them. The member whose output goes at the
-beginning of the document is numbered 0, the next member in the output
-is numbered 1, and so on.
+member names take the form with @file{outputViewer@var{number}.xml} or
+@file{outputViewer@var{number}_heading.xml}, where @var{number} is an
+10-digit decimal number. Each of these members represents some kind
+of output item (a table, a heading, a block of text, etc.) or a group
+of them. The member whose output goes at the beginning of the
+document is numbered 0, the next member in the output is numbered 1,
+and so on.
Structure members contain XML. This XML is sometimes self-contained,
but it often references detail members in the Zip archive, which are
The @file{@var{prefix}} in the names of the detail members is
typically an 11-digit decimal number that increases for each item,
tending to skip values. Older SPV files use different naming
-conventions. Structure member refer to detail members by name, and so
-their exact names do not matter to readers as long as they are unique.
+conventions for detail members. Structure member refer to detail
+members by name, and so their exact names do not matter to readers as
+long as they are unique.
SPSS tolerates corrupted Zip archives that Zip reader libraries tend
to reject. These can be fixed up with @command{zip -FF}.
=> label (container | heading)*
@end example
-The root of a structure member is a @code{heading}, which represents a
-section of output beginning with a @code{label} and
-ordinarily followed by content containers or further nested
-(sub)-sections of output. Unlike heading elements in HTML and other
-common document formats, which precede the content that they head,
-@code{heading} contains the elements that appear below the heading.
-
-The document root heading, only, may contain a @code{pageSetup}
-element.
+A @code{heading} represents a tree of content that appears in an
+output viewer window. It contains a @code{label} text string that is
+shown in the outline view ordinarily followed by content containers or
+further nested (sub)-sections of output. Unlike heading elements in
+HTML and other common document formats, which precede the content that
+they head, @code{heading} contains the elements that appear below the
+heading.
+
+The root of a structure member is a special @code{heading}. The
+direct children of the root @code{heading} elements in all structure
+members in an SPV file are siblings. That is, the root @code{heading}
+in all of the structure members conceptually represent the same node.
+The root heading's @code{label} is ignored (see @pxref{SPV Structure
+label Element}). The root heading in the first structure member in
+the Zip file may contain a @code{pageSetup} element.
+
+The schema implies that any @code{heading} may contain a sequence of
+any number of @code{heading} and @code{container} elements. This does
+not work for the root @code{heading} in practice, which must actually
+contain exactly one @code{container} or @code{heading} child element.
+Furthermore, if the root heading's child is a @code{heading}, then the
+structure member's name must end in @file{_heading.xml}; if it is a
+@code{container} child, then it must not.
The following attributes have been observed on both document root and
nested @code{heading} elements.
@end defvr
@defvr {Attribute} @code{visibility}
-To what degree the output represented by the element is visible.
+If this attribute is absent, the heading's content is expanded in the
+outline view. If it is set to @code{collapsed}, it is collapsed.
+(This attribute is never present in a root @code{heading} because the
+root node is always expanded when a file is loaded, even though the UI
+can be used to collapse it interactively.)
@end defvr
@defvr {Attribute} @code{locale}
The text in @code{label} describes what it labels, often by naming the
statistical procedure that was executed, e.g.@: ``Frequencies'' or
-``T-Test''. The root @code{heading} in a structure member is normally
-``Output''. Labels are often very generic, especially within a
+``T-Test''. Labels are often very generic, especially within a
@code{container}, e.g.@: ``Title'' or ``Warnings'' or ``Notes''.
Label text is localized according to the output language, e.g.@: in
Italian a frequency table procedure is labeled ``Frequenze''.
contains a few examples of empty labels, ones that contain no text,
probably as a result of user editing.
+The root @code{heading} in an SPV file has a @code{label}, like every
+@code{heading}. It normally contains ``Output'' but its content is
+disregarded anyway. The user cannot edit it.
+
@node SPV Structure container Element
@subsection The @code{container} Element
@defvr {Attribute} @code{visibility}
Whether the container's content is displayed. ``Notes'' tables are
-often hidden; other data is usually
+often hidden; other data is usually visible.
@end defvr
@defvr {Attribute} @code{text-align}
The width of the container, e.g.@: @code{1097px}.
@end defvr
+All of the elements that nest inside @code{container} (except the
+@code{label}) have the following optional attribute.
+
+@defvr {Attribute} @code{commandName}
+As on the @code{heading} element. The corpus contains one example
+of where @code{commandName} is present but set to the empty string.
+@end defvr
+
@node SPV Structure text Element (Inside @code{container})
@subsection The @code{text} Element (Inside @code{container})
This element has the following attributes.
-@defvr {Attribute} @code{type}
-The semantics of the text.
+@defvr {Attribute} @code{commandName}
+@xref{SPV Structure container Element}. For output not specific to a
+command, this is simply @code{log}.
@end defvr
-@defvr {Attribute} @code{commandName}
-As on the @code{heading} element. For output not specific to a
-command, this is simply @code{log}. The corpus contains one example
-of where @code{commandName} is present but set to the empty string.
+@defvr {Attribute} @code{type}
+The semantics of the text.
@end defvr
@defvr {Attribute} @code{creator-version}
This element has the following attributes.
@defvr {Attribute} @code{commandName}
-As on the @code{heading} element.
+@xref{SPV Structure container Element}.
@end defvr
@defvr {Attribute} @code{type}
@subsection The @code{object} and @code{image} Elements
@example
-object :type[object_type]=(unknown)? :uri => EMPTY
+object
+ :commandName?
+ :type[object_type]=(unknown)?
+ :uri
+=> EMPTY
-image :VDPId :commandName => dataPath
+image
+ :commandName?
+ :VDPId
+=> dataPath
@end example
These two elements represent an image in PNG format. They are
@item string
@itemx bestring
A 32-bit unsigned integer, in little-endian or big-endian byte order,
-respectively, followed by the specified number of bytes of UTF-8
-encoded character data.
+respectively, followed by the specified number of bytes of character
+data. (The encoding is indicated by the Formats nonterminal.)
@item @var{x}?
@var{x} is optional, e.g.@: 00? is an optional zero byte.
@code{show} is a 32-bit signed integer. It is positive to show the
footnote or negative to hide it. Its magnitude is often 1, and in
other cases tends to be the number of references to the footnote.
+It is safe to write 1 to show a footnote and -1 to hide it.
@node SPV Light Member Areas
@subsection Areas
The TableSettings reflect display settings. The fixed value of
@code{endian} can be used to validate the endianness.
-@code{current-layer} is the displayed layer. The interpretation when
-there is more than one layer dimension is not yet known.
+@code{current-layer} is the displayed layer. Suppose there are
+@math{d} layers, numbered 1 through @math{d} in the order given in the
+Dimensions (@pxref{SPV Light Member Dimensions}), and that the
+displayed value of dimension @math{i} is @math{d_i}, @math{0 \le x_i <
+n_i}, where @math{n_i} is the number of categories in dimension
+@math{i}. Then @code{current-layer} is calculated by the following
+algorithm:
+
+@display
+let @code{current-layer} = 0
+for each @math{i} from @math{d} downto 1:
+ @code{current-layer} = (@math{n_i \times} @code{current-layer}) @math{+} @math{x_i}
+@end display
If @code{omit-empty} is 1, empty rows or columns (ones with nothing in
any cell) are hidden; otherwise, they are shown.
column widths as manually adjusted by the user.
@code{locale} is a locale including an encoding, such as
-@code{en_US.windows-1252} or @code{it_IT.windows-1252}. The encoding
-string (like other strings in the member) is encoded in UTF-8.
+@code{en_US.windows-1252} or @code{it_IT.windows-1252}.
+(@code{locale} is often duplicated in Y1, described below).
@code{epoch} is the year that starts the epoch. A 2-digit year is
interpreted as belonging to the 100 years beginning at the epoch. The
the procedure: for example, NPAR TESTS becomes ``Nonparametric
Tests.'' @code{command-local} is the procedure's name, translated
into the output language; it is often empty and, when it is not,
-sometimes the same as @code{command}.q
+sometimes the same as @code{command}.
@code{missing} is the character used to indicate that a cell contains
a missing value. It is always observed as @samp{.}.
@code{lang} may indicate the language in use. Some values seem to be
0: @t{en}, 1: @t{de}, 2: @t{es}, 3: @t{it}, 5: @t{ko}, 6: @t{pl}, 8:
-@t{zh-tw}, 10: @t{pt_BR}, 11: @t{fr}. The @code{locale} in Formats
-and the @code{language}, @code{charset}, and @code{locale} in X0 are
-more likely to be useful in practice.
+@t{zh-tw}, 10: @t{pt_BR}, 11: @t{fr}.
@code{show-variables} determines how variables are displayed by
default. A value of 1 means to display variable names, 2 to display
A writer may safely use 4 for @code{x21} and omit @code{x22} and the
other optional bytes at the end.
+@subsubheading Encoding
+
+Formats contains several indications of character encoding:
+
+@itemize @bullet
+@item
+@code{locale} in Formats itself.
+
+@item
+@code{locale} in Y1 (in version 1, Y1 is optionally nested inside X0;
+in version 3, Y1 is nested inside X3).
+
+@item
+@code{charset} in version 3, in Y1.
+
+@item
+@code{lang} in X1, in version 3.
+@end itemize
+
+@code{charset}, if present, is a good indication of character
+encoding, and in its absence the encoding suffix on @code{locale} in
+Formats will work.
+
+@code{locale} in Y1 can be disregarded: it is normally the same as
+@code{locale} in Formats, and it is only present if @code{charset} is
+also.
+
+@code{lang} is not helpful and should be ignored for character
+encoding purposes.
+
+However, the corpus contains many examples of light members whose
+strings are encoded in UTF-8 despite declaring some other character
+set. Furthermore, the corpus contains several examples of light
+members in which some strings are encoded in UTF-8 (and contain
+multibyte characters) and other strings are encoded in another
+character set (and contain non-ASCII characters). PSPP treats any
+valid UTF-8 string as UTF-8 and only falls back to the declared
+encoding for strings that are not valid UTF-8.
+
+The @command{pspp-output} program's @command{strings} command can help
+analyze the encoding in an SPV light member. Use @code{pspp-output
+--help-dev} to see its usage.
+
@node SPV Light Member Dimensions
@subsection Dimensions
layers, the next @code{n-rows} integers specify the dimensions
represented by rows, and the final @code{n-columns} integers specify
the dimensions represented by columns. When there is more than one
-dimension of a given kind, the inner dimensions are given first.
+dimension of a given kind, the inner dimensions are given first. (For
+the layer axis, this means that the first dimension is at the bottom
+of the list and the last dimension is at the top when the current
+layer is displayed.)
@node SPV Light Member Cells
@subsection Cells
projects / pspp / blobdiff