X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=doc%2Fstatistics.texi;h=4c65898b8eb6be7f70e2f61b586fd2fb2f8c5ecd;hb=d98583b9425b8a053dc21b539203406bac74adc5;hp=473e57aeb238bce42934774dc683834662a4eeb2;hpb=915ae62e6caddda59ac7c7b5e373fcb9b8da8a22;p=pspp diff --git a/doc/statistics.texi b/doc/statistics.texi index 473e57aeb2..4c65898b8e 100644 --- a/doc/statistics.texi +++ b/doc/statistics.texi @@ -1014,12 +1014,6 @@ In @code{TABLE}, each of @var{rows}, @var{columns}, and @var{layers} is either empty or an axis expression that specifies one or more variables. At least one must specify an axis expression. -@menu -* CTABLES Categorical Variable Basics:: -* CTABLES Scalar Variable Basics:: -* CTABLES Overriding Measurement Level:: -@end menu - @node CTABLES Categorical Variable Basics @subsubsection Categorical Variables @@ -1127,7 +1121,7 @@ When @pspp{} reads data from a file in an external format, such as a text file, variables' measurement levels are often unknown. If @code{CTABLES} runs when a variable has an unknown measurement level, it makes an initial pass through the data to guess measurement levels -using the rules described earlier in this manual (@pxref{Measurement +using the rules described in an earlier section (@pxref{Measurement Level}). Use the @code{VARIABLE LEVEL} command to set or change a variable's measurement level (@pxref{VARIABLE LEVEL}). @@ -1229,13 +1223,6 @@ each function's name is given its default label and format. If no format is listed, then the default format is the print format for the variable being summarized. -@menu -* CTABLES Summary Functions for Individual Cells:: -* CTABLES Summary Functions for Groups of Cells:: -* CTABLES Summary Functions for Adjusted Weights:: -* CTABLES Unweighted Summary Functions:: -@end menu - @node CTABLES Summary Functions for Individual Cells @subsubsection Summary Functions for Individual Cells @@ -1306,12 +1293,17 @@ Options}), or user-missing values excluded because @code{MISSING=EXCLUDE} is in effect on @code{CATEGORIES}, or system-missing values. @code{COUNT} does not count these. +@xref{CTABLES Missing Values for Summary Variables}, for details of +how @code{CTABLES} summarizes missing values. + @item @code{VALIDN} (``Valid N'', F40.0) The sum of valid count weights in included categories. -@code{VALIDN} does not count missing values regardless of whether they -are in included categories via @code{CATEGORIES}. @code{VALIDN} does -not count valid values that are in excluded categories. +For categorical variables, @code{VALIDN} does not count missing values +regardless of whether they are in included categories via +@code{CATEGORIES}. @code{VALIDN} does not count valid values that are +in excluded categories. @xref{CTABLES Missing Values for Summary +Variables}, for details. @item @code{VARIANCE} (``Variance'') The variance. @@ -1388,10 +1380,11 @@ Percentage of the sum of the values within @var{area}. @node CTABLES Summary Functions for Adjusted Weights @subsubsection Summary Functions for Adjusted Weights -If the @code{WEIGHT} subcommand specified an adjustment weight -variable, then the following summary functions use its value instead -of the dictionary weight variable. Otherwise, they are equivalent to -the summary function without the @samp{E}-prefix: +If the @code{WEIGHT} subcommand specified an effective weight variable +(@pxref{CTABLES Effective Weight}), then the following summary functions +use its value instead of the dictionary weight variable. Otherwise, +they are equivalent to the summary function without the +@samp{E}-prefix: @itemize @bullet @item @@ -1434,7 +1427,7 @@ counts: @code{UMISSING} (``Unweighted Missing'') @item -@code{UMODE} (``Unweight Mode'') +@code{UMODE} (``Unweighted Mode'') @item @code{U@i{area}PCT.SUM} (``Unweighted @i{Area} Sum %'', PCT40.1) @@ -1461,8 +1454,6 @@ counts: @code{UVARIANCE} (``Unweighted Variance'', F40.0) @end itemize -@c TODO missing value treatment - @node CTABLES Statistics Positions and Labels @subsection Statistics Positions and Labels @@ -1614,16 +1605,20 @@ variables. @code{CATEGORIES} applies to the table produced by the @t{VARIABLES} is required and must list the variables for the subcommand to affect. -There are two way to specify the Categories to include and their sort -order: +The syntax may specify the categories to include and their sort order +either explicitly or implicitly. The following sections give the +details of each form of syntax, followed by information on totals and +subtotals and the @code{EMPTY} setting. + +@node CTABLES Explicit Categories +@subsubsection Explicit Categories -@table @asis -@item Explicit categories. @anchor{CTABLES Explicit Category List} -To explicitly specify categories to include, list the categories -within square brackets in the desired sort order. Use spaces or -commas to separate values. Categories not covered by the list are -excluded from analysis. + +To use @code{CTABLES} to explicitly specify categories to include, +list the categories within square brackets in the desired sort order. +Use spaces or commas to separate values. Categories not covered by +the list are excluded from analysis. Each element of the list takes one of the following forms: @@ -1654,15 +1649,30 @@ Any non-missing value not covered by any other element of the list @item &@i{postcompute} A computed category name (@pxref{CTABLES Computed Categories}). + +@item SUBTOTAL +@itemx HSUBTOTAL +A subtotal (@pxref{CTABLES Totals and Subtotals}). @end table -Additional forms, described later, allow for subtotals. If multiple elements of the list cover a given category, the last one in the list takes precedence. -@item Implicit categories. -Without an explicit list of categories, @pspp{} sorts -categories automatically. +The following example syntax and output show how an explicit category +can limit the displayed categories: + +@example +CTABLES /TABLE qn1. +CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 [1, 2, 3]. +@end example +@psppoutput {ctables27} + +@node CTABLES Implicit Categories +@subsubsection Implicit Categories + +In the absence of an explicit list of categories, @code{CATEGORIES} +allows @code{KEY}, @code{ORDER}, and @code{MISSING} to specify how to +select and sort categories. The @code{KEY} setting specifies the sort key. By default, or with @code{KEY=VALUE}, categories are sorted by default. Categories may @@ -1683,24 +1693,45 @@ order. Specify @code{ORDER=D} to sort in descending order. User-missing values are excluded by default, or with @code{MISSING=EXCLUDE}. Specify @code{MISSING=INCLUDE} to include user-missing values. The system-missing value is always excluded. -@end table -@subsubheading Totals and Subtotals +The following example syntax and output show how +@code{MISSING=INCLUDE} causes missing values to be included in a +category list. + +@example +CTABLES /TABLE qn1. +CTABLES /TABLE qn1 /CATEGORIES VARIABLES=qn1 MISSING=INCLUDE. +@end example +@psppoutput {ctables28} -@code{CATEGORIES} also controls display of totals and subtotals. -Totals are not displayed with @code{TOTAL=NO}, which is also the -default. Specify @code{TOTAL=YES} to display a total. By default, -the total is labeled ``Total''; use @code{LABEL="@i{label}"} to -override it. +@node CTABLES Totals and Subtotals +@subsubsection Totals and Subtotals + +@code{CATEGORIES} also controls display of totals and subtotals. By +default, or with @code{TOTAL=NO}, totals are not displayed. Use +@code{TOTAL=YES} to display a total. By default, the total is labeled +``Total''; use @code{LABEL="@i{label}"} to override it. Subtotals are also not displayed by default. To add one or more subtotals, use an explicit category list and insert @code{SUBTOTAL} or @code{HSUBTOTAL} in the position or positions where the subtotal -should appear. With @code{SUBTOTAL}, the subtotal becomes an extra -row or column or layer; @code{HSUBTOTAL} additionally hides the -categories that make up the subtotal. Either way, the default label -is ``Subtotal'', use @code{SUBTOTAL="@i{label}"} or -@code{HSUBTOTAL="@i{label}"} to specify a custom label. +should appear. The subtotal becomes an extra row or column or layer. +@code{HSUBTOTAL} additionally hides the categories that make up the +subtotal. Either way, the default label is ``Subtotal'', use +@code{SUBTOTAL="@i{label}"} or @code{HSUBTOTAL="@i{label}"} to specify +a custom label. + +The following example syntax and output show how to use +@code{TOTAL=YES} and @code{SUBTOTAL}: + +@example +CTABLES + /TABLE qn1 + /CATEGORIES VARIABLES=qn1 [OTHERNM, SUBTOTAL='Valid Total', + MISSING, SUBTOTAL='Missing Total'] + TOTAL=YES LABEL='Overall Total'. +@end example +@psppoutput {ctables29} By default, or with @code{POSITION=AFTER}, totals are displayed in the output after the last category and subtotals apply to categories that @@ -1709,12 +1740,34 @@ first category and subtotals apply to categories that follow them. Only categorical variables may have totals and subtotals. Scalar variables may be ``totaled'' indirectly by enabling totals and -subtotals on a categorical variable within which the scalar variable is -summarized. +subtotals on a categorical variable within which the scalar variable +is summarized. For example, the following syntax produces a mean, +count, and valid count across all data by adding a total on the +categorical @code{region} variable, as shown: -@c TODO Specifying summaries for totals and subtotals +@example +CTABLES /TABLE=region > qn20 [MEAN, VALIDN] + /CATEGORIES VARIABLES=region TOTAL=YES LABEL='All regions'. +@end example +@psppoutput {ctables30} -@subsubheading Categories Without Values +By default, @pspp{} uses the same summary functions for totals and +subtotals as other categories. To summarize totals and subtotals +differently, specify the summary functions for totals and subtotals +after the ordinary summary functions inside a nested set of @code{[]} +following @code{TOTALS}. For example, the following syntax displays +@code{COUNT} for individual categories and totals and @code{VALIDN} +for totals, as shown: + +@example +CTABLES + /TABLE qnd7a [COUNT, TOTALS[COUNT, VALIDN]] + /CATEGORIES VARIABLES=qnd7a TOTAL=YES MISSING=INCLUDE. +@end example +@psppoutput {ctables26} + +@node CTABLES Categories Without Values +@subsubsection Categories Without Values Some categories might not be included in the data set being analyzed. For example, our example data set has no cases in the ``15 or @@ -1728,6 +1781,16 @@ categories, they include all the values listed individually and all values with value labels that are covered by ranges or @code{MISSING} or @code{OTHERNM}. +The following example syntax and output show the effect of +@code{EMPTY=EXCLUDE} for the @code{qns1} variable, in which 0 is labeled +``None'' but no cases exist with that value: + +@example +CTABLES /TABLE=qns1. +CTABLES /TABLE=qns1 /CATEGORIES VARIABLES=qns1 EMPTY=EXCLUDE. +@end example +@psppoutput {ctables31} + @node CTABLES Titles @subsection Titles @@ -1738,14 +1801,32 @@ or @code{OTHERNM}. [@t{CORNER=}@i{string}@dots{}] @end display -@c TODO Describe substitution variables - The @code{TITLES} subcommand sets the title, caption, and corner text -for the table output for the previous @code{TABLE} subcommand. The -title appears above the table, the caption below the table, and the -corner text appears in the table's upper left corner. By default, the -title is ``Custom Tables'' and the caption and corner text are empty. -With some table output styles, the corner text is not displayed. +for the table output for the previous @code{TABLE} subcommand. Any +number of strings may be specified for each kind of text, with each +string appearing on a separate line in the output. The title appears +above the table, the caption below the table, and the corner text +appears in the table's upper left corner. By default, the title is +``Custom Tables'' and the caption and corner text are empty. With +some table output styles, the corner text is not displayed. + +The strings provided in this subcommand may contain the following +macro-like keywords that @pspp{} substitutes at the time that it runs +the command: + +@table @code @c ( +@item )DATE +The current date, e.g.@: MM/DD/YY. The format is locale-dependent. + +@c ( +@item )TIME +The current time, e.g.@: HH:MM:SS. The format is locale-dependent. + +@c ( +@item )TABLE +The expression specified on the @code{TABLE} command. Summary +and measurement level specifications are omitted, and variable labels are used in place of variable names. +@end table @node CTABLES Table Formatting @subsection Table Formatting @@ -1770,7 +1851,8 @@ specify a number for either or both of these settings. If both are specified, @code{MAXCOLWIDTH} must be greater than or equal to @code{MINCOLWIDTH}. The default unit, or with @code{UNITS=POINTS}, is points (1/72 inch), or specify @code{UNITS=INCHES} to use inches or -@code{UNITS=CM} for centimeters. +@code{UNITS=CM} for centimeters. @pspp{} does not currently honor any +of these settings. By default, or with @code{EMPTY=ZERO}, zero values are displayed in their usual format. Use @code{EMPTY=BLANK} to use an empty cell @@ -1818,6 +1900,89 @@ Show nothing. @node CTABLES Missing Value Treatment @subsection Missing Value Treatment +The @code{TABLE} subcommand on @code{CTABLES} specifies two different +kinds of variables: variables that divide tables into cells (which are +always categorical) and variables being summarized (which may be +categorical or scale). @pspp{} treats missing values differently in +each kind of variable, as described in the sections below. + +@node CTABLES Missing Values for Cell-Defining Variables +@subsubsection Missing Values for Cell-Defining Variables + +For variables that divide tables into cells, per-variable category +options, as described in @ref{CTABLES Per-Variable Category Options}, +determine which data is analyzed. If any of the categories for such a +variable would exclude a case, then that case is not included. + +As an example, consider the following entirely artificial dataset, in +which @samp{x} and @samp{y} are categorical variables with missing +value 9, and @samp{z} is scale: + +@psppoutput{ctables32} + +Using @samp{x} and @samp{y} to define cells, and summarizing @samp{z}, +by default @pspp{} omits all the cases that have @samp{x} or @samp{y} (or both) +missing: + +@example +CTABLES /TABLE x > y > z [SUM]. +@end example +@psppoutput{ctables33} + +If, however, we add @code{CATEGORIES} specifications to include +missing values for @samp{y} or for @samp{x} and @samp{y}, the output +table includes them, like so: + +@example +CTABLES /TABLE x > y > z [SUM] /CATEGORIES VARIABLES=y MISSING=INCLUDE. +CTABLES /TABLE x > y > z [SUM] /CATEGORIES VARIABLES=x y MISSING=INCLUDE. +@end example +@psppoutput{ctables34} + +@node CTABLES Missing Values for Summary Variables +@subsubsection Missing Values for Summary Variables + +For summary variables, values that are valid and in included +categories are analyzed, and values that are missing or in excluded +categories are not analyzed, with the following exceptions: + +@itemize @bullet +@item +The ``@t{VALIDN}'' summary functions (@code{VALIDN}, @code{EVALIDN}, +@code{UVALIDN}, @code{@i{area}PCT.VALIDN}, and +@code{U@i{area}PCT.VALIDN}) only count valid values in included +categories (not missing values in included categories). + +@item +The ``@t{TOTALN}'' summary functions (@code{TOTALN}, @code{ETOTALN}, +@code{UTOTALN}, @code{@i{area}PCT.TOTALN}), and +@code{U@i{area}PCT.TOTALN} count all values (valid and missing) in +included categories and missing (but not valid) values in excluded +categories. +@end itemize + +@noindent +For categorical variables, system-missing values are never in included +categories. For scale variables, there is no notion of included and +excluded categories, so all values are effectively included. + +The following table provides another view of the above rules: + +@multitable {@w{ }@w{ }@w{ }@w{ }Missing values in excluded categories} {@t{VALIDN}} {other} {@t{TOTALN}} +@headitem @tab @t{VALIDN} @tab other @tab @t{TOTALN} +@item @headitemfont{Categorical variables:} +@item @w{ }@w{ }@w{ }@w{ }Valid values in included categories @tab yes @tab yes @tab yes +@item @w{ }@w{ }@w{ }@w{ }Missing values in included categories @tab --- @tab yes @tab yes +@item @w{ }@w{ }@w{ }@w{ }Missing values in excluded categories @tab --- @tab --- @tab yes +@item @w{ }@w{ }@w{ }@w{ }Valid values in excluded categories @tab --- @tab --- @tab --- +@item @headitemfont{Scale variables:} +@item @w{ }@w{ }@w{ }@w{ }Valid values @tab yes @tab yes @tab yes +@item @w{ }@w{ }@w{ }@w{ }User- or system-missing values @tab --- @tab yes @tab yes +@end multitable + +@node CTABLES Scale Missing Values +@subsubsection Scale Missing Values + @display @t{/SMISSING} @{@t{VARIABLE} @math{|} @t{LISTWISE}@} @end display @@ -1875,19 +2040,30 @@ CTABLES /SMISSING LISTWISE /TABLE (y > x) + (z > x). @display @t{/PCOMPUTE} @t{&}@i{postcompute}@t{=EXPR(}@i{expression}@t{)} +@t{/PPROPERTIES} @t{&}@i{postcompute}@dots{} + [@t{LABEL=}@i{string}] + [@t{FORMAT=}[@i{summary} @i{format}]@dots{}] + [@t{HIDESOURCECATS=}@{@t{NO} @math{|} @t{YES}@} @end display @dfn{Computed categories}, also called @dfn{postcomputes}, are categories created using arithmetic on categories obtained from the -data. The @code{PCOMPUTE} subcommand defines computed categories, -which can then be used in two places: on @code{CATEGORIES} within an -explicit category list (@pxref{CTABLES Explicit Category List}), and on -the @code{PPROPERTIES} subcommand to define further properties for a -given postcompute. +data. The @code{PCOMPUTE} subcommand creates a postcompute, which may +then be used on @code{CATEGORIES} within an explicit category list +(@pxref{CTABLES Explicit Category List}). Optionally, +@code{PPROPERTIES} refines how a postcompute is displayed. The +following sections provide the details. + +@node CTABLES PCOMPUTE +@subsubsection PCOMPUTE + +@display +@t{/PCOMPUTE} @t{&}@i{postcompute}@t{=EXPR(}@i{expression}@t{)} +@end display -@code{PCOMPUTE} must precede the first @code{TABLE} command. It is -optional and it may be used any number of times to define multiple -postcomputes. +The @code{PCOMPUTE} subcommand, which must precede the first +@code{TABLE} command, defines computed categories. It is optional and +may be used any number of times to define multiple postcomputes. Each @code{PCOMPUTE} defines one postcompute. Its syntax consists of a name to identify the postcompute as a @pspp{} identifier prefixed by @@ -1954,8 +2130,30 @@ Normally a named postcompute is defined only once, but if a later @code{PCOMPUTE} redefines a postcompute with the same name as an earlier one, the later one take precedence. -@node CTABLES Computed Category Properties -@subsection Computed Category Properties +The following syntax and output shows how @code{PCOMPUTE} can compute +a total over subtotals, summing the ``Frequent Drivers'' and +``Infrequent Drivers'' subtotals to form an ``All Drivers'' +postcompute. It also shows how to calculate and display a percentage, +in this case the percentage of valid responses that report never +driving. It uses @code{PPROPERTIES} (@pxref{CTABLES PPROPERTIES}) to +display the latter in @code{PCT} format. + +@example +CTABLES + /PCOMPUTE &all_drivers=EXPR([1 THRU 2] + [3 THRU 4]) + /PPROPERTIES &all_drivers LABEL='All Drivers' + /PCOMPUTE &pct_never=EXPR([5] / ([1 THRU 2] + [3 THRU 4] + [5]) * 100) + /PPROPERTIES &pct_never LABEL='% Not Drivers' FORMAT=COUNT PCT40.1 + /TABLE=qn1 BY qns3a + /CATEGORIES VARIABLES=qn1 [1 THRU 2, SUBTOTAL='Frequent Drivers', + 3 THRU 4, SUBTOTAL='Infrequent Drivers', + &all_drivers, 5, &pct_never, + MISSING, SUBTOTAL='Not Drivers or Missing']. +@end example +@psppoutput{ctables35} + +@node CTABLES PPROPERTIES +@subsubsection PPROPERTIES @display @t{/PPROPERTIES} @t{&}@i{postcompute}@dots{} @@ -1975,15 +2173,19 @@ All of the settings on @code{PPROPERTIES} are optional. Use output. The default label for a postcompute is the expression used to define it. -The @code{FORMAT} setting sets summary statistics and display formats -for the postcomputes. +A postcompute always uses same summary functions as the variable whose +categories contain it, but @code{FORMAT} allows control over the +format used to display their values. It takes a list of summary +function names and format specifiers. By default, or with @code{HIDESOURCECATS=NO}, categories referred to by computed categories are displayed like other categories. Use @code{HIDESOURCECATS=YES} to hide them. -@node CTABLES Base Weight -@subsection Base Weight +The previous section provides an example for @code{PPROPERTIES}. + +@node CTABLES Effective Weight +@subsection Effective Weight @display @t{/WEIGHT VARIABLE=}@i{variable} @@ -1991,17 +2193,17 @@ by computed categories are displayed like other categories. Use The @code{WEIGHT} subcommand is optional and must appear before @code{TABLE}. If it appears, it must name a numeric variable, known -as the @dfn{effective base weight} or @dfn{adjustment weight}. The -effective base weight variable stands in for the dictionary's weight +as the @dfn{effective weight} or @dfn{adjustment weight}. The +effective weight variable stands in for the dictionary's weight variable (@pxref{WEIGHT}), if any, in most calculations in @code{CTABLES}. The only exceptions are the @code{COUNT}, @code{TOTALN}, and @code{VALIDN} summary functions, which use the dictionary weight instead. Weights obtained from the @pspp{} dictionary are rounded to the -nearest integer at the case level. Effective base weights are not -rounded. Regardless of the weighting source, @pspp{} does not analyze -cases with zero, missing, or negative effective weights. +nearest integer at the case level. Effective weights are not rounded. +Regardless of the weighting source, @pspp{} does not analyze cases +with zero, missing, or negative effective weights. @node CTABLES Hiding Small Counts @subsection Hiding Small Counts @@ -2011,10 +2213,18 @@ cases with zero, missing, or negative effective weights. @end display The @code{HIDESMALLCOUNTS} subcommand is optional. If it specified, -then count values in output tables less than the value of @i{count} -are shown as @code{<@i{count}} instead of their true values. The -value of @i{count} must be an integer and must be at least 2. Case -weights are considered for deciding whether to hide a count. +then @code{COUNT}, @code{ECOUNT}, and @code{UCOUNT} values in output +tables less than the value of @i{count} are shown as @code{<@i{count}} +instead of their true values. The value of @i{count} must be an +integer and must be at least 2. + +The following syntax and example shows how to use +@code{HIDESMALLCOUNTS}: + +@example +CTABLES /HIDESMALLCOUNTS COUNT=10 /TABLE qn37. +@end example +@psppoutput{ctables36} @node FACTOR @section FACTOR