-@node Data Selection, Conditionals and Looping, Data Manipulation, Top
+@c PSPP - a program for statistical analysis.
+@c Copyright (C) 2017, 2020 Free Software Foundation, Inc.
+@c Permission is granted to copy, distribute and/or modify this document
+@c under the terms of the GNU Free Documentation License, Version 1.3
+@c or any later version published by the Free Software Foundation;
+@c with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
+@c A copy of the license is included in the section entitled "GNU
+@c Free Documentation License".
+@c
+@node Data Selection
@chapter Selecting data for analysis
-This chapter documents PSPP commands that temporarily or permanently
-select data records from the active file for analysis.
+This chapter documents @pspp{} commands that temporarily or permanently
+select data records from the active dataset for analysis.
@menu
* FILTER:: Exclude cases based on a variable.
-* N OF CASES:: Limit the size of the active file.
-* PROCESS IF:: Temporarily excluding cases.
+* N OF CASES:: Limit the size of the active dataset.
* SAMPLE:: Select a specified proportion of cases.
* SELECT IF:: Permanently delete selected cases.
* SPLIT FILE:: Do multiple analyses with one command.
* WEIGHT:: Weight cases by a variable.
@end menu
-@node FILTER, N OF CASES, Data Selection, Data Selection
+@node FILTER
@section FILTER
@vindex FILTER
@display
-FILTER BY var_name.
+FILTER BY @var{var_name}.
FILTER OFF.
@end display
@cmd{FILTER} allows a boolean-valued variable to be used to select
cases from the data stream for processing.
-To set up filtering, specify BY and a variable name. Keyword
+To set up filtering, specify @subcmd{BY} and a variable name. Keyword
BY is optional but recommended. Cases which have a zero or system- or
user-missing value are excluded from analysis, but not deleted from the
data stream. Cases with other values are analyzed.
Filtering takes place immediately before cases pass to a procedure for
analysis. Only one filter variable may be active at a time. Normally,
case filtering continues until it is explicitly turned off with @code{FILTER
-OFF}. However, if @cmd{FILTER} is placed after TEMPORARY, it filters only
+OFF}. However, if @cmd{FILTER} is placed after @cmd{TEMPORARY}, it filters only
the next procedure or procedure-like command.
-@node N OF CASES, PROCESS IF, FILTER, Data Selection
+@node N OF CASES
@section N OF CASES
@vindex N OF CASES
@display
-N [OF CASES] num_of_cases [ESTIMATED].
+N [OF CASES] @var{num_of_cases} [ESTIMATED].
@end display
-Sometimes you may want to disregard cases of your input. @cmd{N} can
-do this. @code{N 100} tells PSPP to disregard all cases after the
-first 100.
-
-If the value specified for @cmd{N} is greater than the number of cases
-read in, the value is ignored.
-
-@cmd{N} does not discard cases or prevent them from being read. It
-just causes cases beyond the last one specified to be ignored by data
-analysis commands.
-
-A later @cmd{N} command can increase or decrease the number of cases
-selected. (To select all the cases without knowing how many there are,
-specify a very high number: 100000 or whatever you think is large enough.)
-
-Transformation procedures performed after @cmd{N} is executed
-@emph{do} cause cases to be discarded.
-
-@cmd{SAMPLE}, @cmd{PROCESS IF}, and @cmd{SELECT IF} have
-precedence over @cmd{N}---the same results are obtained by both of the
-following fragments, given the same random number seeds:
-
-@example
-@i{@dots{}set up, read in data@dots{}}
-N 100.
-SAMPLE .5.
-@i{@dots{}analyze data@dots{}}
-
-@i{@dots{}set up, read in data@dots{}}
-SAMPLE .5.
-N 100.
-@i{@dots{}analyze data@dots{}}
-@end example
-
-Both fragments above first randomly sample approximately half of the
-cases, then select the first 100 of those sampled.
-
-@cmd{N} with the @code{ESTIMATED} keyword gives an
-estimated number of cases before @cmd{DATA LIST} or another command to
-read in data. @code{ESTIMATED} never limits the number of cases
-processed by procedures. PSPP currently does not make use of
-case count estimates.
-
-When @cmd{N} is specified after @cmd{TEMPORARY}, it affects only
-the next procedure (@pxref{TEMPORARY}).
-
-@node PROCESS IF, SAMPLE, N OF CASES, Data Selection
-@section PROCESS IF
-@vindex PROCESS IF
-
-@example
-PROCESS IF expression.
-@end example
-
-@cmd{PROCESS IF} temporarily eliminates cases from the
-data stream. Its effects are active only through the execution of the
-next procedure or procedure-like command.
-
-Specify a boolean expression (@pxref{Expressions}). If the value of the
-expression is true for a particular case, the case will be analyzed. If
-the expression has a false or missing value, then the case will be
-deleted from the data stream for this procedure only.
-
-Regardless of its placement relative to other commands, @cmd{PROCESS IF}
-always takes effect immediately before data passes to the procedure.
-Only one @cmd{PROCESS IF} command may be in effect at any given time.
-
-The effects of @cmd{PROCESS IF} are similar, but not identical, to the
-effects of executing @cmd{TEMPORARY}, then @cmd{SELECT IF}
-(@pxref{SELECT IF}).
-
-The filtering performed by @cmd{PROCESS IF} takes place immediately
-before cases pass to a procedure for analysis. Because @cmd{PROCESS
-IF} affects only a single procedure, its placement relative to
-@cmd{TEMPORARY} is unimportant.
-
-@cmd{PROCESS IF} is deprecated. It is included for compatibility with
-old command files. New syntax files should use @cmd{SELECT IF} or
-@cmd{FILTER} instead.
-
-@node SAMPLE, SELECT IF, PROCESS IF, Data Selection
+@cmd{N OF CASES} limits the number of cases processed by any
+procedures that follow it in the command stream. @code{N OF CASES
+100}, for example, tells @pspp{} to disregard all cases after the first
+100.
+
+When @cmd{N OF CASES} is specified after @cmd{TEMPORARY}, it affects
+only the next procedure (@pxref{TEMPORARY}). Otherwise, cases beyond
+the limit specified are not processed by any later procedure.
+
+If the limit specified on @cmd{N OF CASES} is greater than the number
+of cases in the active dataset, it has no effect.
+
+When @cmd{N OF CASES} is used along with @cmd{SAMPLE} or @cmd{SELECT
+IF}, the case limit is applied to the cases obtained after sampling or
+case selection, regardless of how @cmd{N OF CASES} is placed relative
+to @cmd{SAMPLE} or @cmd{SELECT IF} in the command file. Thus, the
+commands @code{N OF CASES 100} and @code{SAMPLE .5} both randomly
+sample approximately half of the active dataset's cases, then select the
+first 100 of those sampled, regardless of their order in the command
+file.
+
+@cmd{N OF CASES} with the @code{ESTIMATED} keyword gives an estimated
+number of cases before @cmd{DATA LIST} or another command to read in
+data. @code{ESTIMATED} never limits the number of cases processed by
+procedures. @pspp{} currently does not make use of case count estimates.
+
+@node SAMPLE
@section SAMPLE
@vindex SAMPLE
@display
-SAMPLE num1 [FROM num2].
+SAMPLE @var{num1} [FROM @var{num2}].
@end display
@cmd{SAMPLE} randomly samples a proportion of the cases in the active
file. Unless it follows @cmd{TEMPORARY}, it operates as a
-transformation, permanently removing cases from the active file.
+transformation, permanently removing cases from the active dataset.
The proportion to sample can be expressed as a single number between 0
-and 1. If @code{k} is the number specified, and @code{N} is the number
-of currently-selected cases in the active file, then after
-@code{SAMPLE @var{k}.}, approximately @code{k*N} cases will be
+and 1. If @var{k} is the number specified, and @var{N} is the number
+of currently-selected cases in the active dataset, then after
+@subcmd{SAMPLE @var{k}.}, approximately @var{k}*@var{N} cases are
selected.
-The proportion to sample can also be specified in the style @code{SAMPLE
+The proportion to sample can also be specified in the style @subcmd{SAMPLE
@var{m} FROM @var{N}}. With this style, cases are selected as follows:
@enumerate
@item
If @var{N} is equal to the number of currently-selected cases in the
-active file, exactly @var{m} cases will be selected.
+active dataset, exactly @var{m} cases are selected.
@item
If @var{N} is greater than the number of currently-selected cases in the
-active file, an equivalent proportion of cases will be selected.
+active dataset, an equivalent proportion of cases are selected.
@item
If @var{N} is less than the number of currently-selected cases in the
-active, exactly @var{m} cases will be selected @emph{from the first
-@var{N} cases in the active file.}
+active, exactly @var{m} cases are selected @emph{from the first
+@var{N} cases in the active dataset.}
@end enumerate
@cmd{SAMPLE} and @cmd{SELECT IF} are performed in
differing endianness or floating-point formats. By default, the
random number seed is based on the system time.
-@node SELECT IF, SPLIT FILE, SAMPLE, Data Selection
+@node SELECT IF
@section SELECT IF
@vindex SELECT IF
@display
-SELECT IF expression.
+SELECT IF @var{expression}.
@end display
-@cmd{SELECT IF} selects cases for analysis based on the value of a
-boolean expression. Cases not selected are permanently eliminated
-from the active file, unless @cmd{TEMPORARY} is in effect
+@cmd{SELECT IF} selects cases for analysis based on the value of
+@var{expression}. Cases not selected are permanently eliminated
+from the active dataset, unless @cmd{TEMPORARY} is in effect
(@pxref{TEMPORARY}).
Specify a boolean expression (@pxref{Expressions}). If the value of the
-expression is true for a particular case, the case will be analyzed. If
-the expression has a false or missing value, then the case will be
+expression is true for a particular case, the case is analyzed. If
+the expression has a false or missing value, then the case is
deleted from the data stream.
Place @cmd{SELECT IF} as early in the command file as
possible. Cases that are deleted early can be processed more
efficiently in time and space.
+Once cases have been deleted from the active dataset using @cmd{SELECT IF} they
+cannot be re-instated.
+If you want to be able to re-instate cases, then use @cmd{FILTER} (@pxref{FILTER})
+instead.
When @cmd{SELECT IF} is specified following @cmd{TEMPORARY}
(@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
(@pxref{LAG}).
-@node SPLIT FILE, TEMPORARY, SELECT IF, Data Selection
+@subsection Example Select-If
+
+A shop steward is interested in the salaries of younger personnel in a firm.
+The file @file{personnel.sav} provides the salaries of all the workers and their
+dates of birth. The syntax in @ref{select-if:ex} shows how @cmd{SELECT IF} can
+be used to limit analysis only to those persons born after December 31, 1999.
+
+@float Example, select-if:ex
+@psppsyntax {select-if.sps}
+@caption {Using @cmd{SELECT IF} to select persons born on or after a certain date.}
+@end float
+
+From @ref{select-if:res} one can see that there are 56 persons listed in the dataset,
+and 17 of them were born after December 31, 1999.
+
+@float Result, select-if:res
+@psppoutput {select-if}
+@caption {Salary descriptives before and after the @cmd{SELECT IF} transformation.}
+@end float
+
+Note that the @file{personnel.sav} file from which the data were read is unaffected.
+The transformation affects only the active file.
+
+@node SPLIT FILE
@section SPLIT FILE
@vindex SPLIT FILE
@display
-Two possible syntaxes:
- SPLIT FILE BY var_list.
- SPLIT FILE OFF.
+SPLIT FILE [@{LAYERED, SEPARATE@}] BY @var{var_list}.
+SPLIT FILE OFF.
@end display
@cmd{SPLIT FILE} allows multiple sets of data present in one data
An independent analysis is carried out for each group of cases, and the
variable values for the group are printed along with the analysis.
-Groups are formed only by @emph{adjacent} cases. To create a split
+When a list of variable names is specified, one of the keywords
+@subcmd{LAYERED} or @subcmd{SEPARATE} may also be specified. If provided, either
+keyword are ignored.
+
+Groups are formed only by @emph{adjacent} cases. To create a split
using a variable where like values are not adjacent in the working file,
you should first sort the data by that variable (@pxref{SORT CASES}).
-Specify OFF to disable @cmd{SPLIT FILE} and resume analysis of the
-entire active file as a single group of data.
+Specify @subcmd{OFF} to disable @cmd{SPLIT FILE} and resume analysis of the
+entire active dataset as a single group of data.
When @cmd{SPLIT FILE} is specified after @cmd{TEMPORARY}, it affects only
the next procedure (@pxref{TEMPORARY}).
-@node TEMPORARY, WEIGHT, SPLIT FILE, Data Selection
+@subsection Example Split
+
+The file @file{horticulture.sav} contains data describing the @exvar{yield}
+of a number of horticultural specimens which have been subjected to
+various @exvar{treatment}s. If we wanted to investigate linear statistics
+of the @exvar{yeild}, one way to do this is using the @cmd{DESCRIPTIVES} (@pxref{DESCRIPTIVES}).
+However, it is reasonable to expect the mean to be different depending
+on the @exvar{treatment}. So we might want to perform three separate
+procedures --- one for each treatment.
+@footnote{There are other, possibly better, ways to achieve a similar result
+using the @cmd{MEANS} or @cmd{EXAMINE} commands.}
+@ref{split:ex} shows how this can be done automatically using
+the @cmd{SPLIT FILE} command.
+
+@float Example, split:ex
+@psppsyntax {split.sps}
+@caption {Running @cmd{DESCRIPTIVES} on each value of @exvar{treatment}}
+@end float
+
+In @ref{split:res} you can see that the table of descriptive statistics
+appears 3 times --- once for each value of @exvar{treatment}.
+In this example @samp{N}, the number of observations are identical in
+all splits. This is because that experiment was deliberately designed
+that way. However in general one can expect a different @samp{N} for each
+split.
+
+@float Example, split:res
+@psppoutput {split}
+@caption {The results of running @cmd{DESCRIPTIVES} with an active split}
+@end float
+
+Unless @cmd{TEMPORARY} was used, after a split has been defined for
+a dataset it remains active until explicitly disabled.
+In the graphical user interface, the active split variable (if any) is
+displayed in the status bar (@pxref{split-status-bar:scr}.
+If a dataset is saved to a system file (@pxref{SAVE}) whilst a split
+is active, the split stastus is stored in the file and will be
+automatically loaded when that file is loaded.
+
+@float Screenshot, split-status-bar:scr
+@psppimage {split-status-bar}
+@caption {The status bar indicating that the data set is split using the @exvar{treatment} variable}
+@end float
+
+
+@node TEMPORARY
@section TEMPORARY
@vindex TEMPORARY
@end display
@cmd{TEMPORARY} is used to make the effects of transformations
-following its execution temporary. These transformations will
+following its execution temporary. These transformations
affect only the execution of the next procedure or procedure-like
-command. Their effects will not be saved to the active file.
+command. Their effects are not be saved to the active dataset.
The only specification on @cmd{TEMPORARY} is the command name.
Scratch variables cannot be used following @cmd{TEMPORARY}.
-An example may help to clarify:
-
-@example
-DATA LIST /X 1-2.
-BEGIN DATA.
- 2
- 4
-10
-15
-20
-24
-END DATA.
-COMPUTE X=X/2.
-TEMPORARY.
-COMPUTE X=X+3.
-DESCRIPTIVES X.
-DESCRIPTIVES X.
-@end example
+@subsection Example Temporary
+
+In @ref{temporary:ex} there are two @cmd{COMPUTE} transformation. One
+of them immediatly follows a @cmd{TEMPORARY} command, and therefore has
+effect only for the next procedure, which in this case is the first
+@cmd{DESCRIPTIVES} command.
-The data read by the first @cmd{DESCRIPTIVES} are 4, 5, 8,
-10.5, 13, 15. The data read by the first @cmd{DESCRIPTIVES} are 1, 2,
-5, 7.5, 10, 12.
+@float Example, temporary:ex
+@psppsyntax {temporary.sps}
+@caption {Running a @cmd{COMPUTE} transformation after @cmd{TEMPORARY}}
+@end float
-@node WEIGHT, , TEMPORARY, Data Selection
+The data read by the first @cmd{DESCRIPTIVES} procedure are 4, 5, 8,
+10.5, 13, 15. The data read by the second @cmd{DESCRIPTIVES} procedure are 1, 2,
+5, 7.5, 10, 12. This is because the second @cmd{COMPUTE} transformation
+has no effect on the second @cmd{DESCRIPTIVES} procedure. You can check these
+figures in @ref{temporary:res}.
+
+@float Result, temporary:res
+@psppoutput {temporary}
+@caption {The results of running two consecutive @cmd{DESCRIPTIVES} commands after
+ a temporary transformation}
+@end float
+
+
+@node WEIGHT
@section WEIGHT
@vindex WEIGHT
@display
-WEIGHT BY var_name.
+WEIGHT BY @var{var_name}.
WEIGHT OFF.
@end display
@cmd{WEIGHT} assigns cases varying weights,
-changing the frequency distribution of the active file. Execution of
+changing the frequency distribution of the active dataset. Execution of
@cmd{WEIGHT} is delayed until data have been read.
If a variable name is specified, @cmd{WEIGHT} causes the values of that
variable to be used as weighting factors for subsequent statistical
-procedures. Use of keyword BY is optional but recommended. Weighting
+procedures. Use of keyword @subcmd{BY} is optional but recommended. Weighting
variables must be numeric. Scratch variables may not be used for
weighting (@pxref{Scratch Variables}).
-When OFF is specified, subsequent statistical procedures will weight all
+When @subcmd{OFF} is specified, subsequent statistical procedures weight all
cases equally.
-A positive integer weighting factor @var{w} on a case will yield the
+A positive integer weighting factor @var{w} on a case yields the
same statistical output as would replicating the case @var{w} times.
A weighting factor of 0 is treated for statistical purposes as if the
case did not exist in the input. Weighting values need not be
When @cmd{WEIGHT} is specified after @cmd{TEMPORARY}, it affects only
the next procedure (@pxref{TEMPORARY}).
-@cmd{WEIGHT} does not cause cases in the active file to be replicated in
-memory.
-@setfilename ignored
+@cmd{WEIGHT} does not cause cases in the active dataset to be
+replicated in memory.
+
+
+@subsection Example Weights
+
+One could define a dataset containing an inventory of stock items.
+It would be reasonable to use a string variable for a description of the
+item, and a numeric variable for the number in stock, like in @ref{weight:ex}.
+
+@float Example, weight:ex
+@psppsyntax {weight.sps}
+@caption {Setting the weight on the variable @exvar{quantity}}
+@end float
+
+One analysis which most surely would be of interest is
+the relative amounts or each item in stock.
+However without setting a weight variable, @cmd{FREQUENCIES}
+(@pxref{FREQUENCIES}) does not tell us what we want to know, since
+there is only one case for each stock item. @ref{weight:res} shows the
+difference between the weighted and unweighted frequency tables.
+
+@float Example, weight:res
+@psppoutput {weight}
+@caption {Weighted and unweighted frequency tables of @exvar{items}}
+@end float
projects / pspp / blobdiff