1 @node Data Selection, Conditionals and Looping, Data Manipulation, Top
2 @chapter Selecting data for analysis
4 This chapter documents PSPP commands that temporarily or permanently
5 select data records from the active file for analysis.
8 * FILTER:: Exclude cases based on a variable.
9 * N OF CASES:: Limit the size of the active file.
10 * PROCESS IF:: Temporarily excluding cases.
11 * SAMPLE:: Select a specified proportion of cases.
12 * SELECT IF:: Permanently delete selected cases.
13 * SPLIT FILE:: Do multiple analyses with one command.
14 * TEMPORARY:: Make transformations' effects temporary.
15 * WEIGHT:: Weight cases by a variable.
18 @node FILTER, N OF CASES, Data Selection, Data Selection
27 @cmd{FILTER} allows a boolean-valued variable to be used to select
28 cases from the data stream for processing.
30 To set up filtering, specify BY and a variable name. Keyword
31 BY is optional but recommended. Cases which have a zero or system- or
32 user-missing value are excluded from analysis, but not deleted from the
33 data stream. Cases with other values are analyzed.
34 To filter based on a different condition, use
35 transformations such as @cmd{COMPUTE} or @cmd{RECODE} to compute a
36 filter variable of the required form, then specify that variable on
39 @code{FILTER OFF} turns off case filtering.
41 Filtering takes place immediately before cases pass to a procedure for
42 analysis. Only one filter variable may be active at a time. Normally,
43 case filtering continues until it is explicitly turned off with @code{FILTER
44 OFF}. However, if @cmd{FILTER} is placed after TEMPORARY, it filters only
45 the next procedure or procedure-like command.
47 @node N OF CASES, PROCESS IF, FILTER, Data Selection
52 N [OF CASES] num_of_cases [ESTIMATED].
55 Sometimes you may want to disregard cases of your input. @cmd{N} can
56 do this. @code{N 100} tells PSPP to disregard all cases after the
59 If the value specified for @cmd{N} is greater than the number of cases
60 read in, the value is ignored.
62 @cmd{N} does not discard cases or prevent them from being read. It
63 just causes cases beyond the last one specified to be ignored by data
66 A later @cmd{N} command can increase or decrease the number of cases
67 selected. (To select all the cases without knowing how many there are,
68 specify a very high number: 100000 or whatever you think is large enough.)
70 Transformation procedures performed after @cmd{N} is executed
71 @emph{do} cause cases to be discarded.
73 @cmd{SAMPLE}, @cmd{PROCESS IF}, and @cmd{SELECT IF} have
74 precedence over @cmd{N}---the same results are obtained by both of the
75 following fragments, given the same random number seeds:
78 @i{@dots{}set up, read in data@dots{}}
81 @i{@dots{}analyze data@dots{}}
83 @i{@dots{}set up, read in data@dots{}}
86 @i{@dots{}analyze data@dots{}}
89 Both fragments above first randomly sample approximately half of the
90 cases, then select the first 100 of those sampled.
92 @cmd{N} with the @code{ESTIMATED} keyword gives an
93 estimated number of cases before @cmd{DATA LIST} or another command to
94 read in data. @code{ESTIMATED} never limits the number of cases
95 processed by procedures. PSPP currently does not make use of
98 When @cmd{N} is specified after @cmd{TEMPORARY}, it affects only
99 the next procedure (@pxref{TEMPORARY}).
101 @node PROCESS IF, SAMPLE, N OF CASES, Data Selection
106 PROCESS IF expression.
109 @cmd{PROCESS IF} temporarily eliminates cases from the
110 data stream. Its effects are active only through the execution of the
111 next procedure or procedure-like command.
113 Specify a boolean expression (@pxref{Expressions}). If the value of the
114 expression is true for a particular case, the case will be analyzed. If
115 the expression has a false or missing value, then the case will be
116 deleted from the data stream for this procedure only.
118 Regardless of its placement relative to other commands, @cmd{PROCESS IF}
119 always takes effect immediately before data passes to the procedure.
120 Only one @cmd{PROCESS IF} command may be in effect at any given time.
122 The effects of @cmd{PROCESS IF} are similar, but not identical, to the
123 effects of executing @cmd{TEMPORARY}, then @cmd{SELECT IF}
126 The filtering performed by @cmd{PROCESS IF} takes place immediately
127 before cases pass to a procedure for analysis. Because @cmd{PROCESS
128 IF} affects only a single procedure, its placement relative to
129 @cmd{TEMPORARY} is unimportant.
131 @cmd{PROCESS IF} is deprecated. It is included for compatibility with
132 old command files. New syntax files should use @cmd{SELECT IF} or
133 @cmd{FILTER} instead.
135 @node SAMPLE, SELECT IF, PROCESS IF, Data Selection
140 SAMPLE num1 [FROM num2].
143 @cmd{SAMPLE} randomly samples a proportion of the cases in the active
144 file. Unless it follows @cmd{TEMPORARY}, it operates as a
145 transformation, permanently removing cases from the active file.
147 The proportion to sample can be expressed as a single number between 0
148 and 1. If @code{k} is the number specified, and @code{N} is the number
149 of currently-selected cases in the active file, then after
150 @code{SAMPLE @var{k}.}, approximately @code{k*N} cases will be
153 The proportion to sample can also be specified in the style @code{SAMPLE
154 @var{m} FROM @var{N}}. With this style, cases are selected as follows:
158 If @var{N} is equal to the number of currently-selected cases in the
159 active file, exactly @var{m} cases will be selected.
162 If @var{N} is greater than the number of currently-selected cases in the
163 active file, an equivalent proportion of cases will be selected.
166 If @var{N} is less than the number of currently-selected cases in the
167 active, exactly @var{m} cases will be selected @emph{from the first
168 @var{N} cases in the active file.}
171 @cmd{SAMPLE} and @cmd{SELECT IF} are performed in
172 the order specified by the syntax file.
174 @cmd{SAMPLE} is always performed before @code{N OF CASES}, regardless
175 of ordering in the syntax file (@pxref{N OF CASES}).
177 The same values for @cmd{SAMPLE} may result in different samples. To
178 obtain the same sample, use the @code{SET} command to set the random
179 number seed to the same value before each @cmd{SAMPLE}. Different
180 samples may still result when the file is processed on systems with
181 differing endianness or floating-point formats. By default, the
182 random number seed is based on the system time.
184 @node SELECT IF, SPLIT FILE, SAMPLE, Data Selection
189 SELECT IF expression.
192 @cmd{SELECT IF} selects cases for analysis based on the value of a
193 boolean expression. Cases not selected are permanently eliminated
194 from the active file, unless @cmd{TEMPORARY} is in effect
197 Specify a boolean expression (@pxref{Expressions}). If the value of the
198 expression is true for a particular case, the case will be analyzed. If
199 the expression has a false or missing value, then the case will be
200 deleted from the data stream.
202 Place @cmd{SELECT IF} as early in the command file as
203 possible. Cases that are deleted early can be processed more
204 efficiently in time and space.
206 When @cmd{SELECT IF} is specified following @cmd{TEMPORARY}
207 (@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
210 @node SPLIT FILE, TEMPORARY, SELECT IF, Data Selection
215 Two possible syntaxes:
216 SPLIT FILE BY var_list.
220 @cmd{SPLIT FILE} allows multiple sets of data present in one data
221 file to be analyzed separately using single statistical procedure
224 Specify a list of variable names to analyze multiple sets of
225 data separately. Groups of adjacent cases having the same values for these
226 variables are analyzed by statistical procedure commands as one group.
227 An independent analysis is carried out for each group of cases, and the
228 variable values for the group are printed along with the analysis.
230 Groups are formed only by @emph{adjacent} cases. To create a split
231 using a variable where like values are not adjacent in the working file,
232 you should first sort the data by that variable (@pxref{SORT CASES}).
234 Specify OFF to disable @cmd{SPLIT FILE} and resume analysis of the
235 entire active file as a single group of data.
237 When @cmd{SPLIT FILE} is specified after @cmd{TEMPORARY}, it affects only
238 the next procedure (@pxref{TEMPORARY}).
240 @node TEMPORARY, WEIGHT, SPLIT FILE, Data Selection
248 @cmd{TEMPORARY} is used to make the effects of transformations
249 following its execution temporary. These transformations will
250 affect only the execution of the next procedure or procedure-like
251 command. Their effects will not be saved to the active file.
253 The only specification on @cmd{TEMPORARY} is the command name.
255 @cmd{TEMPORARY} may not appear within a @cmd{DO IF} or @cmd{LOOP}
256 construct. It may appear only once between procedures and
257 procedure-like commands.
259 Scratch variables cannot be used following @cmd{TEMPORARY}.
261 An example may help to clarify:
280 The data read by the first @cmd{DESCRIPTIVES} are 4, 5, 8,
281 10.5, 13, 15. The data read by the first @cmd{DESCRIPTIVES} are 1, 2,
284 @node WEIGHT, , TEMPORARY, Data Selection
293 @cmd{WEIGHT} assigns cases varying weights,
294 changing the frequency distribution of the active file. Execution of
295 @cmd{WEIGHT} is delayed until data have been read.
297 If a variable name is specified, @cmd{WEIGHT} causes the values of that
298 variable to be used as weighting factors for subsequent statistical
299 procedures. Use of keyword BY is optional but recommended. Weighting
300 variables must be numeric. Scratch variables may not be used for
301 weighting (@pxref{Scratch Variables}).
303 When OFF is specified, subsequent statistical procedures will weight all
306 A positive integer weighting factor @var{w} on a case will yield the
307 same statistical output as would replicating the case @var{w} times.
308 A weighting factor of 0 is treated for statistical purposes as if the
309 case did not exist in the input. Weighting values need not be
310 integers, but negative and system-missing values for the weighting
311 variable are interpreted as weighting factors of 0. User-missing
312 values are not treated specially.
314 When @cmd{WEIGHT} is specified after @cmd{TEMPORARY}, it affects only
315 the next procedure (@pxref{TEMPORARY}).
317 @cmd{WEIGHT} does not cause cases in the active file to be replicated in