1 @node Data Manipulation, Data Selection, Variable Attributes, Top
2 @chapter Data transformations
3 @cindex transformations
5 The PSPP procedures examined in this chapter manipulate data and
6 prepare the active file for later analyses. They do not produce output,
10 * AGGREGATE:: Summarize multiple cases into a single case.
11 * AUTORECODE:: Automatic recoding of variables.
12 * COMPUTE:: Assigning a variable a calculated value.
13 * COUNT:: Counting variables with particular values.
14 * FLIP:: Exchange variables with cases.
15 * IF:: Conditionally assigning a calculated value.
16 * RECODE:: Mapping values from one set to another.
17 * SORT CASES:: Sort the active file.
20 @node AGGREGATE, AUTORECODE, Data Manipulation, Data Manipulation
26 OUTFILE=@{*,'filename'@}
31 /dest_var['label']@dots{}=agr_func(src_vars, args@dots{})@dots{}
34 @cmd{AGGREGATE} summarizes groups of cases into single cases.
35 Cases are divided into groups that have the same values for one or more
36 variables called @dfn{break variables}. Several functions are available
37 for summarizing case contents.
39 The OUTFILE subcommand is required and must appear first. Specify a
40 system file by file name string or file handle (@pxref{FILE HANDLE}).
41 The aggregated cases are written to this file. If @samp{*} is
42 specified, then the aggregated cases replace the active file.
44 By default, the active file will be sorted based on the break variables
45 before aggregation takes place. If the active file is already sorted
46 or otherwise grouped in terms of the break variables, specify
47 PRESORTED to save time.
49 Specify DOCUMENT to copy the documents from the active file into the
50 aggregate file (@pxref{DOCUMENT}). Otherwise, the aggregate file will
51 not contain any documents, even if the aggregate file replaces the
54 Normally, only a single case (for SD and SD., two cases) need be
55 non-missing in each group for the aggregate variable to be
56 non-missing. Specifying /MISSING=COLUMNWISE inverts this behavior, so
57 that the aggregate variable becomes missing if any aggregated value is
60 If PRESORTED, DOCUMENT, or MISSING are specified, they must appear
61 between OUTFILE and BREAK.
63 At least one break variable must be specified on BREAK, a
64 required subcommand. The values of these variables are used to divide
65 the active file into groups to be summarized. In addition, at least
66 one @var{dest_var} must be specified.
68 One or more sets of aggregation variables must be specified. Each set
69 comprises a list of aggregation variables, an equals sign (@samp{=}),
70 the name of an aggregation function (see the list below), and a list
71 of source variables in parentheses. Some aggregation functions expect
72 additional arguments following the source variable names.
74 Aggregation variables typically are created with no variable label,
75 value labels, or missing values. Their default print and write
76 formats depend on the aggregation function used, with details given in
77 the table below. A variable label for an aggregation variable may be
78 specified just after the variable's name in the aggregation variable
81 Each set must have exactly as many source variables as aggregation
82 variables. Each aggregation variable receives the results of applying
83 the specified aggregation function to the corresponding source
84 variable. The MEAN, SD, and SUM aggregation functions may only be
85 applied to numeric variables. All the rest may be applied to numeric
86 and short and long string variables.
88 The available aggregation functions are as follows:
91 @item FGT(var_name, value)
92 Fraction of values greater than the specified constant. The default
95 @item FIN(var_name, low, high)
96 Fraction of values within the specified inclusive range of constants.
97 The default format is F5.3.
99 @item FLT(var_name, value)
100 Fraction of values less than the specified constant. The default
103 @item FIRST(var_name)
104 First non-missing value in break group. The aggregation variable
105 receives the complete dictionary information from the source variable.
107 @item FOUT(var_name, low, high)
108 Fraction of values strictly outside the specified range of constants.
109 The default format is F5.3.
112 Last non-missing value in break group. The aggregation variable
113 receives the complete dictionary information from the source variable.
116 Maximum value. The aggregation variable receives the complete
117 dictionary information from the source variable.
120 Arithmetic mean. Limited to numeric values. The default format is
124 Minimum value. The aggregation variable receives the complete
125 dictionary information from the source variable.
128 Number of non-missing values. The default format is F7.0 if weighting
129 is not enabled, F8.2 if it is (@pxref{WEIGHT}).
132 Number of cases aggregated to form this group. The default format is
133 F7.0 if weighting is not enabled, F8.2 if it is (@pxref{WEIGHT}).
135 @item NMISS(var_name)
136 Number of missing values. The default format is F7.0 if weighting is
137 not enabled, F8.2 if it is (@pxref{WEIGHT}).
140 Number of non-missing values. Each case is considered to have a weight
141 of 1, regardless of the current weighting variable (@pxref{WEIGHT}).
142 The default format is F7.0.
145 Number of cases aggregated to form this group. Each case is considered
146 to have a weight of 1, regardless of the current weighting variable.
147 The default format is F7.0.
149 @item NUMISS(var_name)
150 Number of missing values. Each case is considered to have a weight of
151 1, regardless of the current weighting variable. The default format is F7.0.
153 @item PGT(var_name, value)
154 Percentage between 0 and 100 of values greater than the specified
155 constant. The default format is F5.1.
157 @item PIN(var_name, low, high)
158 Percentage of values within the specified inclusive range of
159 constants. The default format is F5.1.
161 @item PLT(var_name, value)
162 Percentage of values less than the specified constant. The default
165 @item POUT(var_name, low, high)
166 Percentage of values strictly outside the specified range of
167 constants. The default format is F5.1.
170 Standard deviation of the mean. Limited to numeric values. The
171 default format is F8.2.
174 Sum. Limited to numeric values. The default format is F8.2.
177 Aggregation functions compare string values in terms of internal
178 character codes. On most modern computers, this is a form of ASCII.
180 The aggregation functions listed above exclude all user-missing values
181 from calculations. To include user-missing values, insert a period
182 (@samp{.}) between the function name and left parenthesis
183 (e.g.@: @samp{SUM.}).
185 @cmd{AGGREGATE} both ignores and cancels the current @cmd{SPLIT FILE}
186 settings (@pxref{SPLIT FILE}).
188 @node AUTORECODE, COMPUTE, AGGREGATE, Data Manipulation
193 AUTORECODE VARIABLES=src_vars INTO dest_vars
198 The @cmd{AUTORECODE} procedure considers the @var{n} values that a variable
199 takes on and maps them onto values 1@dots{}@var{n} on a new numeric
202 Subcommand VARIABLES is the only required subcommand and must come
203 first. Specify VARIABLES, an equals sign (@samp{=}), a list of source
204 variables, INTO, and a list of target variables. There must the same
205 number of source and target variables. The target variables must not
208 By default, increasing values of a source variable (for a string, this
209 is based on character code comparisons) are recoded to increasing values
210 of its target variable. To cause increasing values of a source variable
211 to be recoded to decreasing values of its target variable (@var{n} down
212 to 1), specify DESCENDING.
214 PRINT is currently ignored.
216 @cmd{AUTORECODE} is a procedure. It causes the data to be read.
218 @node COMPUTE, COUNT, AUTORECODE, Data Manipulation
223 COMPUTE variable = expression.
225 COMPUTE vector(index) = expression.
228 @cmd{COMPUTE} assigns the value of an expression to a target
229 variable. For each case, the expression is evaluated and its value
230 assigned to the target variable. Numeric and short and long string
231 variables may be assigned. When a string expression's width differs
232 from the target variable's width, the string result of the expression
233 is truncated or padded with spaces on the right as necessary. The
234 expression and variable types must match.
236 For numeric variables only, the target variable need not already
237 exist. Numeric variables created by @cmd{COMPUTE} are assigned an
238 @code{F8.2} output format. String variables must be declared before
239 they can be used as targets for @cmd{COMPUTE}.
241 The target variable may be specified as an element of a vector
242 (@pxref{VECTOR}). In this case, a vector index expression must be
243 specified in parentheses following the vector name. The index
244 expression must evaluate to a numeric value that, after rounding down
245 to the nearest integer, is a valid index for the named vector.
247 Using @cmd{COMPUTE} to assign to a variable specified on @cmd{LEAVE}
248 (@pxref{LEAVE}) resets the variable's left state. Therefore,
249 @code{LEAVE} should be specified following @cmd{COMPUTE}, not before.
251 @cmd{COMPUTE} is a transformation. It does not cause the active file to be
254 When @cmd{COMPUTE} is specified following @cmd{TEMPORARY}
255 (@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
258 @node COUNT, FLIP, COMPUTE, Data Manipulation
263 COUNT var_name = var@dots{} (value@dots{}).
265 Each value takes one of the following forms:
271 In addition, num1 and num2 can be LO or LOWEST, or HI or HIGHEST,
275 @cmd{COUNT} creates or replaces a numeric @dfn{target} variable that
276 counts the occurrence of a @dfn{criterion} value or set of values over
277 one or more @dfn{test} variables for each case.
279 The target variable values are always nonnegative integers. They are
280 never missing. The target variable is assigned an F8.2 output format.
281 @xref{Input/Output Formats}. Any variables, including long and short
282 string variables, may be test variables.
284 User-missing values of test variables are treated just like any other
285 values. They are @strong{not} treated as system-missing values.
286 User-missing values that are criterion values or inside ranges of
287 criterion values are counted as any other values. However (for numeric
288 variables), keyword MISSING may be used to refer to all system-
289 and user-missing values.
291 @cmd{COUNT} target variables are assigned values in the order
292 specified. In the command @code{COUNT A=A B(1) /B=A B(2).}, the
293 following actions occur:
297 The number of occurrences of 1 between @code{A} and @code{B} is counted.
300 @code{A} is assigned this value.
303 The number of occurrences of 1 between @code{B} and the @strong{new}
304 value of @code{A} is counted.
307 @code{B} is assigned this value.
310 Despite this ordering, all @cmd{COUNT} criterion variables must exist
311 before the procedure is executed---they may not be created as target
312 variables earlier in the command! Break such a command into two
315 The examples below may help to clarify.
319 Assuming @code{Q0}, @code{Q2}, @dots{}, @code{Q9} are numeric variables,
320 the following commands:
324 Count the number of times the value 1 occurs through these variables
325 for each case and assigns the count to variable @code{QCOUNT}.
328 Print out the total number of times the value 1 occurs throughout
329 @emph{all} cases using @cmd{DESCRIPTIVES}. @xref{DESCRIPTIVES}, for
334 COUNT QCOUNT=Q0 TO Q9(1).
335 DESCRIPTIVES QCOUNT /STATISTICS=SUM.
339 Given these same variables, the following commands:
343 Count the number of valid values of these variables for each case and
344 assigns the count to variable @code{QVALID}.
347 Multiplies each value of @code{QVALID} by 10 to obtain a percentage of
348 valid values, using @cmd{COMPUTE}. @xref{COMPUTE}, for details.
351 Print out the percentage of valid values across all cases, using
352 @cmd{DESCRIPTIVES}. @xref{DESCRIPTIVES}, for details.
356 COUNT QVALID=Q0 TO Q9 (LO THRU HI).
357 COMPUTE QVALID=QVALID*10.
358 DESCRIPTIVES QVALID /STATISTICS=MEAN.
362 @node FLIP, IF, COUNT, Data Manipulation
367 FLIP /VARIABLES=var_list /NEWNAMES=var_name.
370 @cmd{FLIP} transposes rows and columns in the active file. It
371 causes cases to be swapped with variables, and vice versa.
373 All variables in the transposed active file are numeric. String
374 variables take on the system-missing value in the transposed file.
376 No subcommands are required. The VARIABLES subcommand specifies
377 variables that will be transformed into cases. Variables not specified
378 are discarded. By default, all variables are selected for
381 The variables specified by NEWNAMES, which must be a string variable, is
382 used to give names to the variables created by @cmd{FLIP}. If
384 specified then the default is a variable named CASE_LBL, if it exists.
385 If it does not then the variables created by FLIP are named VAR000
386 through VAR999, then VAR1000, VAR1001, and so on.
388 When a NEWNAMES variable is available, the names must be canonicalized
389 before becoming variable names. Invalid characters are replaced by
390 letter @samp{V} in the first position, or by @samp{_} in subsequent
391 positions. If the name thus generated is not unique, then numeric
392 extensions are added, starting with 1, until a unique name is found or
393 there are no remaining possibilities. If the latter occurs then the
394 FLIP operation aborts.
396 The resultant dictionary contains a CASE_LBL variable, which stores the
397 names of the variables in the dictionary before the transposition. If
398 the active file is subsequently transposed using @cmd{FLIP}, this
400 be used to recreate the original variable names.
402 FLIP honors N OF CASES. It ignores TEMPORARY, so that ``temporary''
403 transformations become permanent.
405 @node IF, RECODE, FLIP, Data Manipulation
410 IF condition variable=expression.
412 IF condition vector(index)=expression.
415 The @cmd{IF} transformation conditionally assigns the value of a target
416 expression to a target variable, based on the truth of a test
419 Specify a boolean-valued expression (@pxref{Expressions}) to be tested
420 following the IF keyword. This expression is evaluated for each case.
421 If the value is true, then the value of the expression is computed and
422 assigned to the specified variable. If the value is false or missing,
423 nothing is done. Numeric and short and long string variables may be
424 assigned. When a string expression's width differs from the target
425 variable's width, the string result of the expression is truncated or
426 padded with spaces on the right as necessary. The expression and
427 variable types must match.
429 The target variable may be specified as an element of a vector
430 (@pxref{VECTOR}). In this case, a vector index expression must be
431 specified in parentheses following the vector name. The index
432 expression must evaluate to a numeric value that, after rounding down
433 to the nearest integer, is a valid index for the named vector.
435 Using @cmd{IF} to assign to a variable specified on @cmd{LEAVE}
436 (@pxref{LEAVE}) resets the variable's left state. Therefore,
437 @code{LEAVE} should be specified following @cmd{IF}, not before.
439 When @cmd{IF} is specified following @cmd{TEMPORARY}
440 (@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
443 @node RECODE, SORT CASES, IF, Data Manipulation
448 RECODE var_list (src_value@dots{}=dest_value)@dots{} [INTO var_list].
450 src_value may take the following forms:
457 Open-ended ranges may be specified using LO or LOWEST for num1
458 or HI or HIGHEST for num2.
460 dest_value may take the following forms:
467 @cmd{RECODE} translates data from one range of values to
468 another, via flexible user-specified mappings. Data may be remapped
469 in-place or copied to new variables. Numeric, short string, and long
470 string data can be recoded.
472 Specify the list of source variables, followed by one or more mapping
473 specifications each enclosed in parentheses. If the data is to be
474 copied to new variables, specify INTO, then the list of target
475 variables. String target variables must already have been declared
476 using @cmd{STRING} or another transformation, but numeric target
478 be created on the fly. There must be exactly as many target variables
479 as source variables. Each source variable is remapped into its
480 corresponding target variable.
482 When INTO is not used, the input and output variables must be of the
483 same type. Otherwise, string values can be recoded into numeric values,
484 and vice versa. When this is done and there is no mapping for a
485 particular value, either a value consisting of all spaces or the
486 system-missing value is assigned, depending on variable type.
488 Mappings are considered from left to right. The first src_value that
489 matches the value of the source variable causes the target variable to
490 receive the value indicated by the dest_value. Literal number, string,
491 and range src_value's should be self-explanatory. MISSING as a
492 src_value matches any user- or system-missing value. SYSMIS matches the
493 system missing value only. ELSE is a catch-all that matches anything.
494 It should be the last src_value specified.
496 Numeric and string dest_value's should also be self-explanatory. COPY
497 causes the input values to be copied to the output. This is only value
498 if the source and target variables are of the same type. SYSMIS
499 indicates the system-missing value.
501 If the source variables are strings and the target variables are
502 numeric, then there is one additional mapping available: (CONVERT),
503 which must be the last specified mapping. CONVERT causes a number
504 specified as a string to be converted to a numeric value. If the string
505 cannot be parsed as a number, then the system-missing value is assigned.
507 Multiple recodings can be specified on a single @cmd{RECODE} invocation.
508 Introduce additional recodings with a slash (@samp{/}) to
509 separate them from the previous recodings.
511 @node SORT CASES, , RECODE, Data Manipulation
516 SORT CASES BY var_list.
519 @cmd{SORT CASES} sorts the active file by the values of one or more
522 Specify BY and a list of variables to sort by. By default, variables
523 are sorted in ascending order. To override sort order, specify (D) or
524 (DOWN) after a list of variables to get descending order, or (A) or (UP)
525 for ascending order. These apply to the entire list of variables
528 @cmd{SORT CASES} is a procedure. It causes the data to be read.
530 @cmd{SORT CASES} attempts to sort the entire active file in main memory.
531 If main memory is exhausted, it falls back to a merge sort algorithm that
532 involves writing and reading numerous temporary files.
534 @cmd{SORT CASES} may not be specified following TEMPORARY.