-@node Data Manipulation, Data Selection, Variable Attributes, Top
+@node Data Manipulation
@chapter Data transformations
@cindex transformations
* SORT CASES:: Sort the active file.
@end menu
-@node AGGREGATE, AUTORECODE, Data Manipulation, Data Manipulation
+@node AGGREGATE
@section AGGREGATE
@vindex AGGREGATE
@display
-AGGREGATE
- /BREAK=var_list
+AGGREGATE
+ OUTFILE=@{*,'file-name',file_handle@}
/PRESORTED
- /OUTFILE=@{*,'filename'@}
/DOCUMENT
/MISSING=COLUMNWISE
- /dest_vars=agr_func(src_vars, args@dots{})@dots{}
+ /BREAK=var_list
+ /dest_var['label']@dots{}=agr_func(src_vars, args@dots{})@dots{}
@end display
@cmd{AGGREGATE} summarizes groups of cases into single cases.
variables called @dfn{break variables}. Several functions are available
for summarizing case contents.
-At least one break variable must be specified on BREAK, the only
-required subcommand. The values of these variables are used to divide
-the active file into groups to be summarized. In addition, at least
-one @var{dest_var} must be specified.
+The OUTFILE subcommand is required and must appear first. Specify a
+system file, portable file, or scratch file by file name or file
+handle (@pxref{File Handles}).
+The aggregated cases are written to this file. If @samp{*} is
+specified, then the aggregated cases replace the active file. Use of
+OUTFILE to write a portable file or scratch file is a PSPP extension.
-By default, the active file is sorted based on the break variables
+By default, the active file will be sorted based on the break variables
before aggregation takes place. If the active file is already sorted
or otherwise grouped in terms of the break variables, specify
PRESORTED to save time.
-The OUTFILE subcommand specifies a system file by file name string or
-file handle (@pxref{FILE HANDLE}). The aggregated cases are written to
-this file. If OUTFILE is not specified, or if @samp{*} is specified,
-then the aggregated cases replace the active file.
-
Specify DOCUMENT to copy the documents from the active file into the
aggregate file (@pxref{DOCUMENT}). Otherwise, the aggregate file will
not contain any documents, even if the aggregate file replaces the
active file.
+Normally, only a single case (for SD and SD., two cases) need be
+non-missing in each group for the aggregate variable to be
+non-missing. Specifying /MISSING=COLUMNWISE inverts this behavior, so
+that the aggregate variable becomes missing if any aggregated value is
+missing.
+
+If PRESORTED, DOCUMENT, or MISSING are specified, they must appear
+between OUTFILE and BREAK.
+
+At least one break variable must be specified on BREAK, a
+required subcommand. The values of these variables are used to divide
+the active file into groups to be summarized. In addition, at least
+one @var{dest_var} must be specified.
+
One or more sets of aggregation variables must be specified. Each set
comprises a list of aggregation variables, an equals sign (@samp{=}),
the name of an aggregation function (see the list below), and a list
of source variables in parentheses. Some aggregation functions expect
additional arguments following the source variable names.
+Aggregation variables typically are created with no variable label,
+value labels, or missing values. Their default print and write
+formats depend on the aggregation function used, with details given in
+the table below. A variable label for an aggregation variable may be
+specified just after the variable's name in the aggregation variable
+list.
+
Each set must have exactly as many source variables as aggregation
variables. Each aggregation variable receives the results of applying
the specified aggregation function to the corresponding source
-variable. Most aggregation functions may be applied to numeric and
-short and long string variables. Others, marked below, are restricted
-to numeric values.
+variable. The MEAN, MEDIAN, SD, and SUM aggregation functions may only be
+applied to numeric variables. All the rest may be applied to numeric
+and string variables.
The available aggregation functions are as follows:
@table @asis
-@item SUM(var_name)
-Sum. Limited to numeric values.
-@item MEAN(var_name)
-Arithmetic mean. Limited to numeric values.
-@item SD(var_name)
-Standard deviation of the mean. Limited to numeric values.
-@item MAX(var_name)
-Maximum value.
-@item MIN(var_name)
-Minimum value.
@item FGT(var_name, value)
-@itemx PGT(var_name, value)
-Fraction between 0 and 1, or percentage between 0 and 100, respectively,
-of values greater than the specified constant.
-@item FLT(var_name, value)
-@itemx PLT(var_name, value)
-Fraction or percentage, respectively, of values less than the specified
-constant.
+Fraction of values greater than the specified constant. The default
+format is F5.3.
+
@item FIN(var_name, low, high)
-@itemx PIN(var_name, low, high)
-Fraction or percentage, respectively, of values within the specified
-inclusive range of constants.
+Fraction of values within the specified inclusive range of constants.
+The default format is F5.3.
+
+@item FLT(var_name, value)
+Fraction of values less than the specified constant. The default
+format is F5.3.
+
+@item FIRST(var_name)
+First non-missing value in break group. The aggregation variable
+receives the complete dictionary information from the source variable.
+The sort performed by AGGREGATE (and by SORT CASES) is stable, so that
+the first case with particular values for the break variables before
+sorting will also be the first case in that break group after sorting.
+
@item FOUT(var_name, low, high)
-@itemx POUT(var_name, low, high)
-Fraction or percentage, respectively, of values strictly outside the
-specified range of constants.
+Fraction of values strictly outside the specified range of constants.
+The default format is F5.3.
+
+@item LAST(var_name)
+Last non-missing value in break group. The aggregation variable
+receives the complete dictionary information from the source variable.
+The sort performed by AGGREGATE (and by SORT CASES) is stable, so that
+the last case with particular values for the break variables before
+sorting will also be the last case in that break group after sorting.
+
+@item MAX(var_name)
+Maximum value. The aggregation variable receives the complete
+dictionary information from the source variable.
+
+@item MEAN(var_name)
+Arithmetic mean. Limited to numeric values. The default format is
+F8.2.
+
+@item MEDIAN(var_name)
+The median value. Limited to numeric values. The default format is F8.2.
+
+@item MIN(var_name)
+Minimum value. The aggregation variable receives the complete
+dictionary information from the source variable.
+
@item N(var_name)
-Number of non-missing values.
+Number of non-missing values. The default format is F7.0 if weighting
+is not enabled, F8.2 if it is (@pxref{WEIGHT}).
+
@item N
-Number of cases aggregated to form this group. Don't supply a source
-variable for this aggregation function.
+Number of cases aggregated to form this group. The default format is
+F7.0 if weighting is not enabled, F8.2 if it is (@pxref{WEIGHT}).
+
+@item NMISS(var_name)
+Number of missing values. The default format is F7.0 if weighting is
+not enabled, F8.2 if it is (@pxref{WEIGHT}).
+
@item NU(var_name)
Number of non-missing values. Each case is considered to have a weight
of 1, regardless of the current weighting variable (@pxref{WEIGHT}).
+The default format is F7.0.
+
@item NU
Number of cases aggregated to form this group. Each case is considered
to have a weight of 1, regardless of the current weighting variable.
-@item NMISS(var_name)
-Number of missing values.
+The default format is F7.0.
+
@item NUMISS(var_name)
Number of missing values. Each case is considered to have a weight of
-1, regardless of the current weighting variable.
-@item FIRST(var_name)
-First value in this group.
-@item LAST(var_name)
-Last value in this group.
+1, regardless of the current weighting variable. The default format is F7.0.
+
+@item PGT(var_name, value)
+Percentage between 0 and 100 of values greater than the specified
+constant. The default format is F5.1.
+
+@item PIN(var_name, low, high)
+Percentage of values within the specified inclusive range of
+constants. The default format is F5.1.
+
+@item PLT(var_name, value)
+Percentage of values less than the specified constant. The default
+format is F5.1.
+
+@item POUT(var_name, low, high)
+Percentage of values strictly outside the specified range of
+constants. The default format is F5.1.
+
+@item SD(var_name)
+Standard deviation of the mean. Limited to numeric values. The
+default format is F8.2.
+
+@item SUM(var_name)
+Sum. Limited to numeric values. The default format is F8.2.
@end table
Aggregation functions compare string values in terms of internal
The aggregation functions listed above exclude all user-missing values
from calculations. To include user-missing values, insert a period
-(@samp{.}) between the function name and left parenthesis
-(e.g.@: @samp{SUM.}).
-
-Normally, only a single case (for SD and SD., two cases) need be
-non-missing in each group for the aggregate variable to be
-non-missing. Specifying /MISSING=COLUMNWISE inverts this behavior, so
-that the aggregate variable becomes missing if any aggregated value is
-missing.
+(@samp{.}) at the end of the function name. (e.g.@: @samp{SUM.}).
+(Be aware that specifying such a function as the last token on a line
+will cause the period to be interpreted as the end of the command.)
@cmd{AGGREGATE} both ignores and cancels the current @cmd{SPLIT FILE}
settings (@pxref{SPLIT FILE}).
-@node AUTORECODE, COMPUTE, AGGREGATE, Data Manipulation
+@node AUTORECODE
@section AUTORECODE
@vindex AUTORECODE
@cmd{AUTORECODE} is a procedure. It causes the data to be read.
-@node COMPUTE, COUNT, AUTORECODE, Data Manipulation
+@node COMPUTE
@section COMPUTE
@vindex COMPUTE
@cmd{COMPUTE} assigns the value of an expression to a target
variable. For each case, the expression is evaluated and its value
-assigned to the target variable. Numeric and short and long string
+assigned to the target variable. Numeric and string
variables may be assigned. When a string expression's width differs
from the target variable's width, the string result of the expression
is truncated or padded with spaces on the right as necessary. The
(@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
(@pxref{LAG}).
-@node COUNT, FLIP, COMPUTE, Data Manipulation
+@node COUNT
@section COUNT
@vindex COUNT
The target variable values are always nonnegative integers. They are
never missing. The target variable is assigned an F8.2 output format.
-@xref{Input/Output Formats}. Any variables, including long and short
+@xref{Input and Output Formats}. Any variables, including
string variables, may be test variables.
User-missing values of test variables are treated just like any other
@end example
@end enumerate
-@node FLIP, IF, COUNT, Data Manipulation
+@node FLIP
@section FLIP
@vindex FLIP
All variables in the transposed active file are numeric. String
variables take on the system-missing value in the transposed file.
-No subcommands are required. The VARIABLES subcommand specifies
-variables that will be transformed into cases. Variables not specified
-are discarded. By default, all variables are selected for
-transposition.
+No subcommands are required. If specified, the VARIABLES subcommand
+selects variables to be transformed into cases, and variables not
+specified are discarded. If the VARIABLES subcommand is omitted, all
+variables are selected for transposition.
The variables specified by NEWNAMES, which must be a string variable, is
-used to give names to the variables created by @cmd{FLIP}. If
+used to give names to the variables created by @cmd{FLIP}. Only the
+first 8 characters of the variable are used. If
NEWNAMES is not
specified then the default is a variable named CASE_LBL, if it exists.
If it does not then the variables created by FLIP are named VAR000
there are no remaining possibilities. If the latter occurs then the
FLIP operation aborts.
-The resultant dictionary contains a CASE_LBL variable, which stores the
-names of the variables in the dictionary before the transposition. If
-the active file is subsequently transposed using @cmd{FLIP}, this
-variable can
-be used to recreate the original variable names.
+The resultant dictionary contains a CASE_LBL variable, a string
+variable of width 8, which stores the names of the variables in the
+dictionary before the transposition. Variables names longer than 8
+characters are truncated. If the active file is subsequently
+transposed using @cmd{FLIP}, this variable can be used to recreate the
+original variable names.
-FLIP honors N OF CASES. It ignores TEMPORARY, so that ``temporary''
+FLIP honors @cmd{N OF CASES} (@pxref{N OF CASES}). It ignores
+@cmd{TEMPORARY} (@pxref{TEMPORARY}), so that ``temporary''
transformations become permanent.
-@node IF, RECODE, FLIP, Data Manipulation
+@node IF
@section IF
@vindex IF
following the IF keyword. This expression is evaluated for each case.
If the value is true, then the value of the expression is computed and
assigned to the specified variable. If the value is false or missing,
-nothing is done. Numeric and short and long string variables may be
+nothing is done. Numeric and string variables may be
assigned. When a string expression's width differs from the target
variable's width, the string result of the expression is truncated or
padded with spaces on the right as necessary. The expression and
(@pxref{TEMPORARY}), the @cmd{LAG} function may not be used
(@pxref{LAG}).
-@node RECODE, SORT CASES, IF, Data Manipulation
+@node RECODE
@section RECODE
@vindex RECODE
@cmd{RECODE} translates data from one range of values to
another, via flexible user-specified mappings. Data may be remapped
-in-place or copied to new variables. Numeric, short string, and long
+in-place or copied to new variables. Numeric and
string data can be recoded.
Specify the list of source variables, followed by one or more mapping
system missing value only. ELSE is a catch-all that matches anything.
It should be the last src_value specified.
-Numeric and string dest_value's should also be self-explanatory. COPY
-causes the input values to be copied to the output. This is only value
+Numeric and string dest_value's should be self-explanatory. COPY
+causes the input values to be copied to the output. This is only valid
if the source and target variables are of the same type. SYSMIS
indicates the system-missing value.
Introduce additional recodings with a slash (@samp{/}) to
separate them from the previous recodings.
-@node SORT CASES, , RECODE, Data Manipulation
+@node SORT CASES
@section SORT CASES
@vindex SORT CASES
@display
-SORT CASES BY var_list.
+SORT CASES BY var_list[(@{D|A@}] [ var_list[(@{D|A@}] ] ...
@end display
@cmd{SORT CASES} sorts the active file by the values of one or more
Specify BY and a list of variables to sort by. By default, variables
are sorted in ascending order. To override sort order, specify (D) or
(DOWN) after a list of variables to get descending order, or (A) or (UP)
-for ascending order. These apply to the entire list of variables
-preceding them.
+for ascending order. These apply to all the listed variables
+up until the preceding (A), (D), (UP) or (DOWN).
+
+The sort algorithms used by @cmd{SORT CASES} are stable. That is,
+records that have equal values of the sort variables will have the
+same relative order before and after sorting. As a special case,
+re-sorting an already sorted file will not affect the ordering of
+cases.
@cmd{SORT CASES} is a procedure. It causes the data to be read.
@cmd{SORT CASES} attempts to sort the entire active file in main memory.
-If main memory is exhausted, it falls back to a merge sort algorithm that
-involves writing and reading numerous temporary files.
+If workspace is exhausted, it falls back to a merge sort algorithm that
+involves creates numerous temporary files.
@cmd{SORT CASES} may not be specified following TEMPORARY.
-@setfilename ignored
projects / pspp-builds.git / blobdiff