X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=doc%2Ftransformation.texi;h=74d972a2e22599534a82a82b1599639fcb283200;hb=6ef257ae34edc6d1022f02bd92de9781170d9325;hp=960e782a0cef785363a77974e118c3f0704b98d9;hpb=1b1837591924226078c96db15888b68beec2ef6d;p=pspp diff --git a/doc/transformation.texi b/doc/transformation.texi index 960e782a0c..74d972a2e2 100644 --- a/doc/transformation.texi +++ b/doc/transformation.texi @@ -1,3 +1,12 @@ +@c PSPP - a program for statistical analysis. +@c Copyright (C) 2017 Free Software Foundation, Inc. +@c Permission is granted to copy, distribute and/or modify this document +@c under the terms of the GNU Free Documentation License, Version 1.3 +@c or any later version published by the Free Software Foundation; +@c with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. +@c A copy of the license is included in the section entitled "GNU +@c Free Documentation License". +@c @node Data Manipulation @chapter Data transformations @cindex transformations @@ -22,7 +31,7 @@ as a rule. @vindex AGGREGATE @display -AGGREGATE +AGGREGATE OUTFILE=@{*,'@var{file_name}',@var{file_handle}@} [MODE=@{REPLACE, ADDVARIABLES@}] /PRESORTED /DOCUMENT @@ -42,16 +51,16 @@ handle (@pxref{File Handles}), or a dataset by its name (@pxref{Datasets}). The aggregated cases are written to this file. If @samp{*} is specified, then the aggregated cases replace the active dataset's data. -Use of OUTFILE to write a portable file is a @pspp{} extension. +Use of @subcmd{OUTFILE} to write a portable file is a @pspp{} extension. -If OUTFILE=@samp{*} is given, then the subcommand MODE may also be +If @subcmd{OUTFILE=*} is given, then the subcommand @subcmd{MODE} may also be specified. The mode subcommand has two possible values: @subcmd{ADDVARIABLES} or @subcmd{REPLACE}. In @subcmd{REPLACE} mode, the entire active dataset is replaced by a new dataset which contains just the break variables and the destination varibles. In this mode, the new file will contain as many cases as there are unique combinations of the break variables. -In @subcmd{ADDVARIABLES} mode, the destination variables will be appended to +In @subcmd{ADDVARIABLES} mode, the destination variables will be appended to the existing active dataset. Cases which have identical combinations of values in their break variables, will receive identical values for the destination variables. @@ -100,110 +109,112 @@ list. Each set must have exactly as many source variables as aggregation variables. Each aggregation variable receives the results of applying the specified aggregation function to the corresponding source -variable. The MEAN, MEDIAN, SD, and SUM aggregation functions may only be +variable. The @subcmd{MEAN}, @subcmd{MEDIAN}, @subcmd{SD}, and @subcmd{SUM} +aggregation functions may only be applied to numeric variables. All the rest may be applied to numeric and string variables. The available aggregation functions are as follows: @table @asis -@item FGT(@var{var_name}, @var{value}) +@item @subcmd{FGT(@var{var_name}, @var{value})} Fraction of values greater than the specified constant. The default format is F5.3. -@item FIN(@var{var_name}, @var{low}, @var{high}) +@item @subcmd{FIN(@var{var_name}, @var{low}, @var{high})} Fraction of values within the specified inclusive range of constants. The default format is F5.3. -@item FLT(@var{var_name}, @var{value}) +@item @subcmd{FLT(@var{var_name}, @var{value})} Fraction of values less than the specified constant. The default format is F5.3. -@item FIRST(@var{var_name}) +@item @subcmd{FIRST(@var{var_name})} First non-missing value in break group. The aggregation variable receives the complete dictionary information from the source variable. The sort performed by @cmd{AGGREGATE} (and by @cmd{SORT CASES}) is stable, so that the first case with particular values for the break variables before sorting will also be the first case in that break group after sorting. -@item FOUT(@var{var_name}, @var{low}, @var{high}) +@item @subcmd{FOUT(@var{var_name}, @var{low}, @var{high})} Fraction of values strictly outside the specified range of constants. The default format is F5.3. -@item LAST(@var{var_name}) +@item @subcmd{LAST(@var{var_name})} Last non-missing value in break group. The aggregation variable receives the complete dictionary information from the source variable. The sort performed by @cmd{AGGREGATE} (and by @cmd{SORT CASES}) is stable, so that the last case with particular values for the break variables before sorting will also be the last case in that break group after sorting. -@item MAX(@var{var_name}) +@item @subcmd{MAX(@var{var_name})} Maximum value. The aggregation variable receives the complete dictionary information from the source variable. -@item MEAN(@var{var_name}) +@item @subcmd{MEAN(@var{var_name})} Arithmetic mean. Limited to numeric values. The default format is F8.2. -@item MEDIAN(@var{var_name}) +@item @subcmd{MEDIAN(@var{var_name})} The median value. Limited to numeric values. The default format is F8.2. -@item MIN(@var{var_name}) +@item @subcmd{MIN(@var{var_name})} Minimum value. The aggregation variable receives the complete dictionary information from the source variable. -@item N(@var{var_name}) +@item @subcmd{N(@var{var_name})} Number of non-missing values. The default format is F7.0 if weighting is not enabled, F8.2 if it is (@pxref{WEIGHT}). -@item N +@item @subcmd{N} Number of cases aggregated to form this group. The default format is F7.0 if weighting is not enabled, F8.2 if it is (@pxref{WEIGHT}). -@item NMISS(@var{var_name}) +@item @subcmd{NMISS(@var{var_name})} Number of missing values. The default format is F7.0 if weighting is not enabled, F8.2 if it is (@pxref{WEIGHT}). -@item NU(@var{var_name}) +@item @subcmd{NU(@var{var_name})} Number of non-missing values. Each case is considered to have a weight of 1, regardless of the current weighting variable (@pxref{WEIGHT}). The default format is F7.0. -@item NU +@item @subcmd{NU} Number of cases aggregated to form this group. Each case is considered to have a weight of 1, regardless of the current weighting variable. The default format is F7.0. -@item NUMISS(@var{var_name}) +@item @subcmd{NUMISS(@var{var_name})} Number of missing values. Each case is considered to have a weight of 1, regardless of the current weighting variable. The default format is F7.0. -@item PGT(@var{var_name}, @var{value}) +@item @subcmd{PGT(@var{var_name}, @var{value})} Percentage between 0 and 100 of values greater than the specified constant. The default format is F5.1. -@item PIN(@var{var_name}, @var{low}, @var{high}) +@item @subcmd{PIN(@var{var_name}, @var{low}, @var{high})} Percentage of values within the specified inclusive range of constants. The default format is F5.1. -@item PLT(@var{var_name}, @var{value}) +@item @subcmd{PLT(@var{var_name}, @var{value})} Percentage of values less than the specified constant. The default format is F5.1. -@item POUT(@var{var_name}, @var{low}, @var{high}) +@item @subcmd{POUT(@var{var_name}, @var{low}, @var{high})} Percentage of values strictly outside the specified range of constants. The default format is F5.1. -@item SD(@var{var_name}) +@item @subcmd{SD(@var{var_name})} Standard deviation of the mean. Limited to numeric values. The default format is F8.2. -@item SUM(var_name) +@item @subcmd{SUM(@var{var_name})} Sum. Limited to numeric values. The default format is F8.2. @end table Aggregation functions compare string values in terms of internal -character codes. On most modern computers, this is a form of ASCII. +character codes. +On most modern computers, this is @acronym{ASCII} or a superset thereof. The aggregation functions listed above exclude all user-missing values from calculations. To include user-missing values, insert a period @@ -236,22 +247,44 @@ variables, @subcmd{INTO}, and a list of target variables. There must the same number of source and target variables. The target variables must not already exist. -By default, increasing values of a source variable (for a string, this -is based on character code comparisons) are recoded to increasing values -of its target variable. To cause increasing values of a source variable -to be recoded to decreasing values of its target variable (@var{n} down -to 1), specify DESCENDING. +@cmd{AUTORECODE} ordinarily assigns each increasing non-missing value +of a source variable (for a string, this is based on character code +comparisons) to consecutive values of its target variable. For +example, the smallest non-missing value of the source variable is +recoded to value 1, the next smallest to 2, and so on. If the source +variable has user-missing values, they are recoded to +consecutive values just above the non-missing values. For example, if +a source variables has seven distinct non-missing values, then the +smallest missing value would be recoded to 8, the next smallest to 9, +and so on. -PRINT is currently ignored. +Use @subcmd{DESCENDING} to reverse the sort order for non-missing +values, so that the largest non-missing value is recoded to 1, the +second-largest to 2, and so on. Even with @subcmd{DESCENDING}, +user-missing values are still recoded in ascending order just above +the non-missing values. + +The system-missing value is always recoded into the system-missing +variable in target variables. + +If a source value has a value label, then that value label is retained +for the new value in the target variable. Otherwise, the source value +itself becomes each new value's label. + +Variable labels are copied from the source to target variables. + +@subcmd{PRINT} is currently ignored. The @subcmd{GROUP} subcommand is relevant only if more than one variable is to be recoded. It causes a single mapping between source and target values to -be used, instead of one map per variable. +be used, instead of one map per variable. With @subcmd{GROUP}, +user-missing values are taken from the first source variable that has +any user-missing values. -If /BLANK=MISSING is given, then string variables which contain only -whitespace are recoded as SYSMIS. If /BLANK=VALID is given then they -will be allocated a value like any other. /BLANK is not relevant -to numeric values. /BLANK=VALID is the default. +If @subcmd{/BLANK=MISSING} is given, then string variables which contain only +whitespace are recoded as SYSMIS. If @subcmd{/BLANK=VALID} is given then they +will be allocated a value like any other. @subcmd{/BLANK} is not relevant +to numeric values. @subcmd{/BLANK=VALID} is the default. @cmd{AUTORECODE} is a procedure. It causes the data to be read. @@ -302,7 +335,8 @@ When @cmd{COMPUTE} is specified following @cmd{TEMPORARY} @vindex COUNT @display -COUNT @var{var_name} = @var{var}@dots{} (@var{value}@dots{}). +COUNT @var{var_name} = @var{var}@dots{} (@var{value}@dots{}) + [/@var{var_name} = @var{var}@dots{} (@var{value}@dots{})]@dots{} Each @var{value} takes one of the following forms: @var{number} @@ -310,8 +344,8 @@ Each @var{value} takes one of the following forms: @var{num1} THRU @var{num2} MISSING SYSMIS -In addition, @var{num1} and @var{num2} can be LO or LOWEST, or HI or HIGHEST, -respectively. +where @var{num1} is a numeric expression or the words @subcmd{LO} or @subcmd{LOWEST} + and @var{num2} is a numeric expression or @subcmd{HI} or @subcmd{HIGHEST}. @end display @cmd{COUNT} creates or replaces a numeric @dfn{target} variable that @@ -327,11 +361,11 @@ User-missing values of test variables are treated just like any other values. They are @strong{not} treated as system-missing values. User-missing values that are criterion values or inside ranges of criterion values are counted as any other values. However (for numeric -variables), keyword MISSING may be used to refer to all system- +variables), keyword @subcmd{MISSING} may be used to refer to all system- and user-missing values. @cmd{COUNT} target variables are assigned values in the order -specified. In the command @code{COUNT @var{A}=@var{A} @var{B}(1) /@var{B}=@var{A} @var{B}(2).}, the +specified. In the command @subcmd{COUNT @var{A}=@var{A} @var{B}(1) /@var{B}=@var{A} @var{B}(2).}, the following actions occur: @itemize @minus @@ -364,7 +398,7 @@ the following commands: @enumerate @item Count the number of times the value 1 occurs through these variables -for each case and assigns the count to variable @code{QCOUNT}. +for each case and assigns the count to variable @code{QCOUNT}. @item Print out the total number of times the value 1 occurs throughout @@ -504,7 +538,7 @@ RECODE @var{src_vars} [INTO @var{dest_vars}]. @end display -Following the RECODE keyword itself comes @var{src_vars} which is a list +Following the @cmd{RECODE} keyword itself comes @var{src_vars} which is a list of variables whose values are to be transformed. These variables may be string variables or they may be numeric. However the list must be homogeneous; you may not mix string variables and @@ -513,11 +547,11 @@ numeric variables in the same recoding. After the list of source variables, there should be one or more @dfn{mappings}. Each mapping is enclosed in parentheses, and contains the source values and a destination value separated by a single @samp{=}. -The source values are used to specify the values in the dataset which +The source values are used to specify the values in the dataset which need to change, and the destination value specifies the new value to which they should be changed. Each @var{src_value} may take one of the following forms: -@itemize @bullet +@table @asis @item @var{number} If the source variables are numeric then @var{src_value} may be a literal number. @@ -526,12 +560,10 @@ If the source variables are string variables then @var{src_value} may be a literal string (like all strings, enclosed in single or double quotes). @item @var{num1} THRU @var{num2} This form is valid only when the source variables are numeric. -It specifies all values in the range [@var{num1}, @var{num2}]. -Normally you would ensure that @var{num2} is greater than or equal to -@var{num1}. -If @var{num1} however is greater than @var{num2}, then the range -[@var{num2},@var{num1}] will be used instead. -Open-ended ranges may be specified using @samp{LO} or @samp{LOWEST} +It specifies all values in the range between @var{num1} and @var{num2}, +including both endpoints of the range. By convention, @var{num1} +should be less than @var{num2}. +Open-ended ranges may be specified using @samp{LO} or @samp{LOWEST} for @var{num1} or @samp{HI} or @samp{HIGHEST} for @var{num2}. @item @samp{MISSING} @@ -545,13 +577,13 @@ It is valid for both numeric variables only. @item @samp{ELSE} The @samp{ELSE} keyword may be used to match any values which are not matched by any other @var{src_value} appearing in the command. -If this keyword appears, it should be used in the last mapping of the +If this keyword appears, it should be used in the last mapping of the command. -@end itemize +@end table After the source variables comes an @samp{=} and then the @var{dest_value}. The @var{dest_value} may take any of the following forms: -@itemize @bullet +@table @asis @item @var{number} A literal numeric value to which the source values should be changed. This implies the destination variable must be numeric. @@ -567,10 +599,10 @@ The special keyword @samp{COPY} means that the source value should not be modified, but copied directly to the destination value. This is meaningful only if @samp{INTO @var{dest_vars}} is specified. -@end itemize +@end table Mappings are considered from left to right. -Therefore, if a value is matched by a @var{src_value} from more than +Therefore, if a value is matched by a @var{src_value} from more than one mapping, the first (leftmost) mapping which matches will be considered. Any subsequent matches will be ignored. @@ -579,20 +611,24 @@ The behaviour of the command is slightly different depending on whether it appears or not. If @samp{INTO @var{dest_vars}} does not appear, then values will be recoded -``in place´´. This means that the recoded values are written back to the +``in place''. +This means that the recoded values are written back to the source variables from whence the original values came. In this case, the @var{dest_value} for every mapping must imply a value which has the same type as the @var{src_value}. For example, if the source value is a string value, it is not permissible for @var{dest_value} to be @samp{SYSMIS} or another forms which implies a numeric result. +It is also not permissible for @var{dest_value} to be longer than the width +of the source variable. + The following example two numeric variables @var{x} and @var{y} are recoded in place. Zero is recoded to 99, the values 1 to 10 inclusive are unchanged, values 1000 and higher are recoded to the system-missing value and all other values are changed to 999: @example -recode @var{x} @var{y} +recode @var{x} @var{y} (0 = 99) (1 THRU 10 = COPY) (1000 THRU HIGHEST = SYSMIS) @@ -604,30 +640,30 @@ into the variables specified in @var{dest_vars}, which must therefore contain a list of valid variable names. The number of variables in @var{dest_vars} must be the same as the number of variables in @var{src_vars} -and the respective order of the variables in @var{dest_vars} corresponds to +and the respective order of the variables in @var{dest_vars} corresponds to the order of @var{src_vars}. -That is to say, recoded values whose +That is to say, recoded values whose original value came from the @var{n}th variable in @var{src_vars} will be placed into the @var{n}th variable in @var{dest_vars}. The source variables will be unchanged. If any mapping implies a string as its destination value, then the respective -destination variable must already exist, or +destination variable must already exist, or have been declared using @cmd{STRING} or another transformation. Numeric variables however will be automatically created if they don't already exist. The following example deals with two source variables, @var{a} and @var{b} which contain string values. Hence there are two destination variables @var{v1} and @var{v2}. -Any cases where @var{a} or @var{b} contain the values @samp{apple}, +Any cases where @var{a} or @var{b} contain the values @samp{apple}, @samp{pear} or @samp{pomegranate} will result in @var{v1} or @var{v2} being -filled with the string @samp{fruit} whilst cases with +filled with the string @samp{fruit} whilst cases with @samp{tomato}, @samp{lettuce} or @samp{carrot} will result in @samp{vegetable}. Any other values will produce the result @samp{unknown}: @example string @var{v1} (a20). string @var{v2} (a20). -recode @var{a} @var{b} +recode @var{a} @var{b} ("apple" "pear" "pomegranate" = "fruit") ("tomato" "lettuce" "carrot" = "vegetable") (ELSE = "unknown") @@ -638,20 +674,20 @@ There is one very special mapping, not mentioned above. If the source variable is a string variable then a mapping may be specified as @samp{(CONVERT)}. This mapping, if it appears must be the last mapping given and -the @samp{INTO @var{dest_vars}} clause must also be given and -must not refer to a string variable. +the @samp{INTO @var{dest_vars}} clause must also be given and +must not refer to a string variable. @samp{CONVERT} causes a number specified as a string to -be converted to a numeric value. +be converted to a numeric value. For example it will convert the string @samp{"3"} into the numeric value 3 (note that it will not convert @samp{three} into 3). If the string cannot be parsed as a number, then the system-missing value is assigned instead. In the following example, cases where the value of @var{x} (a string variable) is the empty string, are recoded to 999 and all others are converted to the -numeric equivalent of the input value. The results are placed into the +numeric equivalent of the input value. The results are placed into the numeric variable @var{y}: @example -recode @var{x} +recode @var{x} ("" = 999) (convert) into @var{y}. @@ -661,14 +697,14 @@ It is possible to specify multiple recodings on a single command. Introduce additional recodings with a slash (@samp{/}) to separate them from the previous recodings: @example -recode - @var{a} (2 = 22) (else = 99) +recode + @var{a} (2 = 22) (else = 99) /@var{b} (1 = 3) into @var{z} . @end example @noindent Here we have two recodings. The first affects the source variable @var{a} and recodes in-place the value 2 into 22 and all other values to 99. -The second recoding copies the values of @var{b} into the the variable @var{z}, +The second recoding copies the values of @var{b} into the variable @var{z}, changing any instances of 1 into 3. @node SORT CASES @@ -700,4 +736,4 @@ cases. If workspace is exhausted, it falls back to a merge sort algorithm that involves creates numerous temporary files. -@cmd{SORT CASES} may not be specified following @cmd{TEMPORARY}. +@cmd{SORT CASES} may not be specified following @cmd{TEMPORARY}.