     [file tut.texi not included]

Using pspp
**********

pspp is a tool for the statistical analysis of sampled data. You can use it to discover patterns in the data, to explain differences in one subset of data in terms of another subset and to find out whether certain beliefs about the data are justified. This chapter does not attempt to introduce the theory behind the statistical analysis, but it shows how such analysis can be performed using pspp.

For the purposes of this tutorial, it is assumed that you are using pspp in its interactive mode from the command line. However, the example commands can also be typed into a file and executed in a post-hoc mode by typing pspp FILENAME at a shell prompt, where FILENAME is the name of the file containing the commands. Alternatively, from the graphical interface, you can select File -> New -> Syntax to open a new syntax window and use the Run menu when a syntax fragment is ready to be executed. Whichever method you choose, the syntax is identical.

When using the interactive method, pspp tells you that it's waiting for your data with a string like PSPP> or data>. In the examples of this chapter, whenever you see text like this, it indicates the prompt displayed by pspp, not something that you should type.

Throughout this chapter reference is made to a number of sample data files. So that you can try the examples for yourself, you should have received these files along with your copy of pspp. (These files contain purely fictitious data.  They should not be used for research purposes.) Normally these files are installed in the directory
example-dir.
If however your system administrator or operating system vendor has
chosen to install them in a different location, you will have to adjust
the examples accordingly.

Preparation of Data Files
=========================

Before analysis can commence,  the data must be loaded into pspp and arranged such that both pspp and humans can understand what the data represents. There are two aspects of data:

  * The variables --- these are the parameters of a quantity which has been measured or estimated in some way. For example height, weight and geographic location are all variables.

  * The observations (also called `cases') of the variables --- each observation represents an instance when the variables were measured or observed.

For example, a data set which has the variables HEIGHT, WEIGHT, and NAME, might have the observations:

     1881 89.2 Ahmed
     1192 107.01 Frank
     1230 67 Julie

The following sections explain how to define a dataset.

Defining Variables
------------------

Variables come in two basic types, viz: numeric and string. Variables such as age, height and satisfaction are numeric, whereas name is a string variable. String variables are best reserved for commentary data to assist the human observer. However they can also be used for nominal or categorical data.

data-list defines two variables FORENAME and HEIGHT, and reads data into them by manual input.

     PSPP> data list list /forename (A12) height.
     PSPP> begin data.
     data> Ahmed 188
     data> Bertram 167
     data> Catherine 134.231
     data> David 109.1
     data> end data
     PSPP>

Example: Manual entry of data using the DATA LIST command.
Two variables
FORENAME and HEIGHT are defined and subsequently filled
with  manually entered data.

There are several things to note about this example.

  * The words data list list are an example of the DATA LIST command. See DATA LIST. It tells pspp to prepare for reading data. The word list intentionally appears twice. The first occurrence is part of the DATA LIST call, whilst the second tells pspp that the data is to be read as free format data with one record per line.

  * The / character is important. It marks the start of the list of variables which you wish to define.

  * The text forename is the name of the first variable, and (A12) says that the variable FORENAME is a string variable and that its maximum length is 12 bytes. The second variable's name is specified by the text height. Since no format is given, this variable has the default format. Normally the default format expects numeric data, which should be entered in the locale of the operating system. Thus, the example is correct for English locales and other locales which use a period (.) as the decimal separator. However if you are using a system with a locale which uses the comma (,) as the decimal separator, then you should in the subsequent lines substitute . with ,. Alternatively, you could explicitly tell pspp that the HEIGHT variable is to be read using a period as its decimal separator by appending the text DOT8.3 after the word height. For more information on data formats, see Input and Output Formats.

  * Normally, pspp displays the  prompt PSPP> whenever it's expecting a command. However, when it's expecting data, the prompt changes to data> so that you know to enter data and not a command.

  * At the end of every command there is a terminating . which tells pspp that the end of a command has been encountered. You should not enter . when data is expected (ie. when the data> prompt is current) since it is appropriate only for terminating commands.

Listing the data
----------------

Once the data has been entered, you could type

     PSPP> list /format=numbered.

to list the data. The optional text /format=numbered requests the case numbers to be shown along with the data. It should show the following output:

     Case#     forename   height
     ----- ------------ --------
         1 Ahmed          188.00
         2 Bertram        167.00
         3 Catherine      134.23
         4 David          109.10

Note that the numeric variable HEIGHT is displayed to 2 decimal places, because the format for that variable is F8.2. For a complete description of the LIST command, see LIST.

Reading data from a text file
-----------------------------

The previous example showed how to define a set of variables and to manually enter the data for those variables. Manual entering of data is tedious work, and often a file containing the data will be have been previously prepared. Let us assume that you have a file called mydata.dat containing the ascii encoded data:

     Ahmed          188.00
     Bertram        167.00
     Catherine      134.23
     David          109.10
                   .
                   .
                   .
     Zachariah      113.02

You can can tell the DATA LIST command to read the data directly from this file instead of by manual entry, with a command like:

     PSPP> data list file='mydata.dat' list /forename (A12) height.

Notice however, that it is still necessary to specify the names of the variables and their formats, since this information is not contained in the file. It is also possible to specify the file's character encoding and other parameters. For full details refer to see DATA LIST.

Reading data from a pre-prepared pspp file
------------------------------------------

When working with other pspp users, or users of other software which uses the pspp data format, you may be given the data in a pre-prepared pspp file. Such files contain not only the data, but the variable definitions, along with their formats, labels and other meta-data. Conventionally, these files (sometimes called ``system'' files) have the suffix .sav, but that is not mandatory. The following syntax loads a file called my-file.sav.

     PSPP> get file='my-file.sav'.

You will encounter several instances of this in future examples.

Saving data to a pspp file.
---------------------------

If you want to save your data, along with the variable definitions so that you or other pspp users can use it later, you can do this with the SAVE command.

The following syntax will save the existing data and variables to a file called my-new-file.sav.

     PSPP> save outfile='my-new-file.sav'.

If my-new-file.sav already exists, then it will be overwritten. Otherwise it will be created.

Reading data from other sources
-------------------------------

Sometimes it's useful to be able to read data from comma separated text, from spreadsheets, databases or other sources. In these instances you should use the GET DATA command (see GET DATA).

Exiting PSPP
------------

Use the FINISH command to exit PSPP:

     PSPP> finish.

Data Screening and Transformation
=================================

Once data has been entered, it is often desirable, or even necessary, to transform it in some way before performing analysis upon it. At the very least, it's good practice to check for errors.

Identifying incorrect data
--------------------------

Data from real sources is rarely error free. pspp has a number of procedures which can be used to help identify data which might be incorrect.

The DESCRIPTIVES command (see DESCRIPTIVES) is used to generate simple linear statistics for a dataset.  It is also useful for identifying potential problems in the data. The example file physiology.sav contains a number of physiological measurements of a sample of healthy adults selected at random. However, the data entry clerk made a number of mistakes when entering the data. descriptives illustrates the use of DESCRIPTIVES to screen this data and identify the erroneous values.

     PSPP> get file='example-dir/physiology.sav'.
     PSPP> descriptives sex, weight, height.

Output:

     DESCRIPTIVES.  Valid cases = 40; cases with missing value(s) = 0.
     +--------#--+-------+-------+-------+-------+
     |Variable# N|  Mean |Std Dev|Minimum|Maximum|
     #========#==#=======#=======#=======#=======#
     |sex     #40|    .45|    .50|    .00|   1.00|
     |height  #40|1677.12| 262.87| 179.00|1903.00|
     |weight  #40|  72.12|  26.70| -55.60|  92.07|
     +--------#--+-------+-------+-------+-------+

Example: Using the DESCRIPTIVES command to display simple
summary information about the data.
In this case, the results show unexpectedly low values in the Minimum
column, suggesting incorrect data entry.

In the output of descriptives, the most interesting column is the minimum value. The WEIGHT variable has a minimum value of less than zero, which is clearly erroneous. Similarly, the HEIGHT variable's minimum value seems to be very low. In fact, it is more than 5 standard deviations from the mean, and is a seemingly bizarre height for an adult person. We can examine the data in more detail with the EXAMINE command (see EXAMINE):

In examine you can see that the lowest value of HEIGHT is 179 (which we suspect to be erroneous), but the second lowest is 1598 which we know from the DESCRIPTIVES command is within 1 standard deviation from the mean. Similarly the WEIGHT variable has a lowest value which is negative but a plausible value for the second lowest value. This suggests that the two extreme values are outliers and probably represent data entry errors.

[... continue from descriptives]

     PSPP> examine height, weight /statistics=extreme(3).

Output:

     #===============================#===========#=======#
     #                               #Case Number| Value #
     #===============================#===========#=======#
     #Height in millimetres Highest 1#         14|1903.00#
     #                              2#         15|1884.00#
     #                              3#         12|1801.65#
     #                     ----------#-----------+-------#
     #                       Lowest 1#         30| 179.00#
     #                              2#         31|1598.00#
     #                              3#         28|1601.00#
     #                     ----------#-----------+-------#
     #Weight in kilograms   Highest 1#         13|  92.07#
     #                              2#          5|  92.07#
     #                              3#         17|  91.74#
     #                     ----------#-----------+-------#
     #                       Lowest 1#         38| -55.60#
     #                              2#         39|  54.48#
     #                              3#         33|  55.45#
     #===============================#===========#=======#

Example: Using the EXAMINE command to see the extremities of the data
for different variables.  Cases 30 and 38 seem to contain values
very much lower than the rest of the data.
They are possibly erroneous.

Dealing with suspicious data
----------------------------

If possible, suspect data should be checked and re-measured. However, this may not always be feasible, in which case the researcher may decide to disregard these values. pspp has a feature whereby data can assume the special value `SYSMIS', and will be disregarded in future analysis. See Missing Observations. You can set the two suspect values to the `SYSMIS' value using the RECODE command.

     pspp> recode height (179 = SYSMIS).
     pspp> recode weight (LOWEST THRU 0 = SYSMIS).

The first command says that for any observation which has a HEIGHT value of 179, that value should be changed to the SYSMIS value. The second command says that any WEIGHT values of zero or less should be changed to SYSMIS. From now on, they will be ignored in analysis. For detailed information about the RECODE command see RECODE.

If you now re-run the DESCRIPTIVES or EXAMINE commands in descriptives and examine you will see a data summary with more plausible parameters. You will also notice that the data summaries indicate the two missing values.

Inverting negatively coded variables
------------------------------------

Data entry errors are not the only reason for wanting to recode data. The sample file hotel.sav comprises data gathered from a customer satisfaction survey of clients at a particular hotel. In reliability, this file is loaded for analysis. The line display dictionary. tells pspp to display the variables and associated data. The output from this command has been omitted from the example for the sake of clarity, but you will notice that each of the variables V1, V2 ... V5  are measured on a 5 point Likert scale, with 1 meaning ``Strongly disagree'' and 5 meaning ``Strongly agree''. Whilst variables V1, V2 and V4 record responses to a positively posed question, variables V3 and V5 are responses to negatively worded questions. In order to perform meaningful analysis, we need to recode the variables so that they all measure in the same direction. We could use the RECODE command, with syntax such as:

     recode v3 (1 = 5) (2 = 4) (4 = 2) (5 = 1).

However an easier and more elegant way uses the COMPUTE command (see COMPUTE). Since the variables are Likert variables in the range (1 ... 5), subtracting their value  from 6 has the effect of inverting them:

     compute VAR = 6 - VAR.

reliability uses this technique to recode the variables V3 and V5. After applying  COMPUTE for both variables, all subsequent commands will use the inverted values.

Testing data consistency
------------------------

A sensible check to perform on survey data is the calculation of reliability. This gives the statistician some confidence that the questionnaires have been completed thoughtfully. If you examine the labels of variables V1,  V3 and V5, you will notice that they ask very similar questions. One would therefore expect the values of these variables (after recoding) to closely follow one another, and we can test that with the RELIABILITY command (see RELIABILITY). reliability shows a pspp session where the user (after recoding negatively scaled variables) requests reliability statistics for V1, V3 and V5.

     PSPP> get file='example-dir/hotel.sav'.
     PSPP> display dictionary.
     PSPP> * recode negatively worded questions.
     PSPP> compute v3 = 6 - v3.
     PSPP> compute v5 = 6 - v5.
     PSPP> reliability v1, v3, v5.

Output (dictionary information omitted for clarity):

     1.1 RELIABILITY.  Case Processing Summary
     #==============#==#======#
     #              # N|   %  #
     #==============#==#======#
     #Cases Valid   #17|100.00#
     #      Excluded# 0|   .00#
     #      Total   #17|100.00#
     #==============#==#======#

     1.2 RELIABILITY.  Reliability Statistics
     #================#==========#
     #Cronbach's Alpha#N of Items#
     #================#==========#
     #             .86#         3#
     #================#==========#

Example: Recoding negatively scaled variables, and testing for
reliability with the RELIABILITY command. The Cronbach Alpha
coefficient suggests a high degree of reliability among variables
V1, V2 and V5.

As a rule of thumb, many statisticians consider a value of Cronbach's Alpha of 0.7 or higher to indicate reliable data. Here, the value is 0.86 so the data and the recoding that we performed are vindicated.

Testing for normality
---------------------

Many statistical tests rely upon certain properties of the data. One common property, upon which many linear tests depend, is that of normality --- the data must have been drawn from a normal distribution. It is necessary then to ensure normality before deciding upon the test procedure to use.  One way to do this uses the EXAMINE command.

In normality, a researcher was examining the failure rates of equipment produced by an engineering company. The file repairs.sav contains the mean time between failures (MTBF) of some items of equipment subject to the study. Before performing linear analysis on the data, the researcher wanted to ascertain that the data is normally distributed.

A normal distribution has a skewness and kurtosis of zero. Looking at the skewness of MTBF in normality it is clear that the mtbf figures have a lot of positive skew and are therefore not drawn from a normally distributed variable. Positive skew can often be compensated for by applying a logarithmic transformation. This is done with the COMPUTE command in the line

     compute mtbf_ln = ln (mtbf).

Rather than redefining the existing variable, this use of COMPUTE defines a new variable MTBF_LN which is the natural logarithm of MTBF. The final command in this example calls EXAMINE on this new variable, and it can be seen from the results that both the skewness and kurtosis for MTBF_LN are very close to zero. This provides some confidence that the MTBF_LN variable is normally distributed and thus safe for linear analysis. In the event that no suitable transformation can be found, then it would be worth considering an appropriate non-parametric test instead of a linear one. See NPAR TESTS, for information about non-parametric tests.

     PSPP> get file='example-dir/repairs.sav'.
     PSPP> examine mtbf
                     /statistics=descriptives.
     PSPP> compute mtbf_ln = ln (mtbf).
     PSPP> examine mtbf_ln
                     /statistics=descriptives.

Output:

     1.2 EXAMINE.  Descriptives
     #====================================================#=========#==========#
     #                                                    #Statistic|Std. Error#
     #====================================================#=========#==========#
     #mtbf    Mean                                        #   8.32  |   1.62   #
     #        95% Confidence Interval for Mean Lower Bound#   4.85  |          #
     #                                         Upper Bound#  11.79  |          #
     #        5% Trimmed Mean                             #   7.69  |          #
     #        Median                                      #   8.12  |          #
     #        Variance                                    #  39.21  |          #
     #        Std. Deviation                              #   6.26  |          #
     #        Minimum                                     #   1.63  |          #
     #        Maximum                                     #  26.47  |          #
     #        Range                                       #  24.84  |          #
     #        Interquartile Range                         #   5.83  |          #
     #        Skewness                                    #   1.85  |    .58   #
     #        Kurtosis                                    #   4.49  |   1.12   #
     #====================================================#=========#==========#

     2.2 EXAMINE.  Descriptives
     #====================================================#=========#==========#
     #                                                    #Statistic|Std. Error#
     #====================================================#=========#==========#
     #mtbf_ln Mean                                        #   1.88  |    .19   #
     #        95% Confidence Interval for Mean Lower Bound#   1.47  |          #
     #                                         Upper Bound#   2.29  |          #
     #        5% Trimmed Mean                             #   1.88  |          #
     #        Median                                      #   2.09  |          #
     #        Variance                                    #   .54   |          #
     #        Std. Deviation                              #   .74   |          #
     #        Minimum                                     #   .49   |          #
     #        Maximum                                     #   3.28  |          #
     #        Range                                       #   2.79  |          #
     #        Interquartile Range                         #   .92   |          #
     #        Skewness                                    #   -.16  |    .58   #
     #        Kurtosis                                    #   -.09  |   1.12   #
     #====================================================#=========#==========#

Example: Testing for normality using the EXAMINE command and applying
a logarithmic transformation.
The MTBF variable has a large positive skew and is therefore
unsuitable for linear statistical analysis.
However the transformed variable (MTBF_LN) is close to normal and
would appear to be more suitable.

Hypothesis Testing
==================

One of the most fundamental purposes of statistical analysis is hypothesis testing. Researchers commonly need to test hypotheses about a set of data. For example, she might want to test whether one set of data comes from the same distribution as another, or whether the mean of a dataset significantly differs from a particular value. This section presents just some of the possible tests that pspp offers.

The researcher starts by making a null hypothesis. Often this is a hypothesis which he suspects to be false. For example, if he suspects that A is greater than B he will state the null hypothesis as  A = B. (This example assumes that it is already proven that B is not greater than A.)

The p-value is a recurring concept in hypothesis testing. It is the highest acceptable probability that the evidence implying a null hypothesis is false, could have been obtained when the null hypothesis is in fact true. Note that this is not the same as ``the probability of making an error'' nor is it the same as ``the probability of rejecting a hypothesis when it is true''.

Testing for differences of means
--------------------------------

A common statistical test involves hypotheses about means. The T-TEST command is used to find out whether or not two separate subsets have the same mean.

t-test uses the file physiology.sav previously encountered. A researcher suspected that the heights and core body temperature of persons might be different depending upon their sex. To investigate this, he posed two null hypotheses:

  * The mean heights of males and females in the population are equal.

  * The mean body temperature of males and females in the population are equal.

For the purposes of the investigation the researcher decided to use a  p-value of 0.05.

In addition to the T-test, the T-TEST command also performs the Levene test for equal variances. If the variances are equal, then a more powerful form of the T-test can be used. However if it is unsafe to assume equal variances, then an alternative calculation is necessary. pspp performs both calculations.

For the HEIGHT variable, the output shows the significance of the Levene test to be 0.33 which means there is a 33% probability that the Levene test produces this outcome when the variances are equal. Had the significance been less than 0.05, then it would have been unsafe to assume that the variances were equal. However, because the value is higher than 0.05 the homogeneity of variances assumption is safe and the ``Equal Variances'' row (the more powerful test) can be used. Examining this row, the two tailed significance for the HEIGHT t-test is less than 0.05, so it is safe to reject the null hypothesis and conclude that the mean heights of males and females are unequal.

For the TEMPERATURE variable, the significance of the Levene test is 0.58 so again, it is safe to use the row for equal variances. The equal variances row indicates that the two tailed significance for TEMPERATURE is 0.20.  Since this is greater than 0.05 we must reject the null hypothesis and conclude that there is insufficient evidence to suggest that the body temperature of male and female persons are different.

     PSPP> get file='example-dir/physiology.sav'.
     PSPP> recode height (179 = SYSMIS).
     PSPP> t-test group=sex(0,1) /variables = height temperature.

Output:

     1.1 T-TEST.  Group Statistics
     #==================#==#=======#==============#========#
     #              sex | N|  Mean |Std. Deviation|SE. Mean#
     #==================#==#=======#==============#========#
     #height      Male  |22|1796.49|         49.71|   10.60#
     #            Female|17|1610.77|         25.43|    6.17#
     #temperature Male  |22|  36.68|          1.95|     .42#
     #            Female|18|  37.43|          1.61|     .38#
     #==================#==#=======#==============#========#
     1.2 T-TEST.  Independent Samples Test
     #===========================#=========#===============================   =#
     #                           # Levene's| t-test for Equality of Means      #
     #                           #----+----+------+-----+------+---------+-   -#
     #                           #    |    |      |     |      |         |     #
     #                           #    |    |      |     |Sig. 2|         |     #
     #                           #  F |Sig.|   t  |  df |tailed|Mean Diff|     #
     #===========================#====#====#======#=====#======#=========#=   =#
     #height      Equal variances# .97| .33| 14.02|37.00|   .00|   185.72| ... #
     #          Unequal variances#    |    | 15.15|32.71|   .00|   185.72| ... #
     #temperature Equal variances# .31| .58| -1.31|38.00|   .20|     -.75| ... #
     #          Unequal variances#    |    | -1.33|37.99|   .19|     -.75| ... #
     #===========================#====#====#======#=====#======#=========#=   =#

Example: The T-TEST command tests for differences of means.
Here, the HEIGHT variable's two tailed significance is less than
0.05, so the null hypothesis can be rejected.
Thus, the evidence suggests there is a difference between the heights of
male and female persons.
However the significance of the test for the TEMPERATURE
variable is greater than 0.05 so the null hypothesis cannot be
rejected, and there is insufficient evidence to suggest a difference
in body temperature.

Linear Regression
-----------------

Linear regression is a technique used to investigate if and how a variable is linearly related to others. If a variable is found to be linearly related, then this can be used to predict future values of that variable.

In example regression, the service department of the company wanted to be able to predict the time to repair equipment, in order to improve the accuracy of their quotations. It was suggested that the time to repair might be related to the time between failures and the duty cycle of the equipment. The p-value of 0.1 was chosen for this investigation. In order to investigate this hypothesis, the REGRESSION command was used. This command not only tests if the variables are related, but also identifies the potential linear relationship. See REGRESSION.

     PSPP> get file='example-dir/repairs.sav'.
     PSPP> regression /variables = mtbf duty_cycle /dependent = mttr.
     PSPP> regression /variables = mtbf /dependent = mttr.

Output:

     1.3(1) REGRESSION.  Coefficients
     #=============================================#====#==========#====#=====#
     #                                             #  B |Std. Error|Beta|  t  #
     #========#====================================#====#==========#====#=====#
     #        |(Constant)                          #9.81|      1.50| .00| 6.54#
     #        |Mean time between failures (months) #3.10|       .10| .99|32.43#
     #        |Ratio of working to non-working time#1.09|      1.78| .02|  .61#
     #        |                                    #    |          |    |     #
     #========#====================================#====#==========#====#=====#

     1.3(2) REGRESSION.  Coefficients
     #=============================================#============#
     #                                             #Significance#
     #========#====================================#============#
     #        |(Constant)                          #         .10#
     #        |Mean time between failures (months) #         .00#
     #        |Ratio of working to non-working time#         .55#
     #        |                                    #            #
     #========#====================================#============#
     2.3(1) REGRESSION.  Coefficients
     #============================================#=====#==========#====#=====#
     #                                            #  B  |Std. Error|Beta|  t  #
     #========#===================================#=====#==========#====#=====#
     #        |(Constant)                         #10.50|       .96| .00|10.96#
     #        |Mean time between failures (months)# 3.11|       .09| .99|33.39#
     #        |                                   #     |          |    |     #
     #========#===================================#=====#==========#====#=====#

     2.3(2) REGRESSION.  Coefficients
     #============================================#============#
     #                                            #Significance#
     #========#===================================#============#
     #        |(Constant)                         #         .06#
     #        |Mean time between failures (months)#         .00#
     #        |                                   #            #
     #========#===================================#============#

Example: Linear regression analysis to find a predictor for
MTTR.
The first attempt, including DUTY_CYCLE, produces some
unacceptable high significance values.
However the second attempt, which excludes DUTY_CYCLE, produces
significance values no higher than 0.06.
This suggests that MTBF alone may be a suitable predictor
for MTTR.

The coefficients in the first table suggest that the formula MTTR = 9.81 + 3.1 \times MTBF + 1.09 \times DUTY_CYCLE can be used to predict the time to repair. However, the significance value for the DUTY_CYCLE coefficient is very high, which would make this an unsafe predictor. For this reason, the test was repeated, but omitting the DUTY_CYCLE variable. This time, the significance of all coefficients no higher than 0.06, suggesting that at the 0.06 level, the formula MTTR = 10.5 + 3.11 \times MTBF is a reliable predictor of the time to repair.