Time-stamp: <2006-04-26 15:15:36 blp> Get rid of need for GNU diff in `make check'. Format specifier code needs to be rewritten for lowered crappiness. CROSSTABS needs to be re-examined. RANK, which is needed for the Wilcoxon signed-rank statistic, Mann-Whitney U, Kruskal-Wallis on NPAR TESTS and for Spearman and the Johnkheere trend test (in other procedures). Add NOT_REACHED() macro. Scratch variables should not be available for use following TEMPORARY. Check our results against the NIST StRD benchmark results at strd.itl.nist.gov/div898/strd Storage of value labels on disk is inefficient. Invent new data structure. Fix spanned joint cells, i.e., EDLEVEL on crosstabs.stat. SELECT IF should be moved before other transformations whenever possible. It should only be impossible when one of the variables referred to in SELECT IF is created or modified by a previous transformation. Figure out a stylesheet for messages displayed by PSPP: i.e., what quotation marks around filenames, etc. From Zvi Grauer and : 1. design of experiments software, specifically Factorial, response surface methodology and mixrture design. These would be EXTREMELY USEFUL for chemists, engineeris, and anyone involved in the production of chemicals or formulations. 2. Multidimensional Scaling analysis (for market analysis) - 3. Preference mapping software for market analysis 4. Hierarchical clustering (as well as partition clustering) 5. Conjoint analysis 6. Categorical data analsys ? Sometimes very wide (or very tall) columns can occur in tables. What is a good way to truncate them? It doesn't seem to cause problems for the ascii or postscript drivers, but it's not good in the general case. Should they be split somehow? (One way that wide columns can occur is through user request, for instance through a wide PRINT request--try time-date.stat with a narrow ascii page or with the postscript driver on letter size paper.) From Moshe Braner : An idea regarding MATCH FILES, again getting BEYOND the state of SPSS: it always bothered me that if I have a large data file and I want to match it to a small lookup table, via MATCH FILES FILE= /TABLE= /BY key, I need to SORT the large file on key, do the match, then (usually) re-sort back into the order I really want it. There is no reason to do this, when the lookup table is small. Even a dumb sequential search through the table, for every case in the big file, is better, in some cases, than the sort. So here's my idea: first look at the /TABLE file, if it is "small enough", read it into memory, and create an index (or hash table, whatever) for it. Then read the /FILE and use the index to match to each case. OTOH, if the /TABLE is too large, then do it the old way, complaining if either file is not sorted on key. ---------------------------------------------------------------------- Statistical procedures: For each case we read from the input program: 1. Execute permanent transformations. If these drop the case, stop. 2. N OF CASES. If we have already written N cases, stop. 3. Write case to replacement active file. 4. Execute temporary transformations. If these drop the case, stop. 5. Post-TEMPORARY N OF CASES. If we have already analyzed N cases, stop. 6. FILTER, PROCESS IF. If these drop the case, stop. 7. Pass case to procedure. Ugly cases: LAG records cases in step 3. AGGREGATE: When output goes to an external file, this is just an ordinary procedure. When output goes to the active file, step 3 should be skipped, because AGGREGATE creates its own case sink and writes to it in step 7. Also, TEMPORARY has no effect and we just cancel it. Regardless of direction of output, we should not implement AGGREGATE through a transformation because that will fail to honor FILTER, PROCESS IF, N OF CASES. ADD FILES: Essentially an input program. It silently cancels unclosed LOOPs and DO IFs. If the active file is used for input, then runs EXECUTE (if there are any transformations) and then steals vfm_source and encapsulates it. If the active file is not used for input, then it cancels all the transformations and deletes the original active file. CASESTOVARS: ??? FLIP: MATCH FILES: Similar to AGGREGATE. This is a procedure. When the active file is used for input, it reads the active file; otherwise, it just cancels all the transformations and deletes the original active file. Step 3 should be skipped, because MATCH FILES creates its own case sink and writes to it in step 7. TEMPORARY is not allowed. MODIFY VARS: REPEATING DATA: SORT CASES: UPDATE: same as ADD FILES. VARSTOCASES: ??? ---------------------------------------------------------------------- N OF CASES * Before TEMPORARY, limits number of cases sent to the sink. * After TEMPORARY, limits number of cases sent to the procedure. * Without TEMPORARY, those are the same cases, so it limits both. SAMPLE * Sample is just a transformation. It has no special properties. FILTER * Always selects cases sent to the procedure. * No effect on cases sent to sink. * Before TEMPORARY, selection is permanent. After TEMPORARY, selection stops after a procedure. PROCESS IF * Always selects cases sent to the procedure. * No effect on cases sent to sink. * Always stops after a procedure. SPLIT FILE * Ignored by AGGREGATE. Used when procedures write matrices. * Always applies to the procedure. * Before TEMPORARY, splitting is permanent. After TEMPORARY, splitting stops after a procedure. TEMPORARY * TEMPORARY has no effect on AGGREGATE when output goes to the active file. * SORT CASES, ADD FILES, RENAME VARIABLES, CASESTOVARS, VARSTOCASES, COMPUTE with a lag function cannot be used after TEMPORARY. * Cannot be used in DO IF...END IF or LOOP...END LOOP. * FLIP ignores TEMPORARY. All transformations become permanent. * MATCH FILES and UPDATE cannot be used after TEMPORARY if active file is an input source. * RENAME VARIABLES is invalid after TEMPORARY. * WEIGHT, SPLIT FILE, N OF CASES, FILTER, PROCESS IF apply only to the next procedure when used after TEMPORARY. WEIGHT * Always applies to the procedure. * Before TEMPORARY, weighting is permanent. After TEMPORARY, weighting stops after a procedure. ------------------------------------------------------------------------------- Local Variables: mode: text fill-column: 79 End: