-Time-stamp: <2006-04-15 18:11:46 blp>
+Time-stamp: <2006-05-06 13:34:32 blp>
-Get rid of need for GNU diff in `make check'.
+Procedure processing:
+
+* Should not need temporary casefile in the common case.
+
+* LAG need not be as much of a special case.
+
+* Push FILTER, N OF CASES out of procedure.c.
-Get rid of need for file name canonicalization.
+* Change sources into state machines?
+
+Get rid of need for GNU diff in `make check'.
Format specifier code needs to be rewritten for lowered crappiness.
Kruskal-Wallis on NPAR TESTS and for Spearman and the Johnkheere trend test (in
other procedures).
-lex_token_representation() should take a buffer to fill.
-
-Make valgrind --leak-check=yes --show-reachable=yes work.
-
Add NOT_REACHED() macro.
-Add compression to casefiles.
-
Scratch variables should not be available for use following TEMPORARY.
Check our results against the NIST StRD benchmark results at
strd.itl.nist.gov/div898/strd
-In debug mode hash table code should verify that collisions are reasonably low.
-
Storage of value labels on disk is inefficient. Invent new data structure.
-Add an output flag which would cause a page break if a table segment could fit
-vertically on a page but it just happens to be positioned such that it won't.
-
Fix spanned joint cells, i.e., EDLEVEL on crosstabs.stat.
-Cell footnotes.
-
-Should be able to bottom-justify cells. It'll be expensive, though, by
-requiring an extra metrics call.
-
SELECT IF should be moved before other transformations whenever possible. It
should only be impossible when one of the variables referred to in SELECT IF is
created or modified by a previous transformation.
-The manual: add text, add index entries, add examples.
-
Figure out a stylesheet for messages displayed by PSPP: i.e., what quotation
marks around filenames, etc.
6. Categorical data analsys ?
-MORE NOTES/IDEAS/BUGS
----------------------
-
Sometimes very wide (or very tall) columns can occur in tables. What is a good
way to truncate them? It doesn't seem to cause problems for the ascii or
postscript drivers, but it's not good in the general case. Should they be
OTOH, if the /TABLE is too large, then do it the old way, complaining if either
file is not sorted on key.
-----------------------------------------------------------------------
-Statistical procedures:
-
-For each case we read from the input program:
-
-1. Execute permanent transformations. If these drop the case, stop.
-2. N OF CASES. If we have already written N cases, stop.
-3. Write case to replacement active file.
-4. Execute temporary transformations. If these drop the case, stop.
-5. Post-TEMPORARY N OF CASES. If we have already analyzed N cases, stop.
-6. FILTER, PROCESS IF. If these drop the case, stop.
-7. Pass case to procedure.
-
-Ugly cases:
-
-LAG records cases in step 3.
-
-AGGREGATE: When output goes to an external file, this is just an ordinary
-procedure. When output goes to the active file, step 3 should be skipped,
-because AGGREGATE creates its own case sink and writes to it in step 7. Also,
-TEMPORARY has no effect and we just cancel it. Regardless of direction of
-output, we should not implement AGGREGATE through a transformation because that
-will fail to honor FILTER, PROCESS IF, N OF CASES.
-
-ADD FILES: Essentially an input program. It silently cancels unclosed LOOPs
-and DO IFs. If the active file is used for input, then runs EXECUTE (if there
-are any transformations) and then steals vfm_source and encapsulates it. If
-the active file is not used for input, then it cancels all the transformations
-and deletes the original active file.
-
-CASESTOVARS: ???
-
-FLIP:
-
-MATCH FILES: Similar to AGGREGATE. This is a procedure. When the active file
-is used for input, it reads the active file; otherwise, it just cancels all the
-transformations and deletes the original active file. Step 3 should be
-skipped, because MATCH FILES creates its own case sink and writes to it in step
-7. TEMPORARY is not allowed.
-
-MODIFY VARS:
-
-REPEATING DATA:
-
-SORT CASES:
-
-UPDATE: same as ADD FILES.
-
-VARSTOCASES: ???
-----------------------------------------------------------------------
-N OF CASES
-
- * Before TEMPORARY, limits number of cases sent to the sink.
-
- * After TEMPORARY, limits number of cases sent to the procedure.
-
- * Without TEMPORARY, those are the same cases, so it limits both.
-
-SAMPLE
-
- * Sample is just a transformation. It has no special properties.
-
-FILTER
-
- * Always selects cases sent to the procedure.
-
- * No effect on cases sent to sink.
-
- * Before TEMPORARY, selection is permanent. After TEMPORARY,
- selection stops after a procedure.
-
-PROCESS IF
-
- * Always selects cases sent to the procedure.
-
- * No effect on cases sent to sink.
-
- * Always stops after a procedure.
-
-SPLIT FILE
-
- * Ignored by AGGREGATE. Used when procedures write matrices.
-
- * Always applies to the procedure.
-
- * Before TEMPORARY, splitting is permanent. After TEMPORARY,
- splitting stops after a procedure.
-
-TEMPORARY
-
- * TEMPORARY has no effect on AGGREGATE when output goes to the active file.
-
- * SORT CASES, ADD FILES, RENAME VARIABLES, CASESTOVARS, VARSTOCASES,
- COMPUTE with a lag function cannot be used after TEMPORARY.
-
- * Cannot be used in DO IF...END IF or LOOP...END LOOP.
-
- * FLIP ignores TEMPORARY. All transformations become permanent.
-
- * MATCH FILES and UPDATE cannot be used after TEMPORARY if active
- file is an input source.
-
- * RENAME VARIABLES is invalid after TEMPORARY.
-
- * WEIGHT, SPLIT FILE, N OF CASES, FILTER, PROCESS IF apply only to
- the next procedure when used after TEMPORARY.
-
-WEIGHT
-
- * Always applies to the procedure.
-
- * Before TEMPORARY, weighting is permanent. After TEMPORARY,
- weighting stops after a procedure.
-
-
--------------------------------------------------------------------------------
Local Variables:
mode: text
fill-column: 79