+Wed Jan 26 11:44:11 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * set.q: Affixed a fix to the previous fix such that we'll be OK now
+ whether or not PAGER is set.
+
+Wed Jan 26 09:25:54 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * set.q: Copied the string produced by getenv("PAGER") thus avoiding
+ "invalid free" errors. Hopefully fixes bug #11722
+
+ * compute.c expr-prs.c: Check that lvalues are populated before
+ attempting to destroy them. Closes bug #11676
+
+Tue Jan 25 21:01:43 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * aggregate.c: Initialised the complete agr_proc structure.
+ Closes bug #11675
+
+
+Sun Jan 23 23:02:21 2005 Ben Pfaff <blp@gnu.org>
+
+ * print.c: (print_trns_free) Close the dfm writer if there is one,
+ fixing a memory leak.
+
+Mon Jan 24 12:24:36 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * glob.c oneway.q q2c.c t-test.q vfm.c: Still *more* memory leaks
+ fixed.
+
+
+Fri Jan 21 19:54:14 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * linked-list.[ch] Added
+
+ * examine.q file-handle.[hq] font.h glob.c groff-font.c postscript.c
+ set.q: Yet more memory leaks
+
+Tue Jan 18 23:12:40 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * t-test.q examine.q : More memory leaks fixed.
+
+Tue Jan 18 19:26:59 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q factor_stats.[ch] get.c pfm-read.c: Plugged numerous
+ memory leaks.
+
+Mon Jan 10 14:43:45 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * ascii.c cartesian.c casefile.c chart.h devind.c
+ examine.q frequencies.q
+ html.c output.h piechart.c plot-chart.c plot-hist.c
+
+ Integrated the chart rendering into the output stream
+ (currently only works for html).
+
+ Removed gratuitous use of ifndef NO_CHARTS, and replaced with
+ dummy-chart.c for compiling without charts.
+
+ * mkfile.[ch] Created.
+
+ * som.[ch] tab.[ch]: Changed name of som_table to som_entity
+ Added type element so we can tell if it's a chart or a table.
+
+ * chart.h examine.q piechart.c plot-chart.c plot-hist.c: changed the
+ API of charts to be more like that of tables.
+
+Thu Jan 13 21:00:02 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * casefile.c main.c: Moved the SIGINT handler from casefile.c to
+ main.c. Removed the handler for SIGQUIT.
+
+Mon Jan 10 14:43:45 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * casefile.c: Added a signal handler to delete temp files on
+ SIGINT and SIGQUIT
+
+ * permissions.c Inhibited the PERMISSIONS command when SAFER is on.
+
+ * command.def Added a lot more unimplemented commands.
+
+ * copyleft.[ch] cmdline.c Moved legal information to copyleft.c
+
+Sat Jan 8 23:58:34 2005 Ben Pfaff <blp@gnu.org>
+
+ * sort.c: (compare_initial_runs) Needed additional level of
+ dereferencing.
+ (merge_once) Fix plenty of stupid mistakes.
+
+Sat Jan 8 23:55:27 2005 Ben Pfaff <blp@gnu.org>
+
+ * casefile.c: (casefile_sleep) Need to flush_buffer() after
+ calling casefile_to_disk() or we will lose the last block in the
+ file if the casefile started out as disk-based.
+ (casefile_get_reader) Initialize reader->destructive to 0.
+ (cmd_debug_casefile) Add new test pattern.
+ (test_casefile) Define new test pattern to make sure
+ casefile_sleep() works properly.
+
+Fri Jan 7 08:00:05 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * Makefile.am chart.[ch] histogram.[ch] piechart.c (Modified);
+ plot-hist.c plot-chart.c (Added) Reorganised these files in an
+ attempt to seperate the creation and processing of charts from their
+ actuall renedering.
+
+ * examine.q frequencies.q generated charts conditional upon the NO_CHARTS
+ macro.
+
+Thu Jan 6 18:48:58 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * main.c Added a signal handler for SIGFPE
+
+ * sort.c Somewhat more robust fix to the previous entry.
+
+Wed Jan 5 21:23:31 2005 Ben Pfaff <blp@gnu.org>
+
+ * sort.c: (merge) Fix assertion for proper Huffman merge pattern:
+ 0 == 1 modulo 1. See Knuth 5.4.9 (vol. 3, 2nd ed.,
+ pp. 361). Thanks to John Darrington <john@cellform.com.au> for
+ reporting the bug.
+
+Wed Jan 5 22:42:26 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * case.h Fixed bug # 11307
+
+Wed Jan 5 08:30:48 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * val-labs.c Fixed bug which caused a crash if VALUE LABELS had
+ a trailing slash.
+
+Mon Jan 3 17:44:37 2005 Ben Pfaff <blp@gnu.org>
+
+ * pfm-read.c: (read_variables) Remove direct manipulation of
+ v->aux, which is no longer needed. Fixes Bug #11483.
+
+Sat Jan 1 19:01:16 WST 2005 John Darrington <john@darrington.wattle.id.au>
+
+ * data-list.c Fixed a bug in parsing delimiters.
+
+ * group.c vars-atr.c Fixed buglet in hash/compare functions for alpha
+ values.
+
+ * percentiles.c Properly handled calculation of Tukey hinges where
+ the number of data is small.
+
+ * oneway.q Used the generic value_to_string function for independent
+ variable instead of trying to do it ourselves.
+
+ * box-whisker.c Fixed a buglet which caused a crash if the number of
+ data was zero
+
+
+Fri Dec 31 16:47:45 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q box-whisker.c chart.h Implemented boxplots in EXAMINE
+
+ * percentiles.c Fixed some bugs when calculating percentiles when
+ there's a small number of cases.
+
+Wed Dec 29 08:18:08 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * percentiles.[ch] Added. Calculates percentiles and Tukey hinges
+
+ * examine.q factor_stats.[ch] Added calculation of percentiles
+
+Fri Dec 24 15:09:11 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * t-test.q Fixed bug #11227 Made t-test work when the independent
+ variable is alpha
+
+Sat Dec 11 11:43:45 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * factor_stats.c Fixed calculation of trimmed mean under various
+ special conditions.
+
+Sat Dec 4 17:14:45 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * histogram.c chart.[ch] factor_stats.c frequencies.q
+
+ Added code to calculate sensible histogram ranges and limits.
+
+Thu Dec 2 13:37:43 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * chart.h Updated to reflect many API changes.
+
+ * cartesian.c chart.c Moved the definitions of chart_write_{xy}scale from
+ cartesian.c and into chart.c
+
+ * factorstats.[ch] Added the histogram calculations
+
+ * casefile.c Removed an unused variable.
+
+ * frequencies.q examine.q histogram.c Reworked the API for
+ histograms.
+
+ * piechart.c Revised the API for piecharts.
+
+ * var.h Moved the definitions of freq_tab and freq out of var.h
+ and into frequencies.q where they belong.
+
+Tue Nov 30 21:10:20 2004 Ben Pfaff <blp@gnu.org>
+
+ * flip.c: (flip_file) Check for off_t separately from fseeko(),
+ using AC_TYPE_OFF_T.
+
+Tue Nov 30 08:47:41 2004 Ben Pfaff <blp@gnu.org>
+
+ * flip.c: (flip_file) If fseeko() is not available, use long int
+ for off_t. Thanks to "Marshall DeBerry" <mdb@radix.net> for
+ reporting the problem.
+
+Mon Nov 29 12:20:59 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q factor_stats.[ch] Changed stderr to se_mean to avoid
+ conflict with stdio.
+
+Sun Nov 21 10:32:41 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * var-labs.c (var_to_string) Now returns null if the variable is null
+
+ * value-labels.c (value_to_string) Made it return null if either the
+ value or the variable is null.
+
+ * hash.c (hsh_clear) Fixed a buglet.
+
+ * examine.q factor_stats.[ch] Largely rewrote, because I'd started
+ with the wrong model.
+
+ * casefile.[ch] Added a function to return the casereader.case_idx
+ member
+
+ * examine.q Implemented the extreme values results.
+
+John Darrington <john@darrington.wattle.id.au>
+
+ * settings.h set.c glob.[ch] frequencies.q q2c.c error.c lexer.[ch]
+ output.[ch] getline.c
+
+ Plugged some memory leaks
+
+Mon Nov 15 23:47:40 2004 Ben Pfaff <blp@gnu.org>
+
+ Adopt GSL random number generators, paving the way for providing
+ the complete suite of random number generators on expressions.
+
+ * Makefile.am: Remove random.c, random.h.
+
+ * random.c: Removed.
+
+ * random.h: Removed.
+
+ * algorithm.c: (algo_default_random) Use GSL functions.
+
+ * casefile.c: (test_casefile) Use GSL RNG functions.
+
+ * expr-evl.c: (expr_evaluate) Use GSL RNG functions for OP_NORMAL,
+ OP_UNIFORM.
+
+ * sample.c: (cmd_sample) Use GSL RNG functions.
+ (sample_trns_proc) Ditto.
+
+ * set.q: (static var set_seed) Removed.
+ (static var seed_flag) Removed.
+ (static var rng) New variable.
+ (aux_stc_custom_seed) No seed value anymore, don't print anything.
+ (stc_custom_seed) Use new seed functions.
+ (seed_is_set) Removed.
+ (get_rng) New function that composes the entire external
+ interface.
+ (set_rng) New function.
+ (random_seed) New function.
+
+Mon Nov 15 22:08:25 2004 Ben Pfaff <blp@gnu.org>
+
+ * expr-evl.c: (expr_evaluate) Fix XDATE.JDAY formula. Thanks to
+ John Darrington <john@darrington.wattle.id.au> for reporting this
+ bug.
+
+Tue Nov 16 13:19:18 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * permissions.c command.def Added the PERMISSIONS command
+
+Mon Nov 15 01:33:32 2004 Ben Pfaff <blp@gnu.org>
+
+ * q2c.c: (dump_header) Don't try to emit #includes at very top of
+ output file because that will precede #include <config.h>, which
+ is bad.
+ (main) Add needed headers to /* (header) */ code.
+
+Mon Nov 15 01:21:36 2004 Ben Pfaff <blp@gnu.org>
+
+ Instead of making system or portable file readers responsible for
+ dropping and reordering variables, make them read full cases and
+ let the caller take care of any changes.
+
+ * get.c: New "case map" structure to handle this. Use for GET,
+ IMPORT, MATCH FILES. Essentially rewrite the whole file.
+
+ * pfm-read.c: (pfm_read_case) Read into provided case. Signature
+ changed appropriately.
+
+ * sfm-read.c: (sfm_read_case) Ditto.
+
+Mon Nov 15 00:47:45 2004 Ben Pfaff <blp@gnu.org>
+
+ Decided that case_serialize() and case_unserialize() were too
+ abstract. Also we need a couple more functions to avoid excessive
+ copying for data in/out fast paths.
+
+ * case.c: (case_serial_size) Removed.
+ (case_serialize) Rename case_to_values() and make its argument
+ explicitly an array of union values.
+ (case_unserialize) Rename case_from_values() and make its argument
+ explicitly an array of union values.
+ (case_data_all) New function.
+ (case_data_all_rw) New function.
+
+ * casefile.c: (struct casefile) Change buffer from array of
+ unsigned char to array of union value for better accuracy.
+ Redefine buffer_used and buffer_size in terms of values, not
+ bytes. Remove case_size because it is now redundant with
+ value_cnt. Fix up all references to these members.
+
+Mon Nov 15 00:45:46 2004 Ben Pfaff <blp@gnu.org>
+
+ * barchart.c: (struct subcat) Make `label' member const to silence
+ GCC warning with -Wwrite-strings.
+
+ * cartesian.c: (struct dataset) Ditto.
+
+ * case.c: Don't re-define NDEBUG if already defined.
+ Add lots of comments.
+
+ * str.c: Fix includes.
+
+ * crosstabs.q: Fix includes.
+
+ * examine.q: Fix includes. Fix GCC warning about unused
+ variables.
+
+ * frequencies.q: (stat macro) Removed and replaced where used by
+ its expansion.
+
+ * list.q: Fix includes.
+
+ * oneway.q: Fix includes.
+
+ * piechart.c: Fix includes. Only define M_PI if not already
+ defined.
+
+ * sfm-read.c: (bswap) New function.
+ (bswap_int32) Write in terms of bswap.
+ (bswap_flt64) Ditto.
+
+ * str.c: (ds_data) Add external definition here, needed because
+ str.h has only an `extern inline' version.
+
+ * value-labels.c: Fix includes.
+
+Mon Nov 15 00:40:55 2004 Ben Pfaff <blp@gnu.org>
+
+ Instead of providing a system or portable file writer with a raw
+ case in the format needed for output, provide it with a regular
+ case. The writer takes care of any needed translation.
+
+ * aggregate.c: Adopt new scheme for AGGREGATE.
+ (struct agr_proc) sfm_agr_case member removed.
+ (write_case_to_sfm) Removed because the new interface is easier to
+ use.
+
+ * get.c: Adopt new scheme for SAVE, XSAVE, EXPORT.
+
+ * pfm-write.c: Implement new scheme.
+
+ * sfm-write.c: Ditto.
+
+Mon Nov 15 00:32:24 2004 Ben Pfaff <blp@gnu.org>
+
+ Instead of treating `struct file_handle' as a class to subclass
+ into data files, system files, and portable files, instead use it
+ as a helper that coordinates access. Now it is opaque, too.
+
+ This means that most references to a struct file_handle are now
+ changed into references to one of struct dfm_reader, struct
+ dfm_writer, struct sfm_reader, struct sfm_writer, struct
+ pfm_reader, or struct pfm_writer, according to what's being read
+ or written.
+
+ Most related changes are only worth summarizing briefly.
+
+ * dictionary.c: (dict_clear) Destroy aux data in deleted
+ variables.
+ (dict_clear_aux) New function.
+ (dict_create_var) Initialize aux, aux_dtor.
+ (dict_delete_var) Destroy aux data in deleted variable.
+
+ * file-handle.h: (struct fh_ext_class) Removed.
+ (struct file_handle) Removed.
+ (fh_init_files) Removed.
+
+ * file-handle.q: Changed references to a handle's `private' member
+ to direct references.
+ (struct private_file_handle) Renamed file_handle.
+ Add next, open_cnt, type, open_mode, aux members.
+ (struct file_handle_list) Removed.
+ (extern var inline_file) Removed.
+ (static var file_handles) Changed from file_handle_list * to
+ file_handle *.
+ (create_file_handle) Initialize new members.
+ (fh_close_handle) Removed.
+ (mode_name) New function.
+ (fh_open) New function.
+ (fh_close) New function.
+ (fh_parse_file_handle) Renamed fh_parse().
+
+ * glob.c: (init_glob) Remove fh_init_files() call.
+
+ * aggregate.c: use sfm_writer.
+ (create_sysfile) Removed because the new interface is simpler.
+
+ * apply-dict.c: Use sfm_reader.
+
+ * data-list.c: Use dfm_reader.
+
+ * file-type.c: Use dfm_reader.
+
+ * get.c: Use sfm_reader, sfm_writer, pfm_reader, pfm_writer.
+
+ * inpt-pgm.c: Use dfm_reader.
+
+ * print.c: Use dfm_writer.
+
+ * sysfile-info: Use sfm_reader.
+
+ * dfm-read.c: Adopt new file handle infrastructure.
+
+ * dfm-write.c: Ditto.
+
+ * pfm-read.c: Ditto.
+
+ * pfm-write.c: Ditto.
+
+ * sfm-read.c: Ditto.
+
+ * sfm-write.c: Ditto.
+
+Mon Nov 15 00:31:44 2004 Ben Pfaff <blp@gnu.org>
+
+ Break dictionary functions into separate header file.
+
+ * dictionary.h: New file.
+
+ * var.h: Moved dict_*() functions to dictionary.h.
+
+Mon Nov 15 00:30:33 2004 Ben Pfaff <blp@gnu.org>
+
+ Get rid of procedure-specific union in struct variable, using
+ instead a void * pointer and a destructor function.
+
+ Most related changes are only worth brief summaries.
+
+ * crosstabs.q: Fix includes. Use new struct var_range in lieu of
+ old p.crs member in struct variable.
+
+ * frequencies.q: Fix includes. Use new struct var_freqs in lieu
+ of old p.frq member in struct variable.
+
+ * histogram.c: (draw_histogram) Takes new freq_tab arg because
+ it's no longer possible to grab this from var->p.frq.
+
+ * piechart.c: (draw_piechart) Ditto.
+
+ * group.c: (group_proc_get) New function.
+
+ * levene.c: Use group_proc_get() in lieu of old p.grp_data member
+ in struct variable.
+
+ * oneway.q: Ditto.
+
+ * t-test.q: Ditto.
+
+ * main.c: (execute_command) Clear aux data in default_dict after
+ each command. (It's debatable whether this should be done.)
+
+ * matrix-data.c: Use new struct mxd_var in lieu of old p.mxd
+ member in struct variable.
+
+ * means.q: Get rid of integer mode, which is not included in
+ recent SPSS and was the only code that wanted per-variable private
+ data.
+
+ * var.h: (struct crosstab_proc) Removed.
+ (struct frequencies_proc) Removed.
+ (struct list_proc) Removed.
+ (struct get_proc) Removed.
+ (struct means_proc) Removed.
+ (struct matrix_data_proc) Removed.
+ (struct match_files_proc) Removed.
+ (lots of enums) Removed.
+ (struct variable) Removed members `p', `get'. Add member
+ `aux_dtor'.
+
+ * vars-atr.c: (var_attach_aux) New function.
+ (var_detach_aux) New function.
+ (var_clear_aux) New function.
+ (var_dtor_free) New function.
+ (discard_variables) Use NULL instead of inline_file.
+
+Fri Nov 12 10:07:11 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * value-labs.c Fixed the implmentation of value_to_string, so
+ that it properly handles alpha values.
+
+ * oneway.q Changed instances where labels were being probed manually,
+ to use the canonical {var,value}_to_string functions
+
+Thu Nov 11 21:01:31 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q cartesian.c chart.[ch] Added normal and detrended normal
+ plots. Changed the API of the cartesian plot to be a much lower level
+ thing.
+
+Sun Nov 7 17:25:04 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q Added some of the parametric calculations
+
+ * factor_stats.[ch] Created
+
+Sat Nov 6 21:24:31 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q Changed the definition of factors to be a composite, and
+ dealt with the consequences.
+
+Sat Nov 6 20:40:38 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * examine.q Fixed problem where examine wasn't dealing properly with
+ splits
+
+Sat Nov 6 14:49:47 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * oneway.q Fixed problem where oneway wasn't dealing properly with
+ splits
+
+Thu Nov 4 11:09:01 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * q2c.c examine.q Fixed a bug (feature?) whereby arrays in the
+ command which had settings didn't get the appropriate code
+ generated.
+
+ * val.h value-labels.[ch] var-labs.c Added v*to_string functions
+ to convert variables/values to strings.
+
+ * examine.q Added framework for the EXAMINE command.
+
+Mon Nov 1 12:46:17 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * q2c.c frequencies.q set.q t-test.q Fixed the q2c parsing of DBL
+ subcommand types. Changed frequencies.q to use it rather then the
+ custom parser. Dealt with the consequences. Added a test for NTILES
+ subcommand of frequencies.
+
+Sat Oct 30 09:16:29 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * oneway.q Fixed up the behaviour when given missing values
+
+ * levene.c oneway.q Fixed a buglet with the levene statistic and
+ incorporated the levene test into the oneway command.
+
+ * group.h t-test.q Moved the CMP_EQ and CMP_LE symbols out of
+ global scope, since they're only relevant to T-TEST
+
+Fri Oct 29 17:39:03 WST 2004 John Darrington <john@darrington.wattle.id.au>
+
+ * group.c group.h group_proc.h levene.c oneway.q t-test.q
+
+ Made the t-test more consistent
+ with the way it handles groups. That is, it now uses a hash instead
+ of an array of 2. Also, made the levene.c file independent of the
+ implementation of the t-test. So now levene should be fine for both
+ t-test and anova.
+
+ * Added an oneway.q file for one way anova
+
+Wed Jun 2 22:08:02 2004 Ben Pfaff <blp@gnu.org>
+
+ * descript.c: (cmd_descriptives) Remove harmless but bogus test in
+ STATISTICS parsing.
+
+Mon May 31 20:45:24 2004 Ben Pfaff <blp@gnu.org>
+
+ Fix memory leaks.
+
+ * data-list.c: (cmd_data_list) Free dls->delims on lossage.
+ (data_list_trns_free) Free dls->delims.
+
+ * t-test.q: (tts_custom_pairs) Free vars.
+ (ssbox_one_sample_init) Fix tab_vline() argument.
+ (ssbox_independent_samples_init) Ditto.
+ (trbox_paired_init) Ditto.
+ (trbox_one_sample_init) Ditto.
+
+Mon May 31 17:19:27 2004 Ben Pfaff <blp@gnu.org>
+
+ Generalize casefiles to the extent that we can use them for
+ sorting and other kinds of data transformations. Change cases to
+ be copy-on-write to improve memory efficiency in common cases.
+ Every access to a member of a `struct ccase' was changed to be a
+ call to a case_*() function, especially case_data(), case_num(),
+ case_str(), or case_data_rw(). Many instances of a local variable
+ named "case_num" were changed to "case_idx" as a consequence.
+ Many `struct ccase *' were changed to actual `struct ccase'
+ because of copying semantics of cases. In several places there
+ was a choice between updating debug code to work with the new ADTs
+ or just deleting it because it was useless; I chose to delete it.
+
+ * Makefile.am: (pspp_SOURCES) Add case.c, case.h.
+
+ * case.c: New file.
+
+ * case.h: New file.
+
+ * aggregate.c: (struct agr_proc) Change type of `sort' to
+ sort_criteria *. Add `break_vars', `break_var_cnt' members.
+ Rename `vars' to `agr_vars', all references updated. Change
+ `agr_case' to type `struct ccase'.
+ (cmd_aggregate) Deal with new members. Use case_create(),
+ sort_active_file_in_place(), sort_active_file_to_casefile().
+ (agr_destroy) Deal with new members.
+ (aggregate_single_case) Ditto.
+ (dump_aggregate_info) Ditto.
+ (initialize_aggregate_info) Ditto.
+ (agr_to_active_file) Ditto.
+ (presorted_agr_to_sysfile) Ditto.
+ (sort_agr_to_sysfile) Removed.
+
+ * alloc.c: (out_of_memory) Make non-static.
+
+ * alloc.h: Prototype out_of_memory().
+
+ * casefile.c: Switched from a linked list in-memory representation
+ to a two-level array-style representation. The linked list was
+ appropriate when we could stick a header onto cases, but that's no
+ longer the case. Also, the two-level array will allow for random
+ in-memory access in case that's ever wanted. Also added the
+ concept of a `destructive casereader', one that destroys cases in
+ the underlying casefile as they are read out.
+ (macro CASES_PER_BLOCK) New macro.
+ (struct casefile) New members `value_cnt', `case_list_size',
+ `case_acct_size', `being_destroyed', `cases'. Removed `head',
+ `tail'.
+ (struct casereader) Removed `cur'. Added `destructive', `c'.
+ (global var casefiles) Made static.
+ (static var case_bytes) New var.
+ (casefile_create) Takes a value count, not a case size in bytes,
+ to conform to the case interface. All callers updated. Deal with
+ new and removed members.
+ (casefile_destroy) Deal with new and removed members.
+ (casefile_sleep) New function.
+ (casefile_get_case_size) Removed.
+ (casefile_get_value_cnt) New function.
+ (casefile_append) Rewritten to deal with new and removed members.
+ (casefile_append_xfer) New function.
+ (write_case_to_disk) Use case_serialize().
+ (call_posix_fadvise) Removed because posix_fadvise64 segfaults.
+ Couldn't figure out why.
+ (casefile_to_disk) Don't call call_posix_fadvise. Rewritten to
+ deal with new and removed members.
+ (merge) Removed.
+ (merge_sort) Removed.
+ (casefile_sort) Removed.
+ (casefile_get_reader) Deal with new and removed members.
+ (casefile_get_destructive_reader) New function.
+ (reader_open_file) Make code more readable. Create case for
+ reader.
+ (casereader_get_casefile) New function.
+ (casereader_read) Deal with new and removed members. Now returns
+ a copy of the case, so that the caller is responsible for
+ destroying the returned case.
+ (casereader_read_xfer) New function.
+ (casereader_destroy) Destroy reader's case.
+ (test_casefile) Second arg is now a value count, all callers
+ updated. Now tests destructive readers too.
+ (get_random_case) Deal with new case ADT.
+ (write_random_case) Ditto.
+ (read_and_verify_random_case) Ditto.
+
+ * crosstabs.q: Remove debug code.
+
+ * descript.q: (calc_descriptives) Deal with new case, casefile
+ ADTs.
+
+ * dfm.c: (cmd_begin_data) There's no storage_source_class anymore.
+
+ * do-if.c: Remove unneeded header inclusion.
+
+ * expr-prs.c: Remove debug code.
+
+ * exprP.h: Remove debug code.
+
+ * flip.c: (flip_file) Use fseeko() if available.
+
+ * formats.c: Remove debug code.
+
+ * get.c: Remove debug code.
+ (struct mtf_file) Change `input' from `union value *' to `struct
+ ccase', all references updated.
+
+ * levene.c: (levene) Deal with new case, casefile ADTs.
+
+ * list.q: Remove debug code.
+
+ * loop.c: Remove debug code.
+
+ * matrix-data.c: Remove debug code.
+
+ * means.q: Remove debug code.
+
+ * mis-val.c: Remove debug code.
+
+ * pfm-read.c: Remove debug code.
+ (pfm_read_code) Change second arg from `union value *' to `struct
+ ccase *', all references updated.
+
+ * recode.c: (string_to_long) Make first arg const.
+ (convert_to_double) Ditto.
+
+ * repeat.c: Remove debug code.
+
+ * sample.c: Remove debug code.
+
+ * sfm-read.c: Remove debug code.
+ (sfm_read_case) Change second arg from `union value *' to `struct
+ ccase *'.
+
+ * sort.c: Redone in terms of casefiles.
+ (enum sort_direction) Moved here from sort.h.
+ (struct sort_criterion) New structure.
+ (struct sort_criteria) New structure.
+ (cmd_sort_cases) Rewritten.
+ (prepare_to_sort_active_file) New function.
+ (sort_active_file_in_place) New function.
+ (sort_active_file_to_casefile) New function.
+ (parse_sort) Renamed sort_parse_criteria(), rewritten & interface
+ changed, all callers updated.
+ (destroy_sort_cases_pgm) Renamed sort_destroy_criteria(),
+ rewritten & interface changed, all callers updated.
+ (sort_cases) Renamed sort_execute(), rewritten & interface
+ changed, all callers updated.
+ (struct internal_sort) Removed.
+ (do_internal_sort) Rewritten, interface changed.
+ (destroy_internal_sort) Removed.
+ (compare_case_dblptrs) Use sort_criteria instead of sort_case_pgm.
+ (struct initial_run) Removed; an initial run is now just a
+ casefile.
+ (compare_initial_runs) Rewritten.
+ (struct external_sort) Changed almost completely.
+ (do_external_sort) Rewritten, interface changed.
+ (destroy_external_sort) Rewritten.
+ [HAVE_MKDTEMP] (make_temp_dir) Removed.
+ [!HAVE_MKDTEMP] (do_mkdir) Removed.
+ [!HAVE_MKDTEMP] (make_temp_dir) Removed.
+ (init_external_sort) Removed.
+ (simulate_error) Removed.
+ (rmdir_temp_dir) Removed.
+ (get_temp_file_name) Removed.
+ (open_temp_file) Removed.
+ (close_temp_file) Removed.
+ (remove_temp_file) Removed.
+ (write_temp_file) Removed.
+ (read_temp_file) Removed.
+ (struct record_run) Change `record' from `struct case_lit *' to
+ `struct ccase'.
+ (struct initial_run_state) Remove `idx_to_fv', `free_list',
+ `file_idx', `output_file'. Add `run', casefile'. Change
+ `last_output' from `struct case_list *' to `struct ccase'.
+ (write_initial_runs) Change interface, rewrite.
+ (sort_sink_write) Renamed process_case(), changed interfaced,
+ rewrote.
+ (destroy_initial_run_state) Rewritten.
+ (allocate_cases) Rewritten.
+ (compare_record) Interface changed, rewritten.
+ (start_run) Rewritten.
+ (end_run) Rewritten.
+ (output_record) Rewritten.
+ (grab_case) Removed.
+ (release_case) Removed.
+ (struct merge_case) Change `cases' from double pointer to single
+ pointer.
+ (merge) Deal with new case and casefile ADTs.
+ (struct run) Removed.
+ (merge_once) Rewritten, interface changed.
+ (fill_run_buffer) Removed.
+ (sort_sink_make_source) Removed.
+ (sort_sink_class) Removed.
+ (struct sort_source_aux) Removed.
+ (sort_source_read_helper) Removed.
+ (sort_source_read) Removed.
+ (read_sort_output) Removed.
+ (read_internal_sort_output) Removed.
+ (read_external_sort_output) Removed.
+ (sort_source_destroy) Removed.
+ (sort_source_class) Removed.
+
+ * sort.h: (struct sort_cases_pgm) Removed.
+ (enum sort_direction) Moved to sort.c.
+
+ * t-test.q: (calculate) Deal with new case, casefile ADTs.
+
+ * tab.c: Remove debug code.
+
+ * var-labs.c: Remove debug code.
+
+ * var.h: (struct ccase) Removed.
+ (struct case_list) Removed.
+
+ * vars-atr.c: (discard_variables) Use free_case_source().
+
+ * vars-prs.c: (parse_vs_variable) Make arg const.
+ (parse_dict_variable) Ditto.
+ (parse_variables) Make struct dictionary * arg const.
+ (parse_var_set_vars) Make struct var_set * arg const.
+ (struct var_set) Add const to some of the function pointers' args.
+ (var_set_get_cnt) Make arg const.
+ (var_set_get_var) Make first arg const.
+ (var_set_lookup_var) Make first arg const.
+ (dict_var_set_get_cnt) Make arg const.
+ (dict_var_set_get_var) Make first arg const.
+ (dict_var_set_lookup_var) Make first arg const.
+ (var_set_create_from_dict) Make arg const. Add cast to aux
+ assignment.
+ (struct array_var_set) Add const to var member.
+ (array_var_set_get_cnt) Make arg const.
+ (array_var_set_get_var) Make first arg const.
+ (array_var_set_lookup_var) Make first arg const.
+ (var_set_create_from_array) Make first arg const. Insert cast.
+
+ * vfm.c: (struct write_case_data) Change trns_case, sink_case
+ members from `struct ccase *' to `struct ccase'.
+ (static var lag_queue) Change from double to single pointer.
+ (procedure) Optimize trivial case.
+ (internal_procedure) Deal with changed case, case_source ADTs.
+ (create_trns_case) Changed interface, rewrote.
+ (open_active_interface) Initialize modified lag queue.
+ (write_case) Deal with changed case ADT.
+ (lag_case) Deal with modified lag queue.
+ (close_active_file) Destroy modified lag queue.
+ Deal with changed case_source, case_sink ADTs.
+ (destroy_storage_stream_info) Make null arg into no-op.
+ (storage_sink_make_source) Set aux in created source.
+ (storage_source_read) Deal with changed case, casefile ADTs.
+ (storage_source_create) New function.
+ (lagged_case) Rewrite.
+ (free_case_source) New function.
+ (free_case_sink) Rewrite.
+ (struct split_aux_data) Changed prev_case from `struct ccase *' to
+ `struct ccase'.
+ (procedure_with_splits) Deal with changed prev_case.
+ (procedure_with_splits_callback) Ditto.
+ (multipass_split_aux_data) Changed prev_case from `struct ccase *' to
+ `struct ccase'.
+ (multipass_procedure_with_splits) Deal with changed prev_case.
+ (multipass_split_callback) Ditto.
+
+
+Mon May 31 17:19:06 2004 Ben Pfaff <blp@gnu.org>
+
+ The workspace idea didn't work out.
+
+ * Makefile.am: (pspp_SOURCES) Remove workspace.c, workspace.h.
+
+ * workspace.c: Removed.
+
+ * workspace.h: Removed.
+
+Sun May 30 18:35:19 2004 Ben Pfaff <blp@gnu.org>
+
+ Fully implement arbitrary delimiters on DATA LIST, extending the
+ half implementation that was already there.
+
+ * data-list.c: (struct data_list_pgm) Remove `delim', add
+ `delims', `delim_cnt'.
+ (cmd_data_list) Initialize new members. Parse delimiters and
+ clean up code a bit.
+ (cut_field) Extract fields with arbitrary delimiters. Also, fix
+ handling of leading commas.
+ (read_from_data_list_fixed) Expand tabs. Adapt to new DFM
+ interfaces.
+ (read_from_data_list_free) Adapt to new DFM interfaces.
+ (read_from_data_list_list) Ditto.
+ (repeating_data_trns_proc) Ditto.
+
+ * dfm.c: Split up reader and writer into separate code, because
+ they do different things. Use struct string instead of explicit
+ allocation code, for clarity.
+ (enum dfm_reader_flags) New enum.
+ (struct dfm_fhuser_ext) Removed.
+ (struct dfm_reader_ext) New.
+ (get_reader) New function, used by just about all the reader
+ functions.
+ (dfm_close) Removed.
+ (close_reader) New function.
+ (dfm_open_for_reading) Rewrite initialization of dfm_fhuser_ext.
+ (dfm_open_for_writing) Ditto.
+ (macro force_line_buffer_expansion) Removed.
+ (count_tabs) Removed.
+ (tabs_to_spaces) Removed.
+ (read_record) Deal with new dfm_reader_ext. Use struct string
+ functions. Don't convert tabs to spaces.
+ (dfm_eof) New function.
+ (dfm_get_record) Changed interface, rewrote.
+ (dfm_expand_tabs) New function.
+ (dfm_fwd_record) Renamed dfm_forward_record(), updated to new
+ dfm_reader_ext, rewritten.
+ (dfm_bkwd_record) Renamed dfm_reread_record(), updated to new
+ dfm_reader_ext, rewritten.
+ (dfm_set_record) Removed in favor of dfm_forward_columns().
+ (dfm_forward_columns) New function.
+ (dfm_get_cur_col) Renamed dfm_column_start, updated to new
+ dfm_reader_ext, rewritten.
+ (static var dfm_r_class) Use close_reader for the destructor.
+ (struct dfm_writer_ext) New.
+ (dfm_put_record) Updated to new dfm_writer_ext, rewritten. Uses
+ bounce buffer now instead of local allocation.
+ (close_writer) New function.
+ (static var dfm_writer_ext) Use close_writer for destructor.
+ (cmd_begin_data) Adapt to new dfm_reader_ext.
+
+ * file-handle.q: Add support for per-file tab width.
+ (struct private_file_handle) Add tab_width member.
+ (q2c specifications) Add tabwidth subcommand.
+ (cmd_file_handle) Put parsed tab width into private_file_handle.
+ (create_file_handle) Set default tab width.
+ (handle_get_tab_width) New function.
+
+ * file-type.c: (file_type_source_read) Adapt to new DFM interface.
+
+ * inpt-pgm.c: (reread_trns_proc) Ditto.
+
+ * matrix-data.c: (context) Ditto.
+ (another_token) Ditto.
+ (mget_token) Ditto.
+ (force_eol) Ditto.
+
+Sun May 30 18:33:59 2004 Ben Pfaff <blp@gnu.org>
+
+ * casefile.c: (casefile_destroy) Fix memory leak by freeing
+ cf->filename.
+ (casereader_destroy) Don't close file descriptor -1.
+
+ * recode.c: (cmd_recode) Fix memory leak.
+
+ * set.q: (q2c specifications) Fix typo in user message.
+
+ * str.c: (st_bare_pad_len_copy) Change memcpy to memmove to avoid
+ undefined behavior for overlapping arguments.
+
+Sun May 30 18:31:48 2004 Ben Pfaff <blp@gnu.org>
+
+ * casefile.c: valgrind doesn't implement posix_fadvise() yet, so
+ don't call it when we're running under valgrind.
+ (call_posix_fadvise) New function.
+ (casefile_to_disk) Use call_posix_fadvise().
+ (reader_open_file) Ditto.
+
+Sun May 30 18:20:12 2004 Ben Pfaff <blp@gnu.org>
+
+ Update our string ADTs, struct string and struct len_string. Get
+ rid of pool support, which was largely unused. Rename lots of
+ functions to have more obvious or consistent names.
+
+ * ascii.c: Get rid of ascii_pool. It was only used for string
+ allocations.
+ (ascii_open_global) Don't create ascii_pool.
+ (ascii_close_driver) Don't destroy ascii_pool.
+ (ascii_postopen_driver) Don't use pool.
+ (ascii_close_driver) Destroy strings manually.
+
+ * str.c: (ds_create) Remove pool argument, all references updated.
+ (ds_init) Ditto.
+ (ds_replace) Remove pool support, make more efficient when we
+ don't need to reallocate.
+ (ds_destroy) Remove pool support.
+ (ds_rpad) New function.
+ (ds_size) Renamed ds_capacity(), all references updated.
+ (ds_value) Renamed ds_c_str(), all references updated.
+ (ds_concat) Renamed ds_puts(), all references updated.
+ (ds_concat_buffer) Renamed ds_concat(), all references updated.
+ (ds_putchar) Renamed ds_putc(), all references updated.
+ (ds_getline) Renamed ds_gets(), all references updated.
+ (ls_create) Remove pool argument, all references updated.
+ (ls_create_buffer) Ditto.
+ (ls_destroy) Removed pool support.
+ (ls_value) Renamed ls_c_str(), all references updated.
+
+ * str.h: (ls_length) [__GNUC__] Add inline version.
+ (ls_c_str) [__GNUC__] Add inline version.
+ (ls_end) [__GNUC__] Add inline version.
+ (struct string) Remove pool member. Rename `size' to `capacity',
+ all references updated.
+
+ * tab.c: (text_format) Instead of using pool argument to
+ ls_create_buffer(), call pool_register() on allocated data.
+
Mon Apr 26 22:40:07 2004 Ben Pfaff <blp@gnu.org>
We're abusing the current ASCII driver by telling it to allocate a