X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=src%2FChangeLog;h=1620db441e32956bb546367e675eeb51c5db859a;hb=b7e33825d30a18360f24a18faf4b7d2e9efb8142;hp=14de86216b6982d1c070c29bab6c64fdd258f28b;hpb=6d34b6cd00bb26603f8acedd3f0ed69a4484e484;p=pspp-builds.git diff --git a/src/ChangeLog b/src/ChangeLog index 14de8621..1620db44 100644 --- a/src/ChangeLog +++ b/src/ChangeLog @@ -1,3 +1,2051 @@ +Thu Apr 14 2005 John Darrington + + * var-display.c: New file. + + * format.h var.h sfm-read.c sfm-write.c dictionary.c : Added + display_width, measure and alignment parameters to variables. + + * aggregate.c command.def compute.c count.c data-list.c descript.c + dictionary.c dictionary.h expr-prs.c file-type.c flip.c get.c + lexer.c lexer.h loop.c modify-vars.c pfm-read.c recode.c repeat.c + sfm-read.c sfm-write.c sfm-write.h sfmP.h val-labs.c val.h var.h + vars-prs.c vector.c : + - Replaced literal constants representing maximum variable name + length with macro definitions. + - Added support for long variable names. + - Changed lexer such that it no longer makes tokens upper + case, but relies upon case insensitive behaviour of commands. + +Mon Apr 4 22:27:34 2005 Ben Pfaff + + * aggregate.c: (parse_aggregate_functions) If dict_create_var() + fails, don't dereference the resulting null pointer (PR 12427). + Also, fix double free error. + +Sat Mar 19 23:06:02 2005 Ben Pfaff + + * aggregate.c: (parse_aggregate_functions) Fix N_NO_VARS format. + (accumulate_aggregate_info) Set int1 to 1 for SUM. + (dump_aggregate_info) Only make SUM non-missing if there was at + least one variate. + +Sat Mar 19 14:48:19 2005 Ben Pfaff + + * aggregate.c: (dump_aggregate_info) Properly test whether the + destination variable is numeric, when making the result + system-missing for columnwise missing values. + +Mon Mar 14 21:52:34 2005 Ben Pfaff + + * misc.h: Remove GCC specializations. + +Mon Mar 14 21:07:23 2005 Ben Pfaff + + Make sort stable (PR 12313). + + * sort.c: Don't need to include some headers anymore. + (static var min_buffers) New variable. + (static var max_buffers) New variable. + (static var allow_internal_sort) New variable. + (cmd_sort_cases) Add test mode. + (sort_execute) Rephrase. + (do_internal_sort) Don't try internal sorting if + allow_internal_sort is set. + (struct external_sort) Renamed `initial_runs' to `runs' and + updated all references. + (macro MIN_BUFFER_TOTAL_SIZE_RECS) Removed. + (macro MIN_BUFFER_SIZE_BYTES) Removed. + (macro MIN_BUFFER_SIZE_RECS) Removed. + (compare_initial_runs) Removed. + (struct record_run) Add member `idx'. + (write_initial_runs) Pass increasing values to process_case() as + index. + (process_case) Add `idx' parameter and use it to initialize new + `idx' member. + (allocate_cases) Limit allocated buffers to max_buffers. + (compare_record_run) Use new `idx' member for last resort + comparison, for stability. + (end_run) Call casefile_sleep() on irs->casefile, to prevent + running out of file descriptors. + (struct merge_state) Removed. + (merge) Don't need to allocate cases. Always use MAX_MERGE_ORDER + unless we have fewer runs left. Always merge consecutive runs, + for stability. Return the final run. + (mod) Removed. + (choose_merge) New function. + (merge_once) Rewritten. + +Mon Mar 14 21:05:42 2005 Ben Pfaff + + * cmdline.c: (static var testing_mode) Move into + parse_command_line(). + +Mon Mar 14 21:05:13 2005 Ben Pfaff + + * algorithm.c: (remove_range) New function. + (remove_element) New function. + + * dictionary.c: (dict_delete_var) Use remove_element(). + + * flip.c: (cmd_flip) Ditto. + +>>>>>>> 1.161 +Sun Mar 13 22:52:05 2005 Ben Pfaff + + * file-handle.q: (struct file_handle) `open_mode' should not be + const. + +Sun Mar 13 22:40:54 2005 Ben Pfaff + + First phase of making SORT CASES stable (PR 12313). + + * sort.c: (struct indexed_case) New structure. + (do_internal_sort) Rewrite to make internal sorting stable. + (compare_case_dblptrs) Removed. + (compare_indexed_cases) New function. + +Sun Mar 13 22:38:40 2005 Ben Pfaff + + Clean-ups. + + * casefile.c: (casereader_read_xfer_assert) New function. + + * dictionary.c: (dict_compact_case) New function. + + * flip.c: (struct flip_pgm) New member idx_to_fv. + (cmd_flip) Initialize idx_to_fv member. + (destroy_flip_pgm) Free idx_to_fv member. + (flip_sink_write) Use struct flip_pgm member instead of case_sink + member. + (flip_sink_write) Ditto. + + * vfm.c: (write_case) Use dict_compact_case() instead of + compact_case(). + (compact_case) Removed. + (storage_source_create) Removed `dict' parameter. All references + updated. + + * vfm.h: (struct case_source) Removed `value_cnt' member. All + references removed. + (struct case_sink) Removed `dict', `idx_to_fv' members. All + references removed. + +Sun Mar 13 22:35:55 2005 Ben Pfaff + + More AGGREGATE fixes. + + * aggregate.c: (accumulate_aggregate_info) Implement NMISS and + NUMISS for strings. Fix FOUT, POUT, FGT, FLT, FIN, FOUT for + strings. + (initialize_aggregate_info) Fix initialization for MIN, MAX for + strings. + +>>>>>>> 1.160 +Sat Mar 12 23:26:21 2005 Ben Pfaff + + Start work on testing and debugging AGGREGATE. + + * aggregate.c: (cmd_aggregate) Use discrete bool variables instead + of a bit-map. Require proper subcommand ordering. Make OUTFILE + subcommand mandatory. + (parse_aggregate_functions) Check that PIN, POUT, FIN, FOUT + functions' arguments are in the correct order and swap them if + not. + (accumulate_aggregate_info) Make SUM include weights. Add various + string functions. + (dump_aggregate_info) Add various string functions. + (initialize_aggregate_info) Initialize int1 for MIN, MAX. + + * sort.c: (sort_parse_criteria) Add parameter for returning + whether any directions were specified. All callers updated. + +Sun Mar 13 14:54:27 WST 2005 John Darrington + + * t-test.q: Fixed erroneous logic in compare_group_binary. + +Sat Mar 12 13:29:21 2005 Ben Pfaff + + * split-file.c: (cmd_split_file) Ignore LAYERED and SEPARATE + keywords (PR 11628). + +Sat Mar 12 13:17:12 2005 Ben Pfaff + + * vfm.c: (procedure_with_splits) Fix PR 11492: end_func() must be + called *before* close_active_file(). + +Sat Mar 12 12:20:57 2005 Ben Pfaff + + * file-handle.q: (struct file_handle) Change open_mode from + character pointer to 3-char array, for safety. Updated all + references. + +Sat Mar 12 12:15:49 2005 Ben Pfaff + + Thanks to Ben Kujala for reporting + these bugs. + + * pfm-read.c: (read_header) Improve error message for many cases + in which the input is not actually a portable file. + + * file-handle.q: (fh_open) When we give an error message, actually + return NULL. + +Fri Mar 11 11:50:30 2005 Ben Pfaff + + * format.c: (check_common_specifier) New function for checks + common to check_input_specifier() and check_output_specifier(). + (check_input_specifier) Use check_common_specifier(). + (check_output_specifier) Use check_common_specifier(). + (check_string_specifier) Removed. + (check_specifier_type) New function. + (check_specifier_width) New function. + (get_format_var_width) Fix bug. + + * formats.c: (internal_cmd_formats) Only accept numeric variables. + + * lexer.c: (check_id) Rename lex_id_to_token(), make public, + update all references. Make case-insensitive. + + * pfm-read.c: Essentially rewrite the whole file. Now much + cleaner. + + * print.c: (check_string_width) New function. + (fixed_parse_compatible) Use check_string_width(), + check_specifier_type(). + (dump_fmt_list) Ditto. + + * str.c: (st_trim_copy) New function. + (st_uppercase) New function. + + * vars-atr.c: (var_is_valid_name) New function. + + * expressions/parse.c: (type_coercion_core) Use + check_specifier_type(). + +Fri Mar 11 11:31:24 2005 Ben Pfaff + + * expressions/evaluate.c: (cmd_debug_evaluate) Fix memory leaks. + + * expressions/parse.c: (no_match) Ditto. + +Wed Mar 9 09:54:27 2005 Ben Pfaff + + * Makefile.am: (pspp_LDADD) Add libgsl-extras.a. + + * expressions/helpers.c: (struct func_params) Removed. + (generalized_idf) Removed. + (cdf_beta) Removed. + (idf_beta) Removed. + (idf_fdist) Use gslextras_cdf_beta_Pinv() instead of idf_beta(). + + * expressions/operations.def: Implement IDF.BETA, CDF.BINOM, + CDF.GEOM, CDF.HYPER, CDF.NEGBIN, CDF.POISSON using gsl-extras. + Implement SIG.F, which I had overlooked previously. + +Tue Mar 8 12:47:53 WST 2005 John Darrington + + * command.c command.def glob.[ch] cmdline.c: Made DEBUG cmds + available only in testing mode. + +Sun Mar 6 23:25:40 2005 Ben Pfaff + + * data-in.c: Use `bool' throughout, where relevant. + +Sun Mar 6 19:52:22 2005 Ben Pfaff + + DATA LIST with free-field formats should not have implied decimal + places (PR 12035). Also clean up data-in.c a bit. + + * data-in.h: (enum) Add DI_IMPLIED_DECIMALS. + + * data-in.c: (apply_implied_decimals) New function. + (parse_numeric) Don't adjust exponent if DI_IMPLIED_DECIMALS not + set. Also, get rid of gotos. + (parse_Z) Use apply_implied_decimals() if the field doesn't + contain a decimal point. + (parse_N) Use apply_implied_decimals(). + (parse_IB) Ditto. + (parse_PIB) Ditto. + (parse_P) Ditto. + (parse_PK) Ditto. + (to_roman) Removed. + (parse_enum) New function. + (macro CHAR_IS_ROMAN) Removed. + (macro ROMAN_VALUE) Removed. + (parse_month) Use parse_enum(). + (parse_weekday) Use parse_enum(). + (parse_DATETIME) Use long for weekday. + + * data-list.c: (read_from_data_list_fixed) Use + DI_IMPLIED_DECIMALS. + +Sun Mar 6 17:07:20 2005 Ben Pfaff + + When the lexer sees something like `-5' in the input, it has to + decide whether it's a negative numeric constant token or a '-' + token followed by a positive numeric constant token. It always + decides on the former, and then the parser can call + lex_negative_to_dash() if it wants the latter. However, this + doesn't work for the case of `-0', because negative zero is + (portably) indistinguishable from positive zero. So now we divide + T_NUM into two tokens, T_POS_NUM and T_NEG_NUM, to make the + distinction clear. This requires a little bit of extra effort, + because there were several references to T_NUM in the code base. + + * lexer.c: (lex_get) Use T_NEG_NUM and T_POS_NUM to distinguish + positive and negative numeric constants. + (lex_double_p) Renamed lex_is_number(). Changed return type to + bool. Updated all relevant references to T_NUM to instead use + this function. + (lex_double) Renamed lex_number(). All references updated. + (lex_integer_p) Renamed lex_is_integer(). Changed return type to + bool. All references updated. + (lex_token_representation) Understand T_NEG_NUM and T_POS_NUM. + (lex_negative_to_dash) Ditto. + (dump_token) Ditto. + + * lexer.h: (enum) Add T_POS_NUM, T_NEG_NUM. Remove T_NUM. + +Sun Mar 6 22:09:20 2005 Ben Pfaff + + * expressions/operations.def: (NUMBER) Use DI_IMPLIED_DECIMALS. + +Sun Mar 6 19:33:24 2005 Ben Pfaff + + * expressions/operations.def: (VEC_ELEM_NUM) Treat user-missing + values as system-missing. + + * expressions/parse.c: (parse_vector_element) Fix order of + arguments in call to expr_allocate_binary(). + +Sun Mar 6 17:51:05 2005 Ben Pfaff + + * expressions/optimize.c: (optimize_tree) Fix optimization bug for + x**2. + + * expressions/parse.c: (type_coercion_core) Set *node to NULL on + failure, as indicated by function comment. + (parse_binary_operators) Always return NULL on type_coercion() + failure. Should have been doing this anyway, but bug in + type_coercion_core() filtered through. + (parse_add) Fix typo in user message. + (parse_primary) Understand T_NEG_NUM and T_POS_NUM. + +Sun Mar 6 10:47:13 2005 Ben Pfaff + + * expressions/operations.def: Add VALUE function. + + * expressions/parse.c: (parse_function) Need an unary composite + node for variables in A TO B, not a variable node. Use + allocate_unary_variable(). + (parse_primary) Use allocate_unary_variable(). + (allocate_unary_variable) New function. + +Thu Mar 3 23:53:32 2005 Ben Pfaff + + * expressions/PSPP_expressions.pm: Renamed it back to generate.pl + but fixed the real problem that was preventing the build from a + separate directory. I liked it my way better ;-) + +Thu Mar 3 23:17:51 2005 Ben Pfaff + + * expressions/parse.c: (expr_parse) Fix parameter type. Thanks to + John Darrington for reporting this + bug. + +Thu Mar 3 22:10:25 WST 2005 John Darrington + + * expressions/Makefile.am expressions/evaluate.h.pl + expressions/evaluate.inc.pl expressions/operations.h.pl + expressions/optimize.inc.pl expressions/parse.inc.p: + + Renamed generate.pl to PSPP_expressions.pm and adjusted *.pl + to suit. + + Fixed everything so that it can be built from an arbitrary + directory. + +Thu Mar 3 22:08:35 WST 2005 John Darrington + + * Makefile.am : Fixed up CLEANFILES target. + +Mon Feb 28 23:49:56 2005 Ben Pfaff + + * str.h: Changed `struct len_string' to `struct fixed_string', a + more accurate name. Updated all references. + +Mon Feb 28 23:35:30 2005 Ben Pfaff + + Redo calendar support. Should now be bug-for-bug compatible. + + * calendar.c: New file. + + * calendar.h: New file. + + * data-in.c: Use new calendar functions. + (parse_sign) Change sense of return value. + (calendar_error) New function. + (ymd_to_ofs) New function. + (ymd_to_date) New function. + (parse_DATE) Use new function. + (parse_ADATE) Ditto. + (parse_EDATE) Ditto. + (parse_SDATE) Ditto. + (parse_JDATE) Ditto. + (parse_QYR) Ditto. + (parse_MOYR) Ditto. + (parse_WKYR) Ditto. + (parse_TIME) Ditto. + (parse_DTIME) Ditto. + (parse_DATETIME) Ditto. + + * data-out.c: (convert_date) Use new calendar functions. + + * error.c: (err_vmsg) Changed interface to be more sensible. + Updated all callers. + (dump_message) Don't double new-lines (why did we do this + anyway?). + +Mon Feb 28 23:30:25 2005 Ben Pfaff + + * sfmP.h: (macro flt64) Moved here from pref.h.orig. + (macro FLT64_MAX) Moved here from pref.h.orig. + +Mon Feb 28 23:28:01 2005 Ben Pfaff + + * set.q: Support SET EPOCH. + (static var set_epoch) New var. + (aux_stc_custom_epoch) New function. + (stc_custom_epoch) New function. + (get_epoch) New function. + (stc_custom_pager) [USE_INTERNAL_PAGER] Fix bug. + + * format.c: Make it possible just to check whether a specifier is + valid without emitting an error message. + (parse_format_specifier_name) Change interface, update all + callers. + (check_input_specifier) Ditto. + (check_output_specifier) Ditto. + (parse_format_specifier) Ditto. + +Mon Feb 28 23:24:08 2005 Ben Pfaff + + * command.def: Add DEBUG POOL. + + * pool.c: (pool_destroy) Fix bug in deleting this pool from its + parent. + (pool_clear) Properly account for size of pool gizmo. + (pool_realloc) Ditto. + (pool_clone) New function. + + * pool.h: Mark our allocation functions MALLOC_LIKE. + +Mon Feb 28 23:21:26 2005 Ben Pfaff + + * Makefile.am: Move many definitions into new top-level + Make.build. Add expressions to SUBDIRS. Add calendar.c, + calendar.h. Remove expr-evl.c, expr-opt.c expr-prs.c, expr.h, + exprP.h, expr.def. + + * case.c: (case_resize) New function. + (case_swap) New function. + + * casefile.c: Include mkfile.h. + +Fri Feb 25 21:11:35 WST 2005 John Darrington + + * sfm-read.c: Fixed a buglet which caused a crash when trying + to read a non-existent file. + +Sun Feb 13 16:11:13 2005 Ben Pfaff + + Fix PR 11955. + + * aggregate.c: (parse_aggregate_functions) Code cleanup. + Important part: get rid of spurious copying of function->format to + destvar->print and destvar->write. + +Fri Feb 11 00:08:36 2005 Ben Pfaff + + Fix PR 11916, which was confusing a variable's `index' member + with the variable's position in a var_set. Although these are + usually the same, they are not for array `var_set's. + + Took advantage of this bug as an opportunity to clean up and + rewrite parse_var_set_vars(). + + * vars-prs.c: (parse_vs_variable_idx) New function. + (parse_vs_variable) Reimplement in terms of + parse_vs_variable_idx(). + (parse_var_idx_class) New function. + (add_variable) New function. + (add_variables) New function. + (parse_var_set_vars) Rewritten. + (struct var_set) Change `lookup_var' member that returns a + variable into `lookup_var_idx' member that returns an int. + Updated the var set implementations in obvious corresponding ways. + Used compare_var_ptr_names(), hash_var_ptr_name() just added. + +Fri Feb 11 00:06:03 2005 Ben Pfaff + + Use our global variable compare & hash functions and give them + better names. Add similar functions for dealing with double + pointers to variables. + + * vars-atr.c: (compare_variables) Renamed compare_var_names(). + (hash_variable) Renamed hash_var_name(). + (compare_var_ptr_names) New function. + (hash_var_ptr_name) New function. + + * t-test.q: (cmd_t_test) Use global compare_var_names(), + hash_var_name(). + (compare_var_name) Removed. + (hash_var_name) Removed. + +Fri Feb 11 00:04:39 2005 Ben Pfaff + + Fix dictionary bug. + + * dictionary.c: (compare_variable_dblptrs) Rename + compare_var_ptrs() and fix it to properly dereference the double + pointers. + +Mon Feb 7 09:58:15 WST 2005 John Darrington + + crosstabs.q examine.q oneway.q q2c.c: Added a q2c feature to + declare subcommands as mandatory. Closed bug #11843 + +Sat Feb 5 20:35:10 WST 2005 John Darrington + + * getline.c command.[ch] command.def: Added (very rudimentary) + support for line completion when in interactive mode. Partially + addresses bug #11693 + +Mon Jan 31 09:52:51 WST 2005 John Darrington + + * examine.q factor_stats.c oneway.q output.c pfm-read.c: Fixed some + problems revealed by valgrind. + + +Wed Jan 26 11:44:11 WST 2005 John Darrington + + * set.q: Affixed a fix to the previous fix such that we'll be OK now + whether or not PAGER is set. + +Wed Jan 26 09:25:54 WST 2005 John Darrington + + * set.q: Copied the string produced by getenv("PAGER") thus avoiding + "invalid free" errors. Hopefully fixes bug #11722 + + * compute.c expr-prs.c: Check that lvalues are populated before + attempting to destroy them. Closes bug #11676 + +Tue Jan 25 21:01:43 WST 2005 John Darrington + + * aggregate.c: Initialised the complete agr_proc structure. + Closes bug #11675 + + +Sun Jan 23 23:02:21 2005 Ben Pfaff + + * print.c: (print_trns_free) Close the dfm writer if there is one, + fixing a memory leak. + +Mon Jan 24 12:24:36 WST 2005 John Darrington + + * glob.c oneway.q q2c.c t-test.q vfm.c: Still *more* memory leaks + fixed. + + +Fri Jan 21 19:54:14 WST 2005 John Darrington + + * linked-list.[ch] Added + + * examine.q file-handle.[hq] font.h glob.c groff-font.c postscript.c + set.q: Yet more memory leaks + +Tue Jan 18 23:12:40 WST 2005 John Darrington + + * t-test.q examine.q : More memory leaks fixed. + +Tue Jan 18 19:26:59 WST 2005 John Darrington + + * examine.q factor_stats.[ch] get.c pfm-read.c: Plugged numerous + memory leaks. + +Mon Jan 10 14:43:45 WST 2005 John Darrington + + * ascii.c cartesian.c casefile.c chart.h devind.c + examine.q frequencies.q + html.c output.h piechart.c plot-chart.c plot-hist.c + + Integrated the chart rendering into the output stream + (currently only works for html). + + Removed gratuitous use of ifndef NO_CHARTS, and replaced with + dummy-chart.c for compiling without charts. + + * mkfile.[ch] Created. + + * som.[ch] tab.[ch]: Changed name of som_table to som_entity + Added type element so we can tell if it's a chart or a table. + + * chart.h examine.q piechart.c plot-chart.c plot-hist.c: changed the + API of charts to be more like that of tables. + +Thu Jan 13 21:00:02 WST 2005 John Darrington + + * casefile.c main.c: Moved the SIGINT handler from casefile.c to + main.c. Removed the handler for SIGQUIT. + +Mon Jan 10 14:43:45 WST 2005 John Darrington + + * casefile.c: Added a signal handler to delete temp files on + SIGINT and SIGQUIT + + * permissions.c Inhibited the PERMISSIONS command when SAFER is on. + + * command.def Added a lot more unimplemented commands. + + * copyleft.[ch] cmdline.c Moved legal information to copyleft.c + +Sat Jan 8 23:58:34 2005 Ben Pfaff + + * sort.c: (compare_initial_runs) Needed additional level of + dereferencing. + (merge_once) Fix plenty of stupid mistakes. + +Sat Jan 8 23:55:27 2005 Ben Pfaff + + * casefile.c: (casefile_sleep) Need to flush_buffer() after + calling casefile_to_disk() or we will lose the last block in the + file if the casefile started out as disk-based. + (casefile_get_reader) Initialize reader->destructive to 0. + (cmd_debug_casefile) Add new test pattern. + (test_casefile) Define new test pattern to make sure + casefile_sleep() works properly. + +Fri Jan 7 08:00:05 WST 2005 John Darrington + + * Makefile.am chart.[ch] histogram.[ch] piechart.c (Modified); + plot-hist.c plot-chart.c (Added) Reorganised these files in an + attempt to seperate the creation and processing of charts from their + actuall renedering. + + * examine.q frequencies.q generated charts conditional upon the NO_CHARTS + macro. + +Thu Jan 6 18:48:58 WST 2005 John Darrington + + * main.c Added a signal handler for SIGFPE + + * sort.c Somewhat more robust fix to the previous entry. + +Wed Jan 5 21:23:31 2005 Ben Pfaff + + * sort.c: (merge) Fix assertion for proper Huffman merge pattern: + 0 == 1 modulo 1. See Knuth 5.4.9 (vol. 3, 2nd ed., + pp. 361). Thanks to John Darrington for + reporting the bug. + +Wed Jan 5 22:42:26 WST 2005 John Darrington + + * case.h Fixed bug # 11307 + +Wed Jan 5 08:30:48 WST 2005 John Darrington + + * val-labs.c Fixed bug which caused a crash if VALUE LABELS had + a trailing slash. + +Mon Jan 3 17:44:37 2005 Ben Pfaff + + * pfm-read.c: (read_variables) Remove direct manipulation of + v->aux, which is no longer needed. Fixes PR 11483. + +Sat Jan 1 19:01:16 WST 2005 John Darrington + + * data-list.c Fixed a bug in parsing delimiters. + + * group.c vars-atr.c Fixed buglet in hash/compare functions for alpha + values. + + * percentiles.c Properly handled calculation of Tukey hinges where + the number of data is small. + + * oneway.q Used the generic value_to_string function for independent + variable instead of trying to do it ourselves. + + * box-whisker.c Fixed a buglet which caused a crash if the number of + data was zero + + +Fri Dec 31 16:47:45 WST 2004 John Darrington + + * examine.q box-whisker.c chart.h Implemented boxplots in EXAMINE + + * percentiles.c Fixed some bugs when calculating percentiles when + there's a small number of cases. + +Wed Dec 29 08:18:08 WST 2004 John Darrington + + * percentiles.[ch] Added. Calculates percentiles and Tukey hinges + + * examine.q factor_stats.[ch] Added calculation of percentiles + +Fri Dec 24 15:09:11 WST 2004 John Darrington + + * t-test.q Fixed bug #11227 Made t-test work when the independent + variable is alpha + +Sat Dec 11 11:43:45 WST 2004 John Darrington + + * factor_stats.c Fixed calculation of trimmed mean under various + special conditions. + +Sat Dec 4 17:14:45 WST 2004 John Darrington + + * histogram.c chart.[ch] factor_stats.c frequencies.q + + Added code to calculate sensible histogram ranges and limits. + +Thu Dec 2 13:37:43 WST 2004 John Darrington + + * chart.h Updated to reflect many API changes. + + * cartesian.c chart.c Moved the definitions of chart_write_{xy}scale from + cartesian.c and into chart.c + + * factorstats.[ch] Added the histogram calculations + + * casefile.c Removed an unused variable. + + * frequencies.q examine.q histogram.c Reworked the API for + histograms. + + * piechart.c Revised the API for piecharts. + + * var.h Moved the definitions of freq_tab and freq out of var.h + and into frequencies.q where they belong. + +Tue Nov 30 21:10:20 2004 Ben Pfaff + + * flip.c: (flip_file) Check for off_t separately from fseeko(), + using AC_TYPE_OFF_T. + +Tue Nov 30 08:47:41 2004 Ben Pfaff + + * flip.c: (flip_file) If fseeko() is not available, use long int + for off_t. Thanks to "Marshall DeBerry" for + reporting the problem. + +Mon Nov 29 12:20:59 WST 2004 John Darrington + + * examine.q factor_stats.[ch] Changed stderr to se_mean to avoid + conflict with stdio. + +Sun Nov 21 10:32:41 WST 2004 John Darrington + + * var-labs.c (var_to_string) Now returns null if the variable is null + + * value-labels.c (value_to_string) Made it return null if either the + value or the variable is null. + + * hash.c (hsh_clear) Fixed a buglet. + + * examine.q factor_stats.[ch] Largely rewrote, because I'd started + with the wrong model. + + * casefile.[ch] Added a function to return the casereader.case_idx + member + + * examine.q Implemented the extreme values results. + +John Darrington + + * settings.h set.c glob.[ch] frequencies.q q2c.c error.c lexer.[ch] + output.[ch] getline.c + + Plugged some memory leaks + +Mon Nov 15 23:47:40 2004 Ben Pfaff + + Adopt GSL random number generators, paving the way for providing + the complete suite of random number generators on expressions. + + * Makefile.am: Remove random.c, random.h. + + * random.c: Removed. + + * random.h: Removed. + + * algorithm.c: (algo_default_random) Use GSL functions. + + * casefile.c: (test_casefile) Use GSL RNG functions. + + * expr-evl.c: (expr_evaluate) Use GSL RNG functions for OP_NORMAL, + OP_UNIFORM. + + * sample.c: (cmd_sample) Use GSL RNG functions. + (sample_trns_proc) Ditto. + + * set.q: (static var set_seed) Removed. + (static var seed_flag) Removed. + (static var rng) New variable. + (aux_stc_custom_seed) No seed value anymore, don't print anything. + (stc_custom_seed) Use new seed functions. + (seed_is_set) Removed. + (get_rng) New function that composes the entire external + interface. + (set_rng) New function. + (random_seed) New function. + +Mon Nov 15 22:08:25 2004 Ben Pfaff + + * expr-evl.c: (expr_evaluate) Fix XDATE.JDAY formula. Thanks to + John Darrington for reporting this + bug. + +Tue Nov 16 13:19:18 WST 2004 John Darrington + + * permissions.c command.def Added the PERMISSIONS command + +Mon Nov 15 01:33:32 2004 Ben Pfaff + + * q2c.c: (dump_header) Don't try to emit #includes at very top of + output file because that will precede #include , which + is bad. + (main) Add needed headers to /* (header) */ code. + +Mon Nov 15 01:21:36 2004 Ben Pfaff + + Instead of making system or portable file readers responsible for + dropping and reordering variables, make them read full cases and + let the caller take care of any changes. + + * get.c: New "case map" structure to handle this. Use for GET, + IMPORT, MATCH FILES. Essentially rewrite the whole file. + + * pfm-read.c: (pfm_read_case) Read into provided case. Signature + changed appropriately. + + * sfm-read.c: (sfm_read_case) Ditto. + +Mon Nov 15 00:47:45 2004 Ben Pfaff + + Decided that case_serialize() and case_unserialize() were too + abstract. Also we need a couple more functions to avoid excessive + copying for data in/out fast paths. + + * case.c: (case_serial_size) Removed. + (case_serialize) Rename case_to_values() and make its argument + explicitly an array of union values. + (case_unserialize) Rename case_from_values() and make its argument + explicitly an array of union values. + (case_data_all) New function. + (case_data_all_rw) New function. + + * casefile.c: (struct casefile) Change buffer from array of + unsigned char to array of union value for better accuracy. + Redefine buffer_used and buffer_size in terms of values, not + bytes. Remove case_size because it is now redundant with + value_cnt. Fix up all references to these members. + +Mon Nov 15 00:45:46 2004 Ben Pfaff + + * barchart.c: (struct subcat) Make `label' member const to silence + GCC warning with -Wwrite-strings. + + * cartesian.c: (struct dataset) Ditto. + + * case.c: Don't re-define NDEBUG if already defined. + Add lots of comments. + + * str.c: Fix includes. + + * crosstabs.q: Fix includes. + + * examine.q: Fix includes. Fix GCC warning about unused + variables. + + * frequencies.q: (stat macro) Removed and replaced where used by + its expansion. + + * list.q: Fix includes. + + * oneway.q: Fix includes. + + * piechart.c: Fix includes. Only define M_PI if not already + defined. + + * sfm-read.c: (bswap) New function. + (bswap_int32) Write in terms of bswap. + (bswap_flt64) Ditto. + + * str.c: (ds_data) Add external definition here, needed because + str.h has only an `extern inline' version. + + * value-labels.c: Fix includes. + +Mon Nov 15 00:40:55 2004 Ben Pfaff + + Instead of providing a system or portable file writer with a raw + case in the format needed for output, provide it with a regular + case. The writer takes care of any needed translation. + + * aggregate.c: Adopt new scheme for AGGREGATE. + (struct agr_proc) sfm_agr_case member removed. + (write_case_to_sfm) Removed because the new interface is easier to + use. + + * get.c: Adopt new scheme for SAVE, XSAVE, EXPORT. + + * pfm-write.c: Implement new scheme. + + * sfm-write.c: Ditto. + +Mon Nov 15 00:32:24 2004 Ben Pfaff + + Instead of treating `struct file_handle' as a class to subclass + into data files, system files, and portable files, instead use it + as a helper that coordinates access. Now it is opaque, too. + + This means that most references to a struct file_handle are now + changed into references to one of struct dfm_reader, struct + dfm_writer, struct sfm_reader, struct sfm_writer, struct + pfm_reader, or struct pfm_writer, according to what's being read + or written. + + Most related changes are only worth summarizing briefly. + + * dictionary.c: (dict_clear) Destroy aux data in deleted + variables. + (dict_clear_aux) New function. + (dict_create_var) Initialize aux, aux_dtor. + (dict_delete_var) Destroy aux data in deleted variable. + + * file-handle.h: (struct fh_ext_class) Removed. + (struct file_handle) Removed. + (fh_init_files) Removed. + + * file-handle.q: Changed references to a handle's `private' member + to direct references. + (struct private_file_handle) Renamed file_handle. + Add next, open_cnt, type, open_mode, aux members. + (struct file_handle_list) Removed. + (extern var inline_file) Removed. + (static var file_handles) Changed from file_handle_list * to + file_handle *. + (create_file_handle) Initialize new members. + (fh_close_handle) Removed. + (mode_name) New function. + (fh_open) New function. + (fh_close) New function. + (fh_parse_file_handle) Renamed fh_parse(). + + * glob.c: (init_glob) Remove fh_init_files() call. + + * aggregate.c: use sfm_writer. + (create_sysfile) Removed because the new interface is simpler. + + * apply-dict.c: Use sfm_reader. + + * data-list.c: Use dfm_reader. + + * file-type.c: Use dfm_reader. + + * get.c: Use sfm_reader, sfm_writer, pfm_reader, pfm_writer. + + * inpt-pgm.c: Use dfm_reader. + + * print.c: Use dfm_writer. + + * sysfile-info: Use sfm_reader. + + * dfm-read.c: Adopt new file handle infrastructure. + + * dfm-write.c: Ditto. + + * pfm-read.c: Ditto. + + * pfm-write.c: Ditto. + + * sfm-read.c: Ditto. + + * sfm-write.c: Ditto. + +Mon Nov 15 00:31:44 2004 Ben Pfaff + + Break dictionary functions into separate header file. + + * dictionary.h: New file. + + * var.h: Moved dict_*() functions to dictionary.h. + +Mon Nov 15 00:30:33 2004 Ben Pfaff + + Get rid of procedure-specific union in struct variable, using + instead a void * pointer and a destructor function. + + Most related changes are only worth brief summaries. + + * crosstabs.q: Fix includes. Use new struct var_range in lieu of + old p.crs member in struct variable. + + * frequencies.q: Fix includes. Use new struct var_freqs in lieu + of old p.frq member in struct variable. + + * histogram.c: (draw_histogram) Takes new freq_tab arg because + it's no longer possible to grab this from var->p.frq. + + * piechart.c: (draw_piechart) Ditto. + + * group.c: (group_proc_get) New function. + + * levene.c: Use group_proc_get() in lieu of old p.grp_data member + in struct variable. + + * oneway.q: Ditto. + + * t-test.q: Ditto. + + * main.c: (execute_command) Clear aux data in default_dict after + each command. (It's debatable whether this should be done.) + + * matrix-data.c: Use new struct mxd_var in lieu of old p.mxd + member in struct variable. + + * means.q: Get rid of integer mode, which is not included in + recent SPSS and was the only code that wanted per-variable private + data. + + * var.h: (struct crosstab_proc) Removed. + (struct frequencies_proc) Removed. + (struct list_proc) Removed. + (struct get_proc) Removed. + (struct means_proc) Removed. + (struct matrix_data_proc) Removed. + (struct match_files_proc) Removed. + (lots of enums) Removed. + (struct variable) Removed members `p', `get'. Add member + `aux_dtor'. + + * vars-atr.c: (var_attach_aux) New function. + (var_detach_aux) New function. + (var_clear_aux) New function. + (var_dtor_free) New function. + (discard_variables) Use NULL instead of inline_file. + +Fri Nov 12 10:07:11 WST 2004 John Darrington + + * value-labs.c Fixed the implmentation of value_to_string, so + that it properly handles alpha values. + + * oneway.q Changed instances where labels were being probed manually, + to use the canonical {var,value}_to_string functions + +Thu Nov 11 21:01:31 WST 2004 John Darrington + + * examine.q cartesian.c chart.[ch] Added normal and detrended normal + plots. Changed the API of the cartesian plot to be a much lower level + thing. + +Sun Nov 7 17:25:04 WST 2004 John Darrington + + * examine.q Added some of the parametric calculations + + * factor_stats.[ch] Created + +Sat Nov 6 21:24:31 WST 2004 John Darrington + + * examine.q Changed the definition of factors to be a composite, and + dealt with the consequences. + +Sat Nov 6 20:40:38 WST 2004 John Darrington + + * examine.q Fixed problem where examine wasn't dealing properly with + splits + +Sat Nov 6 14:49:47 WST 2004 John Darrington + + * oneway.q Fixed problem where oneway wasn't dealing properly with + splits + +Thu Nov 4 11:09:01 WST 2004 John Darrington + + * q2c.c examine.q Fixed a bug (feature?) whereby arrays in the + command which had settings didn't get the appropriate code + generated. + + * val.h value-labels.[ch] var-labs.c Added v*to_string functions + to convert variables/values to strings. + + * examine.q Added framework for the EXAMINE command. + +Mon Nov 1 12:46:17 WST 2004 John Darrington + + * q2c.c frequencies.q set.q t-test.q Fixed the q2c parsing of DBL + subcommand types. Changed frequencies.q to use it rather then the + custom parser. Dealt with the consequences. Added a test for NTILES + subcommand of frequencies. + +Sat Oct 30 09:16:29 WST 2004 John Darrington + + * oneway.q Fixed up the behaviour when given missing values + + * levene.c oneway.q Fixed a buglet with the levene statistic and + incorporated the levene test into the oneway command. + + * group.h t-test.q Moved the CMP_EQ and CMP_LE symbols out of + global scope, since they're only relevant to T-TEST + +Fri Oct 29 17:39:03 WST 2004 John Darrington + + * group.c group.h group_proc.h levene.c oneway.q t-test.q + + Made the t-test more consistent + with the way it handles groups. That is, it now uses a hash instead + of an array of 2. Also, made the levene.c file independent of the + implementation of the t-test. So now levene should be fine for both + t-test and anova. + + * Added an oneway.q file for one way anova + +Wed Jun 2 22:08:02 2004 Ben Pfaff + + * descript.c: (cmd_descriptives) Remove harmless but bogus test in + STATISTICS parsing. + +Mon May 31 20:45:24 2004 Ben Pfaff + + Fix memory leaks. + + * data-list.c: (cmd_data_list) Free dls->delims on lossage. + (data_list_trns_free) Free dls->delims. + + * t-test.q: (tts_custom_pairs) Free vars. + (ssbox_one_sample_init) Fix tab_vline() argument. + (ssbox_independent_samples_init) Ditto. + (trbox_paired_init) Ditto. + (trbox_one_sample_init) Ditto. + +Mon May 31 17:19:27 2004 Ben Pfaff + + Generalize casefiles to the extent that we can use them for + sorting and other kinds of data transformations. Change cases to + be copy-on-write to improve memory efficiency in common cases. + Every access to a member of a `struct ccase' was changed to be a + call to a case_*() function, especially case_data(), case_num(), + case_str(), or case_data_rw(). Many instances of a local variable + named "case_num" were changed to "case_idx" as a consequence. + Many `struct ccase *' were changed to actual `struct ccase' + because of copying semantics of cases. In several places there + was a choice between updating debug code to work with the new ADTs + or just deleting it because it was useless; I chose to delete it. + + * Makefile.am: (pspp_SOURCES) Add case.c, case.h. + + * case.c: New file. + + * case.h: New file. + + * aggregate.c: (struct agr_proc) Change type of `sort' to + sort_criteria *. Add `break_vars', `break_var_cnt' members. + Rename `vars' to `agr_vars', all references updated. Change + `agr_case' to type `struct ccase'. + (cmd_aggregate) Deal with new members. Use case_create(), + sort_active_file_in_place(), sort_active_file_to_casefile(). + (agr_destroy) Deal with new members. + (aggregate_single_case) Ditto. + (dump_aggregate_info) Ditto. + (initialize_aggregate_info) Ditto. + (agr_to_active_file) Ditto. + (presorted_agr_to_sysfile) Ditto. + (sort_agr_to_sysfile) Removed. + + * alloc.c: (out_of_memory) Make non-static. + + * alloc.h: Prototype out_of_memory(). + + * casefile.c: Switched from a linked list in-memory representation + to a two-level array-style representation. The linked list was + appropriate when we could stick a header onto cases, but that's no + longer the case. Also, the two-level array will allow for random + in-memory access in case that's ever wanted. Also added the + concept of a `destructive casereader', one that destroys cases in + the underlying casefile as they are read out. + (macro CASES_PER_BLOCK) New macro. + (struct casefile) New members `value_cnt', `case_list_size', + `case_acct_size', `being_destroyed', `cases'. Removed `head', + `tail'. + (struct casereader) Removed `cur'. Added `destructive', `c'. + (global var casefiles) Made static. + (static var case_bytes) New var. + (casefile_create) Takes a value count, not a case size in bytes, + to conform to the case interface. All callers updated. Deal with + new and removed members. + (casefile_destroy) Deal with new and removed members. + (casefile_sleep) New function. + (casefile_get_case_size) Removed. + (casefile_get_value_cnt) New function. + (casefile_append) Rewritten to deal with new and removed members. + (casefile_append_xfer) New function. + (write_case_to_disk) Use case_serialize(). + (call_posix_fadvise) Removed because posix_fadvise64 segfaults. + Couldn't figure out why. + (casefile_to_disk) Don't call call_posix_fadvise. Rewritten to + deal with new and removed members. + (merge) Removed. + (merge_sort) Removed. + (casefile_sort) Removed. + (casefile_get_reader) Deal with new and removed members. + (casefile_get_destructive_reader) New function. + (reader_open_file) Make code more readable. Create case for + reader. + (casereader_get_casefile) New function. + (casereader_read) Deal with new and removed members. Now returns + a copy of the case, so that the caller is responsible for + destroying the returned case. + (casereader_read_xfer) New function. + (casereader_destroy) Destroy reader's case. + (test_casefile) Second arg is now a value count, all callers + updated. Now tests destructive readers too. + (get_random_case) Deal with new case ADT. + (write_random_case) Ditto. + (read_and_verify_random_case) Ditto. + + * crosstabs.q: Remove debug code. + + * descript.q: (calc_descriptives) Deal with new case, casefile + ADTs. + + * dfm.c: (cmd_begin_data) There's no storage_source_class anymore. + + * do-if.c: Remove unneeded header inclusion. + + * expr-prs.c: Remove debug code. + + * exprP.h: Remove debug code. + + * flip.c: (flip_file) Use fseeko() if available. + + * formats.c: Remove debug code. + + * get.c: Remove debug code. + (struct mtf_file) Change `input' from `union value *' to `struct + ccase', all references updated. + + * levene.c: (levene) Deal with new case, casefile ADTs. + + * list.q: Remove debug code. + + * loop.c: Remove debug code. + + * matrix-data.c: Remove debug code. + + * means.q: Remove debug code. + + * mis-val.c: Remove debug code. + + * pfm-read.c: Remove debug code. + (pfm_read_code) Change second arg from `union value *' to `struct + ccase *', all references updated. + + * recode.c: (string_to_long) Make first arg const. + (convert_to_double) Ditto. + + * repeat.c: Remove debug code. + + * sample.c: Remove debug code. + + * sfm-read.c: Remove debug code. + (sfm_read_case) Change second arg from `union value *' to `struct + ccase *'. + + * sort.c: Redone in terms of casefiles. + (enum sort_direction) Moved here from sort.h. + (struct sort_criterion) New structure. + (struct sort_criteria) New structure. + (cmd_sort_cases) Rewritten. + (prepare_to_sort_active_file) New function. + (sort_active_file_in_place) New function. + (sort_active_file_to_casefile) New function. + (parse_sort) Renamed sort_parse_criteria(), rewritten & interface + changed, all callers updated. + (destroy_sort_cases_pgm) Renamed sort_destroy_criteria(), + rewritten & interface changed, all callers updated. + (sort_cases) Renamed sort_execute(), rewritten & interface + changed, all callers updated. + (struct internal_sort) Removed. + (do_internal_sort) Rewritten, interface changed. + (destroy_internal_sort) Removed. + (compare_case_dblptrs) Use sort_criteria instead of sort_case_pgm. + (struct initial_run) Removed; an initial run is now just a + casefile. + (compare_initial_runs) Rewritten. + (struct external_sort) Changed almost completely. + (do_external_sort) Rewritten, interface changed. + (destroy_external_sort) Rewritten. + [HAVE_MKDTEMP] (make_temp_dir) Removed. + [!HAVE_MKDTEMP] (do_mkdir) Removed. + [!HAVE_MKDTEMP] (make_temp_dir) Removed. + (init_external_sort) Removed. + (simulate_error) Removed. + (rmdir_temp_dir) Removed. + (get_temp_file_name) Removed. + (open_temp_file) Removed. + (close_temp_file) Removed. + (remove_temp_file) Removed. + (write_temp_file) Removed. + (read_temp_file) Removed. + (struct record_run) Change `record' from `struct case_lit *' to + `struct ccase'. + (struct initial_run_state) Remove `idx_to_fv', `free_list', + `file_idx', `output_file'. Add `run', casefile'. Change + `last_output' from `struct case_list *' to `struct ccase'. + (write_initial_runs) Change interface, rewrite. + (sort_sink_write) Renamed process_case(), changed interfaced, + rewrote. + (destroy_initial_run_state) Rewritten. + (allocate_cases) Rewritten. + (compare_record) Interface changed, rewritten. + (start_run) Rewritten. + (end_run) Rewritten. + (output_record) Rewritten. + (grab_case) Removed. + (release_case) Removed. + (struct merge_case) Change `cases' from double pointer to single + pointer. + (merge) Deal with new case and casefile ADTs. + (struct run) Removed. + (merge_once) Rewritten, interface changed. + (fill_run_buffer) Removed. + (sort_sink_make_source) Removed. + (sort_sink_class) Removed. + (struct sort_source_aux) Removed. + (sort_source_read_helper) Removed. + (sort_source_read) Removed. + (read_sort_output) Removed. + (read_internal_sort_output) Removed. + (read_external_sort_output) Removed. + (sort_source_destroy) Removed. + (sort_source_class) Removed. + + * sort.h: (struct sort_cases_pgm) Removed. + (enum sort_direction) Moved to sort.c. + + * t-test.q: (calculate) Deal with new case, casefile ADTs. + + * tab.c: Remove debug code. + + * var-labs.c: Remove debug code. + + * var.h: (struct ccase) Removed. + (struct case_list) Removed. + + * vars-atr.c: (discard_variables) Use free_case_source(). + + * vars-prs.c: (parse_vs_variable) Make arg const. + (parse_dict_variable) Ditto. + (parse_variables) Make struct dictionary * arg const. + (parse_var_set_vars) Make struct var_set * arg const. + (struct var_set) Add const to some of the function pointers' args. + (var_set_get_cnt) Make arg const. + (var_set_get_var) Make first arg const. + (var_set_lookup_var) Make first arg const. + (dict_var_set_get_cnt) Make arg const. + (dict_var_set_get_var) Make first arg const. + (dict_var_set_lookup_var) Make first arg const. + (var_set_create_from_dict) Make arg const. Add cast to aux + assignment. + (struct array_var_set) Add const to var member. + (array_var_set_get_cnt) Make arg const. + (array_var_set_get_var) Make first arg const. + (array_var_set_lookup_var) Make first arg const. + (var_set_create_from_array) Make first arg const. Insert cast. + + * vfm.c: (struct write_case_data) Change trns_case, sink_case + members from `struct ccase *' to `struct ccase'. + (static var lag_queue) Change from double to single pointer. + (procedure) Optimize trivial case. + (internal_procedure) Deal with changed case, case_source ADTs. + (create_trns_case) Changed interface, rewrote. + (open_active_interface) Initialize modified lag queue. + (write_case) Deal with changed case ADT. + (lag_case) Deal with modified lag queue. + (close_active_file) Destroy modified lag queue. + Deal with changed case_source, case_sink ADTs. + (destroy_storage_stream_info) Make null arg into no-op. + (storage_sink_make_source) Set aux in created source. + (storage_source_read) Deal with changed case, casefile ADTs. + (storage_source_create) New function. + (lagged_case) Rewrite. + (free_case_source) New function. + (free_case_sink) Rewrite. + (struct split_aux_data) Changed prev_case from `struct ccase *' to + `struct ccase'. + (procedure_with_splits) Deal with changed prev_case. + (procedure_with_splits_callback) Ditto. + (multipass_split_aux_data) Changed prev_case from `struct ccase *' to + `struct ccase'. + (multipass_procedure_with_splits) Deal with changed prev_case. + (multipass_split_callback) Ditto. + + +Mon May 31 17:19:06 2004 Ben Pfaff + + The workspace idea didn't work out. + + * Makefile.am: (pspp_SOURCES) Remove workspace.c, workspace.h. + + * workspace.c: Removed. + + * workspace.h: Removed. + +Sun May 30 18:35:19 2004 Ben Pfaff + + Fully implement arbitrary delimiters on DATA LIST, extending the + half implementation that was already there. + + * data-list.c: (struct data_list_pgm) Remove `delim', add + `delims', `delim_cnt'. + (cmd_data_list) Initialize new members. Parse delimiters and + clean up code a bit. + (cut_field) Extract fields with arbitrary delimiters. Also, fix + handling of leading commas. + (read_from_data_list_fixed) Expand tabs. Adapt to new DFM + interfaces. + (read_from_data_list_free) Adapt to new DFM interfaces. + (read_from_data_list_list) Ditto. + (repeating_data_trns_proc) Ditto. + + * dfm.c: Split up reader and writer into separate code, because + they do different things. Use struct string instead of explicit + allocation code, for clarity. + (enum dfm_reader_flags) New enum. + (struct dfm_fhuser_ext) Removed. + (struct dfm_reader_ext) New. + (get_reader) New function, used by just about all the reader + functions. + (dfm_close) Removed. + (close_reader) New function. + (dfm_open_for_reading) Rewrite initialization of dfm_fhuser_ext. + (dfm_open_for_writing) Ditto. + (macro force_line_buffer_expansion) Removed. + (count_tabs) Removed. + (tabs_to_spaces) Removed. + (read_record) Deal with new dfm_reader_ext. Use struct string + functions. Don't convert tabs to spaces. + (dfm_eof) New function. + (dfm_get_record) Changed interface, rewrote. + (dfm_expand_tabs) New function. + (dfm_fwd_record) Renamed dfm_forward_record(), updated to new + dfm_reader_ext, rewritten. + (dfm_bkwd_record) Renamed dfm_reread_record(), updated to new + dfm_reader_ext, rewritten. + (dfm_set_record) Removed in favor of dfm_forward_columns(). + (dfm_forward_columns) New function. + (dfm_get_cur_col) Renamed dfm_column_start, updated to new + dfm_reader_ext, rewritten. + (static var dfm_r_class) Use close_reader for the destructor. + (struct dfm_writer_ext) New. + (dfm_put_record) Updated to new dfm_writer_ext, rewritten. Uses + bounce buffer now instead of local allocation. + (close_writer) New function. + (static var dfm_writer_ext) Use close_writer for destructor. + (cmd_begin_data) Adapt to new dfm_reader_ext. + + * file-handle.q: Add support for per-file tab width. + (struct private_file_handle) Add tab_width member. + (q2c specifications) Add tabwidth subcommand. + (cmd_file_handle) Put parsed tab width into private_file_handle. + (create_file_handle) Set default tab width. + (handle_get_tab_width) New function. + + * file-type.c: (file_type_source_read) Adapt to new DFM interface. + + * inpt-pgm.c: (reread_trns_proc) Ditto. + + * matrix-data.c: (context) Ditto. + (another_token) Ditto. + (mget_token) Ditto. + (force_eol) Ditto. + +Sun May 30 18:33:59 2004 Ben Pfaff + + * casefile.c: (casefile_destroy) Fix memory leak by freeing + cf->filename. + (casereader_destroy) Don't close file descriptor -1. + + * recode.c: (cmd_recode) Fix memory leak. + + * set.q: (q2c specifications) Fix typo in user message. + + * str.c: (st_bare_pad_len_copy) Change memcpy to memmove to avoid + undefined behavior for overlapping arguments. + +Sun May 30 18:31:48 2004 Ben Pfaff + + * casefile.c: valgrind doesn't implement posix_fadvise() yet, so + don't call it when we're running under valgrind. + (call_posix_fadvise) New function. + (casefile_to_disk) Use call_posix_fadvise(). + (reader_open_file) Ditto. + +Sun May 30 18:20:12 2004 Ben Pfaff + + Update our string ADTs, struct string and struct len_string. Get + rid of pool support, which was largely unused. Rename lots of + functions to have more obvious or consistent names. + + * ascii.c: Get rid of ascii_pool. It was only used for string + allocations. + (ascii_open_global) Don't create ascii_pool. + (ascii_close_driver) Don't destroy ascii_pool. + (ascii_postopen_driver) Don't use pool. + (ascii_close_driver) Destroy strings manually. + + * str.c: (ds_create) Remove pool argument, all references updated. + (ds_init) Ditto. + (ds_replace) Remove pool support, make more efficient when we + don't need to reallocate. + (ds_destroy) Remove pool support. + (ds_rpad) New function. + (ds_size) Renamed ds_capacity(), all references updated. + (ds_value) Renamed ds_c_str(), all references updated. + (ds_concat) Renamed ds_puts(), all references updated. + (ds_concat_buffer) Renamed ds_concat(), all references updated. + (ds_putchar) Renamed ds_putc(), all references updated. + (ds_getline) Renamed ds_gets(), all references updated. + (ls_create) Remove pool argument, all references updated. + (ls_create_buffer) Ditto. + (ls_destroy) Removed pool support. + (ls_value) Renamed ls_c_str(), all references updated. + + * str.h: (ls_length) [__GNUC__] Add inline version. + (ls_c_str) [__GNUC__] Add inline version. + (ls_end) [__GNUC__] Add inline version. + (struct string) Remove pool member. Rename `size' to `capacity', + all references updated. + + * tab.c: (text_format) Instead of using pool argument to + ls_create_buffer(), call pool_register() on allocated data. + +Mon Apr 26 22:40:07 2004 Ben Pfaff + + We're abusing the current ASCII driver by telling it to allocate a + 9999-line, 9999-character page in the tests. This causes some + systems to curl up and die because it allocates 20 MB of + contiguous RAM. This change alleviates at least part of the + problem. It is mostly a stop-gap until the new output system is + ready. + + * ascii.c: (struct line) New structure. + (struct ascii_driver_ext) Remove `page', `page_size', `line_len', + `line_len_size', `n_output' members. Add `lines', `lines_cap'. + (ascii_preopen_driver) Initialize new members, not old ones. + (ascii_close_driver) Destroy new members, not old ones. + (ascii_open_page) Allocate new members, not old ones. + (expand_line) Allocate room in line. + (draw_line) Use new members. + (ascii_line_horz) Ditto. + (ascii_line_vert) Ditto. + (ascii_line_intersection) Ditto. + (text_draw) Ditto. + (output_lines) Ditto. + (ascii_close_page) Ditto. + +Sun Apr 25 23:40:15 2004 Ben Pfaff + + * matrix.c: Dead code. Removed. + + * matrix.h: Dead code. Removed. + +Fri Apr 16 23:59:51 2004 Ben Pfaff + + Contrary to what I'd always understood, there is an efficient + algorithm for deletion from a hash table populated via linear + probing. This change implements it. + + * hash.c: (hsh_rehash) Probe in increasing order. + (hsh_probe) Ditto. + (locate_matching_entry) Ditto. + (hsh_delete) Use Knuth's Algorithm 6.4R for deletion. + +Tue Apr 13 19:24:15 2004 Ben Pfaff + + * moments.c (calc_moments): Adjust calculation of kurtosis to + avoid subtracting huge numbers from huge numbers, on Michael + Kiefte's advice. + +Sun Apr 11 14:22:12 2004 Ben Pfaff + + Rework moments routines for improved numerical stability based on + Michael Kiefte's advice. Any bugs or remaining numerical problems + are still mine though. + + There is now a struct moments1 for use with one-pass moments. It + uses a provisional means algorithm as an attempt to improve + accuracy of higher moments. The older struct moments now only + handles two-pass moments. + + * aggregate.c: Use moments1 instead moments. + + * descript.c: Revert previous change, which is no longer needed + due to the moments revision. + + * moments.c: (calc_moments) New function for calculating variance, + skewness, kurtosis. + (moments_pass_one) Only accumulate weights bigger than zero. + (moments_calculate) Allow calculating the mean on pass one, others + require pass two. Implement in terms of calc_moments(). + (struct moments1) New structure. + (init_moments1) New function. + (moments1_clear) Ditto. + (moments1_create) Ditto. + (moments1_add) Ditto. + (moments1_calculate) Ditto. + (moments1_destroy) Ditto. + (cmd_debug_moments) Deal with `struct moments' or `struct + moments1' as requested by user. + +Sun Apr 11 14:21:55 2004 Ben Pfaff + + * Makefile.am (pspp_SOURCES): Remove debug.c. + + * debug.c: Removed. It was empty anyway. + +Fri Apr 9 20:04:49 2004 Ben Pfaff + + * descript.c (calc_descriptives): Fix assert failure when only + MOMENT_MEAN is needed. + +2004-04-09 Michael Kiefte + + * descript.c: + + fixed problem with parsing in match_statistic() causing + "DESCRIPTIVE STAT=MEAN." to barf. + + "MEAN" is now default if "SORT" given without specification. + + Fixed infinite loop with "DESCRIPT GIBBERISH=ALL." Parsing is + generally less forgiving of syntax errors: better to have it do + nothing and type it in again then to not know what it actually did + instead. + + z-score transformation now checks score for user-missing values + and checks std_dev for SYSMIS. + +2004-04-06 Michael Kiefte + + * aggregate.c, crosstabs.q, descript.c, dictionary.c, frequencies.q, levene.c, t-test.q, var.h: + Changed dict_get_case_weight() to accept an additional int * flag + to complain about system-missing, user-missing, zero, or negative + weights and updated existing functions to pass int * to + dict_get_case_weight(). + +2004-04-05 jmd + + * main.c: Fixed configuration problems with gsl + + * t-test.q: Fixed some problems encountered when compiling under Cygwin + +2004-04-03 blp + + * lexer.c, ChangeLog: + Fix infinite loop on comment at end of file, add test. + +2004-04-03 jmd + + * settings.h, var.h, ChangeLog, Makefile.am, cmdline.c, command.c, command.h, error.h, filename.c, frequencies.q, lexer.h, main.c, q2c.c, set.q: + Fixed the calculation of percentiles and added --syntax and --algorithm options + +Sat Apr 3 11:43:37 2004 Ben Pfaff + + * lexer.c: (lex_skip_comment) Handle end-of-file correctly (I + hope). + +Sat Apr 3 15:00:18 WST 2004 John Darrington + + * frequencies.q: Fixed the calculation of percentiles + + * Makefile.am: Added the --ansi flag and dealt with the + consequences. Added some entries to PSPP_sources so that + make distcheck would pass + + * cmdline.c: Added the --syntax and --algorithm options + + * q2c.c: Added an implicit /ALGORITHM subcommand to everything. + +Fri Apr 2 11:25:22 WAST 2004 John Darrington + + * t-test.q, levene.c, levene.h Converted t-test (incl levene) to + use the new multipass_split_... mechanism. + +Wed Mar 31 22:36:22 2004 Ben Pfaff + + * frequencies.q: (calc_stats) Use moments data structure and + calc_seskew(), calc_sekurt() functions. + + * set.q main.c settings.h Added support for --syntax and --algorithm + options + +Tue Mar 30 22:04:19 2004 Ben Pfaff + + * vfm.c: Had to get last call to multipass_split_output() inside + open_active_file()/close_active_file() pairing, so introduce new + function. + (internal_procedure) Move procedure() code here, except for calls + to open_active_file() and close_active_file(). + (procedure) Wrap open_active_file() and close_active_file() around + internal_procedure(). + (multipass_procedure_with_splits) Wrap open_active_file() and + close_active_file() around internal_procedure(). + +Tue Mar 30 22:01:57 2004 Ben Pfaff + + * descript.c: (cmd_descriptives) Free `vars' to avoid memory leak. + +Mon Mar 29 16:26:40 2004 Ben Pfaff + + * debug.c: Removed. Moved cmd_debug_evaluate() into expr-evl.c. + + * expr-evl.c: (cmd_debug_evaluate) Moved here from debug.c. + +Mon Mar 29 16:03:08 2004 Ben Pfaff + + * algorithm.c: By default turn off some of the more expensive + assertions. + (expensive_assert) New macro which expands to assert if + EXTRA_CHECKS is defined, to nothing otherwise. + (unique) Use expensive_assert(). + (binary_search) Ditto. + (push_heap) Ditto. + (pop_heap) Ditto. + (make_heap) Ditto. + (sort_heap) Ditto. + + * command.c: (conflicting_3char_prefixes) Words that are the same + don't cause conflicts when they are abbreviated to the first three + letters. + + * expr-evl.c: (CONCAT_func) Fix memory leak by incrementing struct + nonterm_node's n earlier. + (generic_str_func) Ditto. + +Mon Mar 29 15:32:17 2004 Ben Pfaff + + Add support for multipass procedures. Rewrite DESCRIPTIVES to + test multipass support, take advantage of new moments + calculation, and to not be such crappy code. Get rid of q2c + processing for DESCRIPTIVES. + + * vfm.c: (struct multipass_split_aux_data) New structure. + (multipass_procedure_with_splits) New function. + (multipass_split_callback) New function. + (multipass_split_output) New function. + * descript.q: Removed. + + * descript.c: New file. + + * var.h: Removed descriptives enums. + (struct descriptives_proc) Removed. + (struct variable) Removed p.dsc. + + * Makefile.am: (q_sources_c) Remove descript.c. + (q_sources_q) Removed descript.q. + +Mon Mar 29 15:31:55 2004 Ben Pfaff + + New manager for keeping track of used workspace. + + * workspace.c: New file. + + * workspace.h: New file. + + * Makefile.am: (pspp_SOURCES) Add workspace.c, workspace.h. + + * sort.c: (do_internal_sort) Use workspace_malloc(). + (destroy_internal_sort) Use workspace_free(). + +Mon Mar 29 15:31:08 2004 Ben Pfaff + + New `struct casefile' for managing sets of cases. + + * casefile.c: New file. + + * casefile.h: New file. + + * command.def: Add DEBUG CASEFILE command. + + * Makefile.am: (pspp_SOURCES) Add casefile.c, casefile.h. + + * sort.c: (sort_cases) Move logic for sending storage file to disk + into do_external_sort(). + (struct internal_sort) Use an array of ccase pointers instead of a + case_list. + (do_internal_sort) Rewrite to handle casefiles. + (compare_case_list) Removed. + (compare_cases) New function. + (compare_case_dblptrs) New function. + (read_internal_sort_output) Deal with new struct internal_sort. + + * vfm.c: (static var workspace_overflow) Removed. + (struct storage_stream_info) Removed all the members. Added + struct casefile * member. + (storage_sink_open) Use casefile. + (open_storage_file) Removed. + (write_storage_file) Removed. + (storage_to_disk) Removed. + (destroy_storage_stream_info) Use casefile. + (storage_sink_write) Use casefile. + (storage_sink_make_source) Use casefile. + (storage_source_count) Use casefile. + (storage_source_read) Use casefile. + (storage_source_on_disk) Removed. + (storage_source_get_cases) Removed. + (storage_source_set_cases) Removed. + (storage_source_get_casefile) New function. + +Mon Mar 29 15:30:09 2004 Ben Pfaff + + New `struct moments' for calculating moments. + + * stats.c: Removed. + + * stats.h: Removed. + + * moments.c: New file. + + * moments.h: New file. + + * command.def: Add DEBUG MOMENTS command. + + * Makefile.am: (pspp_SOURCES) Add moments.c, moments.h. Remove + stats.c, stats.h. + + * aggregate.c: Modify AGGREGATE to use the new moments + calculation, even if not in such a great way. + (struct agr_var) Add `moments' member. + (parse_aggregate_functions) Set `moments' member to null. + (agr_destroy) Destroy `moments' member. + (accumulate_aggregate_info) Use `moments' for standard deviation. + (dump_aggregate_info) Ditto. + (initialize_aggregate_info) Create or clear `moments'. + + * misc.h: Add pow2(), pow3(), pow4() functions in place of sqr(), + cube(), pow4() that were in stats.h. All references updated. + + * crosstabs.q: stats.h had chi-square significance functions. Use + GSL instead. + (display_chisq) Use gsl_cdf_chisq_Q() instead of chisq_sig(). + + * expr-evl.c: (expr_evaluate) Use moments_of_values() for + OP_CFVAR, OP_MEAN, OP_SD, OP_VARIANCE. + +Fri Mar 26 14:21:23 2004 Ben Pfaff + + * dictionary.c: (dict_compact_values) Compacted values need to + start off from 0. + +Fri Mar 26 00:54:57 2004 Ben Pfaff + + * var-labs.c: (cmd_variable_labels) For compatibility, don't allow + `/' at start. Check return value of parse_variables() for error + return. + +Fri Mar 26 00:19:27 2004 Ben Pfaff + + Revamp expressions: make the code a little nicer, and fix bugs + found in testing. + + * expr-evl.c: (expr_evaluate) Make expression argument const. + Support OP_ADD, OP_SUB, OP_MUL, OP_DIV instead of OP_PLUS, OP_MUL. + OP_POW is missing for arg 2 <= 0. OP_LOG is natural log, not + base-10 log. Fix OP_ANY, OP_ANY_STRING, OP_RANGE, OP_RANGE_STRING + off-by-ones. Add OP_MAX_STRING, OP_MIN_STRING. Fix OP_TIME_HMS, + OP_DATE_WKYR boundary conditions. Add OP_CTIME_DAYS, + OP_CTIME_HOURS, OP_CTIME_MINUTES, OP_CTIME_DAYS, OP_CTIME_SECONDS. + Support OP_INDEX_2, OP_INDEX_3, OP_RINDEX_2, OP_RINDEX_3 instead + of OP_INDEX, OP_INDEX_OPT, OP_RINDEX, OP_RINDEX_OPT. Merge + OP_LPAD_OPT into OP_LPAD, OP_RPAD_OPT into OP_RPAD, OP_LTRIM_OPT + into OP_LTRIM, OP_RTRIM_OPT into OP_RTRIM, OP_NUMBER_OPT into + OP_NUMBER. Fix OP_RTRIM fragility. Support OP_SUBSTR_2, + OP_SUBSTR_3 instead of OP_SUBSTR, OP_SUBSTR_OPT. Remove OP_INV. + Simplify OP_SYSMIS. Remove OP_STR_MIS. + + * expr-opt.c: (optimize_expression) Rewrite. + (macro n0) Removed. + (macro n1) Removed. + (macro n2) Removed. + (macro s0) Removed. + (macro s0l) Removed. + (macro s1) Removed. + (macro s1l) Removed. + (macro s2) Removed. + (macro s2l) Removed. + (macro s) Removed. + (macro sl) Removed. + (eq_num_con) New function. + (optimize_tree) New function. + (macro rnc) Removed. + (macro frnc) Removed. + (str_search) Add const to string params. + (str_rsearch) Ditto. + (evaluate_tree_no_missing) Renamed from evaluate_tree. Add num[], + str[], str_len[] locals to substitute for most of removed macros. + Support OP_ADD, OP_SUB, OP_MUL, OP_DIV instead of OP_PLUS, OP_MUL. + Removed support for missing values because we're never called with + missing values. Use set_number() or set_number_errno() instead of + rnc or frnc. Removed any stuff that caused trouble in testing. + We can re-add it later if it really slows anything. Fix some + random problems. + (evaluate_tree_with_missing) Not yet supported. To be added later + if it's important. + (repl_num_con) Removed. + (collapse_node) New function. + (force_repl_num_con) Removed. + (set_number) New function. + (set_number_errno) New function. + (repl_str_con) Removed. + (set_string) New function. + (yrmoda) Tighten boundary conditions. Adopt 2030 cut-off for + 2-digit years. + (dump_node) No special case for OP_AND, OP_OR. + + * expr-prs.c: (expr_prs) Honor EXPR_NO_OPTIMIZE bit. Rewrite. + (expr_get_type) New function. + (type_check) Rewrite. + (type_coercion) New function. + (struct operator) New structure. + (match_operator New function. + (parse_binary_operators) New function. + (parse_inverting_unary_operator) New function. + (parse_or) Rewritten. + (parse_and) Rewritten. + (parse_not) Rewritten. + (parse_rel) Rewritten. + (parse_add) Rewritten. + (parse_mul) Rewritten. + (parse_neg) Rewritten. + (parse_exp) Rewritten. + (parse_sysvar) Add $TRUE, $FALSE system variables. + Get $LENGTH, $WIDTH from get_viewlength(), get_viewwidth(). + (parse_primary) Use allocate_var_node(), allocate_num_con(), + allocate_str_con(). + (struct function) Remove desc, change `func' prototype. + (unary_func) Remove special cases. + (MISSING_func) Reduce to unary_func() that just returns a boolean. + (SYSMIS_func) Handle SYSMIS((x)) like SYSMIS(x). + (VALUE_func) Use allocate_var_node(). + (nary_num_func) Allow MIN and MAX for strings. + Use allocate_var_node(). Properly clean up. + Fix return type. + (generic_str_func) Use local table instead of removed `desc' + member. Mostly rewrite. + (get_num_args) Revise error message. + (parse_function) Return EXPR_ERROR, not 0 on error. + (macro op) Removed. + (macro varies) Removed. + (ops[]) Use expr.def. + (free_node) Do nothing if node is null. + (allocate_num_con) New function. + (allocate_str_con) New function. + (allocate_var_node) New function. + (allocate_binary_nonterminal) New function. + (append_nonterminal_arg) Removed. + (static var func_tab[]) Revised. + (expr_debug_print_postfix) Make parameter const. + Use printf() instead of debug_printf(). + + * expr.def: New file. + + * expr.h: Change PXP_* to EXPR_*, all references updated. Also + use named enum instead of unnamed, all references updated. Add + EXPR_ANY, EXPR_NO_OPTIMIZE. + + * exprP.h: Remove EX_*. Add DEFINE_OPERATOR. Use expr.def + instead of defining OP_* directly. + (macro IS_TERMINAL) New macro. + (macro IS_NONTERMINAL) New macro. + (enum OP_NO_FLAGS) New. + +Fri Mar 26 00:18:01 2004 Ben Pfaff + + * error.c: (err_assert_fail) msg variable needs to be non-const. + +Fri Mar 26 00:17:24 2004 Ben Pfaff + + * debug.c: (cmd_debug_evaluate) Rewrite. + +Fri Mar 26 00:15:13 2004 Ben Pfaff + + Fix some CROSSTABS bit rot stupidity. + + * crosstabs.q: Reorder the CELLS subcommands for compatibility. + (internal_cmd_crosstabs) Initializes cells[] correctly. + (float_M_suffix) Rename format_cell_entry(), change prototype, + rewrite. + (display_crosstabulation) Fix cell formatting. + +Fri Mar 26 00:14:09 2004 Ben Pfaff + + Make lex_rest_of_line(), lex_entire_end() not discard lines. Have + to call lex_discard_line() to do that. + + * command.c: (run_command) Call lex_discard_line() after + lex_rest_of_line(). + + * lexer.c: (lex_entire_end) Change behavior. + (lex_rest_of_line) Change behavior. Return const char *. + (lex_discard_line) Don't clear getl_buf, don't emit message. + + * main.c: (handle_error) Emit message here. + + * repeat.c: (internal_cmd_do_repeat) Use lex_discard_line() + instead of lex_entire_line(). + + * str.c: (mm_find_reverse) Make length params size_t. Rewrite. + + * title.c: (get_title) Call lex_discard_line() after + lex_rest_of_line(). + (cmd_file_label) Ditto. + (cmd_document) Deal with const char * return value. + +Fri Mar 26 00:10:16 2004 Ben Pfaff + + Removed REMARK command. + + * command.c: (extract_prefix) Removed. + (output_line) Removed. + (cmd_remark) Removed. + + * command.def: Remove REMARK. + +Fri Mar 26 00:08:38 2004 Ben Pfaff + + Added abort() after lots of assert(0) invocations to avoid some + compiler warnings. We really need a NOT_REACHED macro. + +Tue Mar 23 08:00:42 WAST 2004 John Darrington + + * sort.c: Added missing call to temp_file_close. Changed error + messages to warnings. + + * set.q: Improved setting of set_view{length,width} to be more tolerant + of buggy OSes. + +Sun Mar 21 10:11:14 WST 2004 John Darrington + + * val-labs.c: Fixed a bug where PSPP would crash, if VALUE LABELS was + used with incorrect syntax. + + * error.c, error.h et al: Overridden definition of assert for a + custom one. + + * test-q.c: Fixed a buglet where it would crash if no /VARIABLES + subcommand was given when it ought to have been. + Sat Mar 20 22:19:08 2004 Ben Pfaff * tab.c: (tab_vline) Fix assertions to respect row_ofs and