From: Ben Pfaff Date: Tue, 1 Jun 2004 04:57:11 +0000 (+0000) Subject: Fix memory leaks. X-Git-Tag: v0.4.0~260 X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?p=pspp-builds.git;a=commitdiff_plain;h=06f9ee45954e5e71fa7f6262dbf37defa1dbf996 Fix memory leaks. Generalize casefiles to the extent that we can use them for sorting and other kinds of data transformations. Change cases to be copy-on-write to improve memory efficiency in common cases. Every access to a member of a `struct ccase' was changed to be a call to a case_*() function, especially case_data(), case_num(), case_str(), or case_data_rw(). Many instances of a local variable named "case_num" were changed to "case_idx" as a consequence. Many `struct ccase *' were changed to actual `struct ccase' because of copying semantics of cases. In several places there was a choice between updating debug code to work with the new ADTs or just deleting it because it was useless; I chose to delete it. The workspace idea didn't work out. --- diff --git a/ChangeLog b/ChangeLog index 66714b68..a0e4b6b5 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,6 +1,12 @@ +Mon May 31 17:21:25 2004 Ben Pfaff + + * configure.ac: Check for large file support. Get rid of + posix_fadvise check--for some reason glibc 2.3.2 segfaults when I + call it and I just couldn't figure out what was going on. + Sun May 30 18:19:03 2004 Ben Pfaff - * config.ac: Check for valgrind/valgrind.h. + * configure.ac: Check for valgrind/valgrind.h. Mon Mar 29 15:22:48 2004 Ben Pfaff diff --git a/TODO b/TODO index fb3d8c7e..794ed43a 100644 --- a/TODO +++ b/TODO @@ -1,8 +1,10 @@ -Time-stamp: <2004-05-30 18:09:06 blp> +Time-stamp: <2004-05-31 13:14:29 blp> What Ben's working on now. -------------------------- +Workspace exhaustion heuristics. + Does SET work correctly? Update q2c input format description. @@ -18,6 +20,12 @@ other procedures). TODO ---- +Make valgrind --leak-check=yes --show-reachable=yes work. + +Add Boolean type. + +Add NOT_REACHED() macro. + Add compression to casefiles. Expressions need to be able to abbreviate function names. XDATE.QUARTER diff --git a/configure.ac b/configure.ac index 4859d654..738ca16a 100644 --- a/configure.ac +++ b/configure.ac @@ -19,6 +19,8 @@ AM_GNU_GETTEXT AM_GNU_GETTEXT_VERSION dnl Prevents autoreconf complaint. dnl Checks for libraries. +AC_SYS_LARGEFILE +AC_FUNC_FSEEKO AC_CHECK_LIB(m, sin) AC_CHECK_LIB(gmp, mpf_get_str,, AC_CHECK_LIB(gmp, __gmpf_get_str,, @@ -94,7 +96,7 @@ AC_REPLACE_FUNCS([memmove memset stpcpy strpbrk strerror strtol strtoul \ memchr getline getdelim strcasecmp strncasecmp memmem \ strtok_r]) AC_CHECK_FUNCS([gethostname strstr strtod __setfpucw isinf isnan finite \ - getpid feholdexcept mkdtemp posix_fadvise]) + getpid feholdexcept mkdtemp]) AC_PROG_LN_S diff --git a/src/ChangeLog b/src/ChangeLog index 13b90dcc..63f0439c 100644 --- a/src/ChangeLog +++ b/src/ChangeLog @@ -1,3 +1,289 @@ +Mon May 31 20:45:24 2004 Ben Pfaff + + Fix memory leaks. + + * data-list.c: (cmd_data_list) Free dls->delims on lossage. + (data_list_trns_free) Free dls->delims. + + * t-test.q: (tts_custom_pairs) Free vars. + (ssbox_one_sample_init) Fix tab_vline() argument. + (ssbox_independent_samples_init) Ditto. + (trbox_paired_init) Ditto. + (trbox_one_sample_init) Ditto. + +Mon May 31 17:19:27 2004 Ben Pfaff + + Generalize casefiles to the extent that we can use them for + sorting and other kinds of data transformations. Change cases to + be copy-on-write to improve memory efficiency in common cases. + Every access to a member of a `struct ccase' was changed to be a + call to a case_*() function, especially case_data(), case_num(), + case_str(), or case_data_rw(). Many instances of a local variable + named "case_num" were changed to "case_idx" as a consequence. + Many `struct ccase *' were changed to actual `struct ccase' + because of copying semantics of cases. In several places there + was a choice between updating debug code to work with the new ADTs + or just deleting it because it was useless; I chose to delete it. + + * Makefile.am: (pspp_SOURCES) Add case.c, case.h. + + * case.c: New file. + + * case.h: New file. + + * aggregate.c: (struct agr_proc) Change type of `sort' to + sort_criteria *. Add `break_vars', `break_var_cnt' members. + Rename `vars' to `agr_vars', all references updated. Change + `agr_case' to type `struct ccase'. + (cmd_aggregate) Deal with new members. Use case_create(), + sort_active_file_in_place(), sort_active_file_to_casefile(). + (agr_destroy) Deal with new members. + (aggregate_single_case) Ditto. + (dump_aggregate_info) Ditto. + (initialize_aggregate_info) Ditto. + (agr_to_active_file) Ditto. + (presorted_agr_to_sysfile) Ditto. + (sort_agr_to_sysfile) Removed. + + * alloc.c: (out_of_memory) Make non-static. + + * alloc.h: Prototype out_of_memory(). + + * casefile.c: Switched from a linked list in-memory representation + to a two-level array-style representation. The linked list was + appropriate when we could stick a header onto cases, but that's no + longer the case. Also, the two-level array will allow for random + in-memory access in case that's ever wanted. Also added the + concept of a `destructive casereader', one that destroys cases in + the underlying casefile as they are read out. + (macro CASES_PER_BLOCK) New macro. + (struct casefile) New members `value_cnt', `case_list_size', + `case_acct_size', `being_destroyed', `cases'. Removed `head', + `tail'. + (struct casereader) Removed `cur'. Added `destructive', `c'. + (global var casefiles) Made static. + (static var case_bytes) New var. + (casefile_create) Takes a value count, not a case size in bytes, + to conform to the case interface. All callers updated. Deal with + new and removed members. + (casefile_destroy) Deal with new and removed members. + (casefile_sleep) New function. + (casefile_get_case_size) Removed. + (casefile_get_value_cnt) New function. + (casefile_append) Rewritten to deal with new and removed members. + (casefile_append_xfer) New function. + (write_case_to_disk) Use case_serialize(). + (call_posix_fadvise) Removed because posix_fadvise64 segfaults. + Couldn't figure out why. + (casefile_to_disk) Don't call call_posix_fadvise. Rewritten to + deal with new and removed members. + (merge) Removed. + (merge_sort) Removed. + (casefile_sort) Removed. + (casefile_get_reader) Deal with new and removed members. + (casefile_get_destructive_reader) New function. + (reader_open_file) Make code more readable. Create case for + reader. + (casereader_get_casefile) New function. + (casereader_read) Deal with new and removed members. Now returns + a copy of the case, so that the caller is responsible for + destroying the returned case. + (casereader_read_xfer) New function. + (casereader_destroy) Destroy reader's case. + (test_casefile) Second arg is now a value count, all callers + updated. Now tests destructive readers too. + (get_random_case) Deal with new case ADT. + (write_random_case) Ditto. + (read_and_verify_random_case) Ditto. + + * crosstabs.q: Remove debug code. + + * descript.q: (calc_descriptives) Deal with new case, casefile + ADTs. + + * dfm.c: (cmd_begin_data) There's no storage_source_class anymore. + + * do-if.c: Remove unneeded header inclusion. + + * expr-prs.c: Remove debug code. + + * exprP.h: Remove debug code. + + * flip.c: (flip_file) Use fseeko() if available. + + * formats.c: Remove debug code. + + * get.c: Remove debug code. + (struct mtf_file) Change `input' from `union value *' to `struct + ccase', all references updated. + + * levene.c: (levene) Deal with new case, casefile ADTs. + + * list.q: Remove debug code. + + * loop.c: Remove debug code. + + * matrix-data.c: Remove debug code. + + * means.q: Remove debug code. + + * mis-val.c: Remove debug code. + + * pfm-read.c: Remove debug code. + (pfm_read_code) Change second arg from `union value *' to `struct + ccase *', all references updated. + + * recode.c: (string_to_long) Make first arg const. + (convert_to_double) Ditto. + + * repeat.c: Remove debug code. + + * sample.c: Remove debug code. + + * sfm-read.c: Remove debug code. + (sfm_read_case) Change second arg from `union value *' to `struct + ccase *'. + + * sort.c: Redone in terms of casefiles. + (enum sort_direction) Moved here from sort.h. + (struct sort_criterion) New structure. + (struct sort_criteria) New structure. + (cmd_sort_cases) Rewritten. + (prepare_to_sort_active_file) New function. + (sort_active_file_in_place) New function. + (sort_active_file_to_casefile) New function. + (parse_sort) Renamed sort_parse_criteria(), rewritten & interface + changed, all callers updated. + (destroy_sort_cases_pgm) Renamed sort_destroy_criteria(), + rewritten & interface changed, all callers updated. + (sort_cases) Renamed sort_execute(), rewritten & interface + changed, all callers updated. + (struct internal_sort) Removed. + (do_internal_sort) Rewritten, interface changed. + (destroy_internal_sort) Removed. + (compare_case_dblptrs) Use sort_criteria instead of sort_case_pgm. + (struct initial_run) Removed; an initial run is now just a + casefile. + (compare_initial_runs) Rewritten. + (struct external_sort) Changed almost completely. + (do_external_sort) Rewritten, interface changed. + (destroy_external_sort) Rewritten. + [HAVE_MKDTEMP] (make_temp_dir) Removed. + [!HAVE_MKDTEMP] (do_mkdir) Removed. + [!HAVE_MKDTEMP] (make_temp_dir) Removed. + (init_external_sort) Removed. + (simulate_error) Removed. + (rmdir_temp_dir) Removed. + (get_temp_file_name) Removed. + (open_temp_file) Removed. + (close_temp_file) Removed. + (remove_temp_file) Removed. + (write_temp_file) Removed. + (read_temp_file) Removed. + (struct record_run) Change `record' from `struct case_lit *' to + `struct ccase'. + (struct initial_run_state) Remove `idx_to_fv', `free_list', + `file_idx', `output_file'. Add `run', casefile'. Change + `last_output' from `struct case_list *' to `struct ccase'. + (write_initial_runs) Change interface, rewrite. + (sort_sink_write) Renamed process_case(), changed interfaced, + rewrote. + (destroy_initial_run_state) Rewritten. + (allocate_cases) Rewritten. + (compare_record) Interface changed, rewritten. + (start_run) Rewritten. + (end_run) Rewritten. + (output_record) Rewritten. + (grab_case) Removed. + (release_case) Removed. + (struct merge_case) Change `cases' from double pointer to single + pointer. + (merge) Deal with new case and casefile ADTs. + (struct run) Removed. + (merge_once) Rewritten, interface changed. + (fill_run_buffer) Removed. + (sort_sink_make_source) Removed. + (sort_sink_class) Removed. + (struct sort_source_aux) Removed. + (sort_source_read_helper) Removed. + (sort_source_read) Removed. + (read_sort_output) Removed. + (read_internal_sort_output) Removed. + (read_external_sort_output) Removed. + (sort_source_destroy) Removed. + (sort_source_class) Removed. + + * sort.h: (struct sort_cases_pgm) Removed. + (enum sort_direction) Moved to sort.c. + + * t-test.q: (calculate) Deal with new case, casefile ADTs. + + * tab.c: Remove debug code. + + * var-labs.c: Remove debug code. + + * var.h: (struct ccase) Removed. + (struct case_list) Removed. + + * vars-atr.c: (discard_variables) Use free_case_source(). + + * vars-prs.c: (parse_vs_variable) Make arg const. + (parse_dict_variable) Ditto. + (parse_variables) Make struct dictionary * arg const. + (parse_var_set_vars) Make struct var_set * arg const. + (struct var_set) Add const to some of the function pointers' args. + (var_set_get_cnt) Make arg const. + (var_set_get_var) Make first arg const. + (var_set_lookup_var) Make first arg const. + (dict_var_set_get_cnt) Make arg const. + (dict_var_set_get_var) Make first arg const. + (dict_var_set_lookup_var) Make first arg const. + (var_set_create_from_dict) Make arg const. Add cast to aux + assignment. + (struct array_var_set) Add const to var member. + (array_var_set_get_cnt) Make arg const. + (array_var_set_get_var) Make first arg const. + (array_var_set_lookup_var) Make first arg const. + (var_set_create_from_array) Make first arg const. Insert cast. + + * vfm.c: (struct write_case_data) Change trns_case, sink_case + members from `struct ccase *' to `struct ccase'. + (static var lag_queue) Change from double to single pointer. + (procedure) Optimize trivial case. + (internal_procedure) Deal with changed case, case_source ADTs. + (create_trns_case) Changed interface, rewrote. + (open_active_interface) Initialize modified lag queue. + (write_case) Deal with changed case ADT. + (lag_case) Deal with modified lag queue. + (close_active_file) Destroy modified lag queue. + Deal with changed case_source, case_sink ADTs. + (destroy_storage_stream_info) Make null arg into no-op. + (storage_sink_make_source) Set aux in created source. + (storage_source_read) Deal with changed case, casefile ADTs. + (storage_source_create) New function. + (lagged_case) Rewrite. + (free_case_source) New function. + (free_case_sink) Rewrite. + (struct split_aux_data) Changed prev_case from `struct ccase *' to + `struct ccase'. + (procedure_with_splits) Deal with changed prev_case. + (procedure_with_splits_callback) Ditto. + (multipass_split_aux_data) Changed prev_case from `struct ccase *' to + `struct ccase'. + (multipass_procedure_with_splits) Deal with changed prev_case. + (multipass_split_callback) Ditto. + + +Mon May 31 17:19:06 2004 Ben Pfaff + + The workspace idea didn't work out. + + * Makefile.am: (pspp_SOURCES) Remove workspace.c, workspace.h. + + * workspace.c: Removed. + + * workspace.h: Removed. + Sun May 30 18:35:19 2004 Ben Pfaff Fully implement arbitrary delimiters on DATA LIST, extending the diff --git a/src/Makefile.am b/src/Makefile.am index b96d96e8..1cd16e88 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -50,6 +50,7 @@ frequencies.q list.q means.q set.q t-test.q pspp_SOURCES = $(q_sources_c) aggregate.c algorithm.c algorithm.h \ alloc.c alloc.h apply-dict.c ascii.c autorecode.c bitvector.h \ +case.c case.h \ casefile.c casefile.h cmdline.c cmdline.h command.c command.def \ command.h compute.c copyleft.c copyleft.h count.c data-in.c data-in.h \ data-list.c data-list.h data-out.c date.c debug-print.h descript.c \ @@ -69,7 +70,7 @@ split-file.c str.c str.h sysfile-info.c tab.c tab.h temporary.c \ stat.h \ title.c t-test.h val.h val-labs.c value-labels.c value-labels.h \ var-labs.c var.h vars-atr.c vars-prs.c vector.c version.c version.h \ -vfm.c vfm.h vfmP.h weight.c workspace.c workspace.h +vfm.c vfm.h vfmP.h weight.c pspp_LDADD = ../lib/julcal/libjulcal.a \ ../lib/misc/libmisc.a \ diff --git a/src/aggregate.c b/src/aggregate.c index 4f2f1f45..ec601dbf 100644 --- a/src/aggregate.c +++ b/src/aggregate.c @@ -21,6 +21,8 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" +#include "casefile.h" #include "command.h" #include "error.h" #include "file-handle.h" @@ -117,13 +119,17 @@ struct agr_proc struct file_handle *out_file; /* Output file, or null if none. */ struct case_sink *sink; /* Sink, or null if none. */ + /* Break variables. */ + struct sort_criteria *sort; /* Sort criteria. */ + struct variable **break_vars; /* Break variables. */ + size_t break_var_cnt; /* Number of break variables. */ + union value *prev_break; /* Last values of break variables. */ + enum missing_treatment missing; /* How to treat missing values. */ - struct sort_cases_pgm *sort; /* Sort program. */ - struct agr_var *vars; /* First aggregate variable. */ + struct agr_var *agr_vars; /* First aggregate variable. */ struct dictionary *dict; /* Aggregate dictionary. */ int case_cnt; /* Counts aggregated cases. */ - union value *prev_break; /* Last values of break variables. */ - struct ccase *agr_case; /* Aggregate case for output. */ + struct ccase agr_case; /* Aggregate case for output. */ flt64 *sfm_agr_case; /* Aggregate case in SFM format. */ }; @@ -144,7 +150,6 @@ static int agr_to_active_file (struct ccase *, void *aux); /* Aggregating to a system file. */ static void write_case_to_sfm (struct agr_proc *agr); static int presorted_agr_to_sysfile (struct ccase *, void *aux); -static int sort_agr_to_sysfile (const struct ccase *, void *aux); /* Parsing. */ @@ -161,7 +166,8 @@ cmd_aggregate (void) agr.sink = NULL; agr.missing = ITEMWISE; agr.sort = NULL; - agr.vars = NULL; + agr.break_vars = NULL; + agr.agr_vars = NULL; agr.dict = NULL; agr.case_cnt = 0; agr.prev_break = NULL; @@ -210,6 +216,8 @@ cmd_aggregate (void) seen |= 4; else if (lex_match_id ("BREAK")) { + int i; + if (seen & 8) { msg (SE, _("%s subcommand given multiple times."),"BREAK"); @@ -218,22 +226,17 @@ cmd_aggregate (void) seen |= 8; lex_match ('='); - agr.sort = parse_sort (); + agr.sort = sort_parse_criteria (default_dict, + &agr.break_vars, &agr.break_var_cnt); if (agr.sort == NULL) goto lossage; - { - int i; - - for (i = 0; i < agr.sort->var_cnt; i++) - { - struct variable *v; - - v = dict_clone_var (agr.dict, agr.sort->vars[i], - agr.sort->vars[i]->name); - assert (v != NULL); - } - } + for (i = 0; i < agr.break_var_cnt; i++) + { + struct variable *v = dict_clone_var (agr.dict, agr.break_vars[i], + agr.break_vars[i]->name); + assert (v != NULL); + } } else break; } @@ -255,7 +258,7 @@ cmd_aggregate (void) /* Initialize. */ agr.case_cnt = 0; - agr.agr_case = xmalloc (dict_get_case_size (agr.dict)); + case_create (&agr.agr_case, dict_get_next_value_idx (agr.dict)); initialize_aggregate_info (&agr); /* Output to active file or external file? */ @@ -266,7 +269,7 @@ cmd_aggregate (void) cancel_temporary (); if (agr.sort != NULL && (seen & 4) == 0) - sort_cases (agr.sort, 0); + sort_active_file_in_place (agr.sort); agr.sink = create_case_sink (&storage_sink_class, agr.dict, NULL); if (agr.sink->class->open != NULL) @@ -275,8 +278,8 @@ cmd_aggregate (void) procedure (agr_to_active_file, &agr); if (agr.case_cnt > 0) { - dump_aggregate_info (&agr, agr.agr_case); - agr.sink->class->write (agr.sink, agr.agr_case); + dump_aggregate_info (&agr, &agr.agr_case); + agr.sink->class->write (agr.sink, &agr.agr_case); } dict_destroy (default_dict); default_dict = agr.dict; @@ -292,8 +295,22 @@ cmd_aggregate (void) if (agr.sort != NULL && (seen & 4) == 0) { /* Sorting is needed. */ - sort_cases (agr.sort, 1); - read_sort_output (agr.sort, sort_agr_to_sysfile, NULL); + struct casefile *dst; + struct casereader *reader; + struct ccase c; + + dst = sort_active_file_to_casefile (agr.sort); + if (dst == NULL) + goto lossage; + reader = casefile_get_destructive_reader (dst); + while (casereader_read_xfer (reader, &c)) + { + if (aggregate_single_case (&agr, &c, &agr.agr_case)) + write_case_to_sfm (&agr); + case_destroy (&c); + } + casereader_destroy (reader); + casefile_destroy (dst); } else { @@ -303,7 +320,7 @@ cmd_aggregate (void) if (agr.case_cnt > 0) { - dump_aggregate_info (&agr, agr.agr_case); + dump_aggregate_info (&agr, &agr.agr_case); write_case_to_sfm (&agr); } fh_close_handle (agr.out_file); @@ -506,10 +523,10 @@ parse_aggregate_functions (struct agr_proc *agr) struct agr_var *v = xmalloc (sizeof *v); /* Add variable to chain. */ - if (agr->vars != NULL) + if (agr->agr_vars != NULL) tail->next = v; else - agr->vars = v; + agr->agr_vars = v; tail = v; tail->next = NULL; v->moments = NULL; @@ -650,8 +667,9 @@ agr_destroy (struct agr_proc *agr) if (agr->dict != NULL) dict_destroy (agr->dict); if (agr->sort != NULL) - destroy_sort_cases_pgm (agr->sort); - for (iter = agr->vars; iter; iter = next) + sort_destroy_criteria (agr->sort); + free (agr->break_vars); + for (iter = agr->agr_vars; iter; iter = next) { next = iter->next; @@ -670,7 +688,7 @@ agr_destroy (struct agr_proc *agr) free (iter); } free (agr->prev_break); - free (agr->agr_case); + case_destroy (&agr->agr_case); } /* Execution. */ @@ -695,8 +713,8 @@ aggregate_single_case (struct agr_proc *agr, { int i; - for (i = 0; i < agr->sort->var_cnt; i++) - n_elem += agr->sort->vars[i]->nv; + for (i = 0; i < agr->break_var_cnt; i++) + n_elem += agr->break_vars[i]->nv; } agr->prev_break = xmalloc (sizeof *agr->prev_break * n_elem); @@ -706,15 +724,15 @@ aggregate_single_case (struct agr_proc *agr, union value *iter = agr->prev_break; int i; - for (i = 0; i < agr->sort->var_cnt; i++) + for (i = 0; i < agr->break_var_cnt; i++) { - struct variable *v = agr->sort->vars[i]; + struct variable *v = agr->break_vars[i]; if (v->type == NUMERIC) - (iter++)->f = input->data[v->fv].f; + (iter++)->f = case_num (input, v->fv); else { - memcpy (iter->s, input->data[v->fv].s, v->width); + memcpy (iter->s, case_str (input, v->fv), v->width); iter += v->nv; } } @@ -731,19 +749,19 @@ aggregate_single_case (struct agr_proc *agr, union value *iter = agr->prev_break; int i; - for (i = 0; i < agr->sort->var_cnt; i++) + for (i = 0; i < agr->break_var_cnt; i++) { - struct variable *v = agr->sort->vars[i]; + struct variable *v = agr->break_vars[i]; switch (v->type) { case NUMERIC: - if (input->data[v->fv].f != iter->f) + if (case_num (input, v->fv) != iter->f) goto not_equal; iter++; break; case ALPHA: - if (memcmp (input->data[v->fv].s, iter->s, v->width)) + if (memcmp (case_str (input, v->fv), iter->s, v->width)) goto not_equal; iter += v->nv; break; @@ -770,15 +788,15 @@ not_equal: union value *iter = agr->prev_break; int i; - for (i = 0; i < agr->sort->var_cnt; i++) + for (i = 0; i < agr->break_var_cnt; i++) { - struct variable *v = agr->sort->vars[i]; + struct variable *v = agr->break_vars[i]; if (v->type == NUMERIC) - (iter++)->f = input->data[v->fv].f; + (iter++)->f = case_num (input, v->fv); else { - memcpy (iter->s, input->data[v->fv].s, v->width); + memcpy (iter->s, case_str (input, v->fv), v->width); iter += v->nv; } } @@ -798,10 +816,10 @@ accumulate_aggregate_info (struct agr_proc *agr, weight = dict_get_case_weight (default_dict, input, &bad_warn); - for (iter = agr->vars; iter; iter = iter->next) + for (iter = agr->agr_vars; iter; iter = iter->next) if (iter->src) { - const union value *v = &input->data[iter->src->fv]; + const union value *v = case_data (input, iter->src->fv); if ((!iter->include_missing && is_missing (v, iter->src)) || (iter->include_missing && iter->src->type == NUMERIC @@ -954,23 +972,25 @@ static void dump_aggregate_info (struct agr_proc *agr, struct ccase *output) { { - int n_elem = 0; - - { - int i; + int value_idx = 0; + int i; - for (i = 0; i < agr->sort->var_cnt; i++) - n_elem += agr->sort->vars[i]->nv; - } - memcpy (output->data, agr->prev_break, sizeof (union value) * n_elem); + for (i = 0; i < agr->break_var_cnt; i++) + { + int nv = agr->break_vars[i]->nv; + memcpy (case_data_rw (output, value_idx), + &agr->prev_break[value_idx], + sizeof (union value) * nv); + value_idx += nv; + } } { struct agr_var *i; - for (i = agr->vars; i; i = i->next) + for (i = agr->agr_vars; i; i = i->next) { - union value *v = &output->data[i->dest->fv]; + union value *v = case_data_rw (output, i->dest->fv); if (agr->missing == COLUMNWISE && i->missing != 0 && (i->function & FUNC) != N && (i->function & FUNC) != NU @@ -1079,7 +1099,7 @@ initialize_aggregate_info (struct agr_proc *agr) { struct agr_var *iter; - for (iter = agr->vars; iter; iter = iter->next) + for (iter = agr->agr_vars; iter; iter = iter->next) { iter->missing = 0; switch (iter->function) @@ -1117,8 +1137,8 @@ agr_to_active_file (struct ccase *c, void *agr_) { struct agr_proc *agr = agr_; - if (aggregate_single_case (agr, c, agr->agr_case)) - agr->sink->class->write (agr->sink, agr->agr_case); + if (aggregate_single_case (agr, c, &agr->agr_case)) + agr->sink->class->write (agr->sink, &agr->agr_case); return 1; } @@ -1137,7 +1157,7 @@ write_case_to_sfm (struct agr_proc *agr) if (v->type == NUMERIC) { - double src = agr->agr_case->data[v->fv].f; + double src = case_num (&agr->agr_case, v->fv); if (src == SYSMIS) *p++ = -FLT64_MAX; else @@ -1145,7 +1165,7 @@ write_case_to_sfm (struct agr_proc *agr) } else { - memcpy (p, agr->agr_case->data[v->fv].s, v->width); + memcpy (p, case_str (&agr->agr_case, v->fv), v->width); memset (&((char *) p)[v->width], ' ', REM_RND_UP (v->width, sizeof (flt64))); p += DIV_RND_UP (v->width, sizeof (flt64)); @@ -1159,19 +1179,10 @@ write_case_to_sfm (struct agr_proc *agr) breakpoint. */ static int presorted_agr_to_sysfile (struct ccase *c, void *agr_) -{ - sort_agr_to_sysfile (c, agr_); - return 1; -} - -/* Aggregate the current case and output it if we passed a - breakpoint. */ -static int -sort_agr_to_sysfile (const struct ccase *c, void *agr_) { struct agr_proc *agr = agr_; - if (aggregate_single_case (agr, c, agr->agr_case)) + if (aggregate_single_case (agr, c, &agr->agr_case)) write_case_to_sfm (agr); return 1; diff --git a/src/alloc.c b/src/alloc.c index 0da8f744..1aaba305 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -23,8 +23,6 @@ #include #include #include "str.h" - -static void out_of_memory (void); /* Public functions. */ @@ -106,11 +104,9 @@ xstrdup (const char *s) memcpy (t, s, size); return t; } - -/* Private functions. */ /* Report an out-of-memory condition and abort execution. */ -static void +void out_of_memory (void) { fprintf (stderr, "virtual memory exhausted\n"); diff --git a/src/alloc.h b/src/alloc.h index b44f0f38..b1b1cc5e 100644 --- a/src/alloc.h +++ b/src/alloc.h @@ -27,6 +27,7 @@ void *xmalloc (size_t size); void *xcalloc (size_t size); void *xrealloc (void *ptr, size_t size); char *xstrdup (const char *s); +void out_of_memory (void) NO_RETURN; /* alloca() wrapper functions. */ #if defined (HAVE_ALLOCA) || defined (C_ALLOCA) diff --git a/src/autorecode.c b/src/autorecode.c index 9a9b3a07..5bf12bc0 100644 --- a/src/autorecode.c +++ b/src/autorecode.c @@ -21,6 +21,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "hash.h" @@ -262,7 +263,7 @@ recode (const struct autorecode_pgm *arc) static int autorecode_trns_proc (struct trns_header * trns, struct ccase * c, - int case_num UNUSED) + int case_idx UNUSED) { struct autorecode_trns *t = (struct autorecode_trns *) trns; int i; @@ -271,17 +272,15 @@ autorecode_trns_proc (struct trns_header * trns, struct ccase * c, { struct arc_spec *spec = &t->specs[i]; struct arc_item *item; + union value v; if (spec->src->type == NUMERIC) - item = hsh_force_find (spec->items, &c->data[spec->src->fv].f); + v.f = case_num (c, spec->src->fv); else - { - union value v; - v.c = c->data[spec->src->fv].s; - item = hsh_force_find (spec->items, &v); - } + v.c = (char *) case_str (c, spec->src->fv); + item = hsh_force_find (spec->items, &v); - c->data[spec->dest->fv].f = item->to; + case_data_rw (c, spec->dest->fv)->f = item->to; } return -1; } @@ -346,9 +345,9 @@ autorecode_proc_func (struct ccase *c, void *arc_) union value v, *vp, **vpp; if (arc->src_vars[i]->type == NUMERIC) - v.f = c->data[arc->src_vars[i]->fv].f; + v.f = case_num (c, arc->src_vars[i]->fv); else - v.c = c->data[arc->src_vars[i]->fv].s; + v.c = (char *) case_str (c, arc->src_vars[i]->fv); vpp = (union value **) hsh_probe (arc->src_values[i], &v); if (*vpp == NULL) diff --git a/src/case.c b/src/case.c new file mode 100644 index 00000000..6c427df5 --- /dev/null +++ b/src/case.c @@ -0,0 +1,293 @@ +/* PSPP - computes sample statistics. + Copyright (C) 2004 Free Software Foundation, Inc. + Written by Ben Pfaff . + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA + 02111-1307, USA. */ + +#include +#include "case.h" +#include +#include +#include "val.h" +#include "alloc.h" +#include "str.h" + +#ifdef GLOBAL_DEBUGGING +#undef NDEBUG +#else +#define NDEBUG +#endif +#include + +void +case_unshare (struct ccase *c) +{ + struct case_data *cd; + + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 1); + + cd = c->case_data; + cd->ref_cnt--; + case_create (c, c->case_data->value_cnt); + memcpy (c->case_data->values, cd->values, + sizeof *cd->values * cd->value_cnt); +} + +static inline size_t +case_size (size_t value_cnt) +{ + return (offsetof (struct case_data, values) + + value_cnt * sizeof (union value)); +} + +#ifdef GLOBAL_DEBUGGING +void +case_nullify (struct ccase *c) +{ + c->case_data = NULL; + c->this = c; +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +int +case_is_null (const struct ccase *c) +{ + return c->case_data == NULL; +} +#endif /* GLOBAL_DEBUGGING */ + +void +case_create (struct ccase *c, size_t value_cnt) +{ + if (!case_try_create (c, value_cnt)) + out_of_memory (); +} + +#ifdef GLOBAL_DEBUGGING +void +case_clone (struct ccase *clone, const struct ccase *orig) +{ + assert (orig != NULL); + assert (orig->this == orig); + assert (orig->case_data != NULL); + assert (orig->case_data->ref_cnt > 0); + assert (clone != NULL); + + if (clone != orig) + { + *clone = *orig; + clone->this = clone; + } + orig->case_data->ref_cnt++; +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +void +case_move (struct ccase *dst, struct ccase *src) +{ + assert (src != NULL); + assert (src->this == src); + assert (src->case_data != NULL); + assert (src->case_data->ref_cnt > 0); + assert (dst != NULL); + + *dst = *src; + dst->this = dst; + case_nullify (src); +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +void +case_destroy (struct ccase *c) +{ + struct case_data *cd; + + assert (c != NULL); + assert (c->this == c); + + cd = c->case_data; + if (cd != NULL && --cd->ref_cnt == 0) + { + memset (cd->values, 0xcc, sizeof *cd->values * cd->value_cnt); + cd->value_cnt = 0xdeadbeef; + free (cd); + } +} +#endif /* GLOBAL_DEBUGGING */ + +int +case_try_create (struct ccase *c, size_t value_cnt) +{ + c->case_data = malloc (case_size (value_cnt)); + if (c->case_data != NULL) + { +#ifdef GLOBAL_DEBUGGING + c->this = c; +#endif + c->case_data->value_cnt = value_cnt; + c->case_data->ref_cnt = 1; + return 1; + } + else + { +#ifdef GLOBAL_DEBUGGING + c->this = c; +#endif + return 0; + } +} + +int +case_try_clone (struct ccase *clone, const struct ccase *orig) +{ + case_clone (clone, orig); + return 1; +} + +#ifdef GLOBAL_DEBUGGING +void +case_copy (struct ccase *dst, size_t dst_idx, + const struct ccase *src, size_t src_idx, + size_t value_cnt) +{ + assert (dst != NULL); + assert (dst->this == dst); + assert (dst->case_data != NULL); + assert (dst->case_data->ref_cnt > 0); + assert (dst_idx + value_cnt <= dst->case_data->value_cnt); + + assert (src != NULL); + assert (src->this == src); + assert (src->case_data != NULL); + assert (src->case_data->ref_cnt > 0); + assert (src_idx + value_cnt <= dst->case_data->value_cnt); + + if (dst->case_data->ref_cnt > 1) + case_unshare (dst); + if (dst->case_data != src->case_data || dst_idx != src_idx) + memmove (dst->case_data->values + dst_idx, + src->case_data->values + src_idx, + sizeof *dst->case_data->values * value_cnt); +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +size_t +case_serial_size (size_t value_cnt) +{ + return value_cnt * sizeof (union value); +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +void +case_serialize (const struct ccase *c, void *output, + size_t output_size UNUSED) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (output_size == case_serial_size (c->case_data->value_cnt)); + assert (output != NULL || output_size == 0); + + memcpy (output, c->case_data->values, + case_serial_size (c->case_data->value_cnt)); +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +void +case_unserialize (struct ccase *c, const void *input, + size_t input_size UNUSED) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (input_size == case_serial_size (c->case_data->value_cnt)); + assert (input != NULL || input_size == 0); + + if (c->case_data->ref_cnt > 1) + case_unshare (c); + memcpy (c->case_data->values, input, + case_serial_size (c->case_data->value_cnt)); +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +const union value * +case_data (const struct ccase *c, size_t idx) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (idx < c->case_data->value_cnt); + + return &c->case_data->values[idx]; +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +double +case_num (const struct ccase *c, size_t idx) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (idx < c->case_data->value_cnt); + + return c->case_data->values[idx].f; +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +const char * +case_str (const struct ccase *c, size_t idx) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (idx < c->case_data->value_cnt); + + return c->case_data->values[idx].s; +} +#endif /* GLOBAL_DEBUGGING */ + +#ifdef GLOBAL_DEBUGGING +union value * +case_data_rw (struct ccase *c, size_t idx) +{ + assert (c != NULL); + assert (c->this == c); + assert (c->case_data != NULL); + assert (c->case_data->ref_cnt > 0); + assert (idx < c->case_data->value_cnt); + + if (c->case_data->ref_cnt > 1) + case_unshare (c); + return &c->case_data->values[idx]; +} +#endif /* GLOBAL_DEBUGGING */ diff --git a/src/case.h b/src/case.h new file mode 100644 index 00000000..9e8b6e4f --- /dev/null +++ b/src/case.h @@ -0,0 +1,180 @@ +/* PSPP - computes sample statistics. + Copyright (C) 2004 Free Software Foundation, Inc. + Written by Ben Pfaff . + + This program is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2 of the + License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA + 02111-1307, USA. */ + +#ifndef HEADER_CASE +#define HEADER_CASE + +#include +#include "val.h" + +/* Opaque structure that represents a case. Use accessor + functions instead of accessing any members directly. Use + case_move() or case_clone() instead of copying. */ +struct ccase + { + struct case_data *case_data; +#if GLOBAL_DEBUGGING + struct ccase *this; +#endif + }; + +/* Invisible to user code. */ +struct case_data + { + size_t value_cnt; + unsigned ref_cnt; + union value values[1]; + }; + +#ifdef GLOBAL_DEBUGGING +#define CASE_INLINE +#else +#define CASE_INLINE static +#endif + +CASE_INLINE void case_nullify (struct ccase *); +CASE_INLINE int case_is_null (const struct ccase *); + +void case_create (struct ccase *, size_t value_cnt); +CASE_INLINE void case_clone (struct ccase *, const struct ccase *); +CASE_INLINE void case_move (struct ccase *, struct ccase *); +CASE_INLINE void case_destroy (struct ccase *); + +int case_try_create (struct ccase *, size_t value_cnt); +int case_try_clone (struct ccase *, const struct ccase *); + +CASE_INLINE void case_copy (struct ccase *dst, size_t dst_idx, + const struct ccase *src, size_t src_idx, + size_t cnt); + +CASE_INLINE size_t case_serial_size (size_t value_cnt); +CASE_INLINE void case_serialize (const struct ccase *, void *, size_t); +CASE_INLINE void case_unserialize (struct ccase *, const void *, size_t); + +CASE_INLINE const union value *case_data (const struct ccase *, size_t idx); +CASE_INLINE double case_num (const struct ccase *, size_t idx); +CASE_INLINE const char *case_str (const struct ccase *, size_t idx); + +CASE_INLINE union value *case_data_rw (struct ccase *, size_t idx); + +void case_unshare (struct ccase *); + +#ifndef GLOBAL_DEBUGGING +#include +#include "str.h" + +static inline void +case_nullify (struct ccase *c) +{ + c->case_data = NULL; +} + +static inline int +case_is_null (const struct ccase *c) +{ + return c->case_data == NULL; +} + +static inline void +case_clone (struct ccase *clone, const struct ccase *orig) +{ + *clone = *orig; + orig->case_data->ref_cnt++; +} + +static inline void +case_move (struct ccase *dst, struct ccase *src) +{ + *dst = *src; + src->case_data = NULL; +} + +static inline void +case_destroy (struct ccase *c) +{ + struct case_data *cd = c->case_data; + if (cd != NULL && --cd->ref_cnt == 0) + free (cd); +} + +static inline void +case_copy (struct ccase *dst, size_t dst_idx, + const struct ccase *src, size_t src_idx, + size_t value_cnt) +{ + if (dst->case_data->ref_cnt > 1) + case_unshare (dst); + if (dst->case_data != src->case_data || dst_idx != src_idx) + memmove (dst->case_data->values + dst_idx, + src->case_data->values + src_idx, + sizeof *dst->case_data->values * value_cnt); +} + +static inline size_t +case_serial_size (size_t value_cnt) +{ + return value_cnt * sizeof (union value); +} + +static inline void +case_serialize (const struct ccase *c, void *output, + size_t output_size UNUSED) +{ + memcpy (output, c->case_data->values, + case_serial_size (c->case_data->value_cnt)); +} + +static inline void +case_unserialize (struct ccase *c, const void *input, + size_t input_size UNUSED) +{ + if (c->case_data->ref_cnt > 1) + case_unshare (c); + memcpy (c->case_data->values, input, + case_serial_size (c->case_data->value_cnt)); +} + +static inline const union value * +case_data (const struct ccase *c, size_t idx) +{ + return &c->case_data->values[idx]; +} + +static inline double +case_num (const struct ccase *c, size_t idx) +{ + return c->case_data->values[idx].f; +} + +static inline const char * +case_str (const struct ccase *c, size_t idx) +{ + return c->case_data->values[idx].s; +} + +static inline union value * +case_data_rw (struct ccase *c, size_t idx) +{ + if (c->case_data->ref_cnt > 1) + case_unshare (c); + return &c->case_data->values[idx]; +} +#endif /* !GLOBAL_DEBUGGING */ + +#endif /* case.h */ diff --git a/src/casefile.c b/src/casefile.c index d2c10383..2ff3a574 100644 --- a/src/casefile.c +++ b/src/casefile.c @@ -27,10 +27,11 @@ #include #include #include "alloc.h" +#include "case.h" #include "error.h" #include "misc.h" +#include "settings.h" #include "var.h" -#include "workspace.h" #ifdef HAVE_VALGRIND_VALGRIND_H #include @@ -45,20 +46,27 @@ file. Once any cases have been read, no more cases may be appended. The entire file is discarded at once. */ +/* In-memory cases are arranged in an array of arrays. The top + level is variable size and the size of each bottom level array + is fixed at the number of cases defined here. */ +#define CASES_PER_BLOCK 128 + /* A casefile. */ struct casefile { /* Basic data. */ struct casefile *next, *prev; /* Next, prev in global list. */ + size_t value_cnt; /* Case size in `union value's. */ size_t case_size; /* Case size in bytes. */ - size_t case_list_size; /* Bytes to allocate for case_lists. */ + size_t case_acct_size; /* Case size for accounting. */ unsigned long case_cnt; /* Number of cases stored. */ enum { MEMORY, DISK } storage; /* Where cases are stored. */ enum { WRITE, READ } mode; /* Is writing or reading allowed? */ struct casereader *readers; /* List of our readers. */ + int being_destroyed; /* Does a destructive reader exist? */ /* Memory storage. */ - struct case_list *head, *tail; /* Beginning, end of list of cases. */ + struct ccase **cases; /* Pointer to array of cases. */ /* Disk storage. */ int fd; /* File descriptor, -1 if none. */ @@ -68,23 +76,26 @@ struct casefile size_t buffer_size; /* Buffer size in bytes. */ }; -/* For reading out the casing in a casefile. */ +/* For reading out the cases in a casefile. */ struct casereader { struct casereader *next, *prev; /* Next, prev in casefile's list. */ struct casefile *cf; /* Our casefile. */ unsigned long case_idx; /* Case number of current case. */ - - /* Memory storage. */ - struct case_list *cur; /* Current case. */ + int destructive; /* Is this a destructive reader? */ /* Disk storage. */ int fd; /* File descriptor. */ unsigned char *buffer; /* I/O buffer. */ size_t buffer_pos; /* Byte offset of buffer position. */ + struct ccase c; /* Current case. */ }; -struct casefile *casefiles; +/* Doubly linked list of all casefiles. */ +static struct casefile *casefiles; + +/* Number of bytes of case allocated in in-memory casefiles. */ +static size_t case_bytes; static void register_atexit (void); static void exit_handler (void); @@ -99,9 +110,10 @@ static int safe_close (int fd); static int full_read (int fd, char *buffer, size_t size); static int full_write (int fd, const char *buffer, size_t size); -/* Creates and returns a casefile to store cases of CASE_SIZE bytes each. */ +/* Creates and returns a casefile to store cases of VALUE_CNT + `union value's each. */ struct casefile * -casefile_create (size_t case_size) +casefile_create (size_t value_cnt) { struct casefile *cf = xmalloc (sizeof *cf); cf->next = casefiles; @@ -109,19 +121,21 @@ casefile_create (size_t case_size) if (cf->next != NULL) cf->next->prev = cf; casefiles = cf; - cf->case_size = case_size; - cf->case_list_size = sizeof *cf->head + case_size - sizeof *cf->head->c.data; + cf->value_cnt = value_cnt; + cf->case_size = case_serial_size (value_cnt); + cf->case_acct_size = cf->case_size + 4 * sizeof (void *); cf->case_cnt = 0; cf->storage = MEMORY; cf->mode = WRITE; cf->readers = NULL; - cf->head = cf->tail = NULL; + cf->being_destroyed = 0; + cf->cases = NULL; cf->fd = -1; cf->filename = NULL; cf->buffer = NULL; - cf->buffer_size = ROUND_UP (case_size, IO_BUF_SIZE); - if (case_size > 0 && cf->buffer_size % case_size > 512) - cf->buffer_size = case_size; + cf->buffer_size = ROUND_UP (cf->case_size, IO_BUF_SIZE); + if (cf->case_size > 0 && cf->buffer_size % cf->case_size > 512) + cf->buffer_size = cf->case_size; cf->buffer_used = 0; register_atexit (); return cf; @@ -133,8 +147,6 @@ casefile_destroy (struct casefile *cf) { if (cf != NULL) { - struct case_list *iter, *next; - if (cf->next != NULL) cf->next->prev = cf->prev; if (cf->prev != NULL) @@ -145,10 +157,24 @@ casefile_destroy (struct casefile *cf) while (cf->readers != NULL) casereader_destroy (cf->readers); - for (iter = cf->head; iter != NULL; iter = next) + if (cf->cases != NULL) { - next = iter->next; - workspace_free (iter, cf->case_list_size); + size_t idx, block_cnt; + + case_bytes -= cf->case_cnt * cf->case_acct_size; + for (idx = 0; idx < cf->case_cnt; idx++) + { + size_t block_idx = idx / CASES_PER_BLOCK; + size_t case_idx = idx % CASES_PER_BLOCK; + struct ccase *c = &cf->cases[block_idx][case_idx]; + case_destroy (c); + } + + block_cnt = DIV_RND_UP (cf->case_cnt, CASES_PER_BLOCK); + for (idx = 0; idx < block_cnt; idx++) + free (cf->cases[idx]); + + free (cf->cases); } if (cf->fd != -1) @@ -175,13 +201,44 @@ casefile_in_core (const struct casefile *cf) return cf->storage == MEMORY; } -/* Returns the number of bytes in a case for CF. */ +/* Puts a casefile to "sleep", that is, minimizes the resources + needed for it by closing its file descriptor and freeing its + buffer. This is useful if we need so many casefiles that we + might not have enough memory and file descriptors to go + around. + + For simplicity, this implementation always converts the + casefile to reader mode. If this turns out to be a problem, + with a little extra work we could also support sleeping + writers. */ +void +casefile_sleep (const struct casefile *cf_) +{ + struct casefile *cf = (struct casefile *) cf_; + assert (cf != NULL); + + casefile_mode_reader (cf); + casefile_to_disk (cf); + + if (cf->fd != -1) + { + safe_close (cf->fd); + cf->fd = -1; + } + if (cf->buffer != NULL) + { + free (cf->buffer); + cf->buffer = NULL; + } +} + +/* Returns the number of `union value's in a case for CF. */ size_t -casefile_get_case_size (const struct casefile *cf) +casefile_get_value_cnt (const struct casefile *cf) { assert (cf != NULL); - return cf->case_size; + return cf->value_cnt; } /* Returns the number of cases in casefile CF. */ @@ -193,8 +250,8 @@ casefile_get_case_cnt (const struct casefile *cf) return cf->case_cnt; } -/* Appends case C to casefile CF. Not valid after any reader for CF has been - created. */ +/* Appends a copy of case C to casefile CF. Not valid after any + reader for CF has been created. */ void casefile_append (struct casefile *cf, const struct ccase *c) { @@ -202,23 +259,31 @@ casefile_append (struct casefile *cf, const struct ccase *c) assert (c != NULL); assert (cf->mode == WRITE); - cf->case_cnt++; - /* Try memory first. */ if (cf->storage == MEMORY) { - struct case_list *new_case; - - new_case = workspace_malloc (cf->case_list_size); - if (new_case != NULL) + if (case_bytes < get_max_workspace ()) { - memcpy (&new_case->c, c, cf->case_size); - new_case->next = NULL; - if (cf->head != NULL) - cf->tail->next = new_case; - else - cf->head = new_case; - cf->tail = new_case; + size_t block_idx = cf->case_cnt / CASES_PER_BLOCK; + size_t case_idx = cf->case_cnt % CASES_PER_BLOCK; + struct ccase new_case; + + case_bytes += cf->case_acct_size; + case_clone (&new_case, c); + if (case_idx == 0) + { + if ((block_idx & (block_idx - 1)) == 0) + { + size_t block_cap = block_idx == 0 ? 1 : block_idx * 2; + cf->cases = xrealloc (cf->cases, + sizeof *cf->cases * block_cap); + } + + cf->cases[block_idx] = xmalloc (sizeof **cf->cases + * CASES_PER_BLOCK); + } + + case_move (&cf->cases[block_idx][case_idx], &new_case); } else { @@ -229,6 +294,17 @@ casefile_append (struct casefile *cf, const struct ccase *c) } else write_case_to_disk (cf, c); + + cf->case_cnt++; +} + +/* Appends case C to casefile CF, which takes over ownership of + C. Not valid after any reader for CF has been created. */ +void +casefile_append_xfer (struct casefile *cf, struct ccase *c) +{ + casefile_append (cf, c); + case_destroy (c); } /* Writes case C to casefile CF's disk buffer, first flushing the buffer to @@ -236,13 +312,14 @@ casefile_append (struct casefile *cf, const struct ccase *c) static void write_case_to_disk (struct casefile *cf, const struct ccase *c) { - memcpy (cf->buffer + cf->buffer_used, c->data, cf->case_size); + case_serialize (c, cf->buffer + cf->buffer_used, cf->case_size); cf->buffer_used += cf->case_size; if (cf->buffer_used + cf->case_size > cf->buffer_size) flush_buffer (cf); - } +/* If any bytes in CF's output buffer are used, flush them to + disk. */ static void flush_buffer (struct casefile *cf) { @@ -285,35 +362,20 @@ make_temp_file (int *fd, char **filename) return 1; } -static void -call_posix_fadvise (int fd UNUSED, - off_t offset UNUSED, off_t len UNUSED, - int advice UNUSED) -{ -#ifdef HAVE_VALGRIND_VALGRIND_H - /* Valgrind doesn't know about posix_fadvise() as of this - writing. */ - if (RUNNING_ON_VALGRIND) - return; -#endif - -#ifdef HAVE_POSIX_FADVISE - posix_fadvise (fd, offset, len, advice); -#endif -} - /* If CF is currently stored in memory, writes it to disk. Readers, if any, retain their current positions. */ void -casefile_to_disk (struct casefile *cf) +casefile_to_disk (const struct casefile *cf_) { - struct case_list *iter, *next; + struct casefile *cf = (struct casefile *) cf_; struct casereader *reader; assert (cf != NULL); if (cf->storage == MEMORY) { + size_t idx, block_cnt; + assert (cf->filename == NULL); assert (cf->fd == -1); assert (cf->buffer_used == 0); @@ -321,114 +383,41 @@ casefile_to_disk (struct casefile *cf) cf->storage = DISK; if (!make_temp_file (&cf->fd, &cf->filename)) err_failure (); - call_posix_fadvise (cf->fd, 0, 0, POSIX_FADV_SEQUENTIAL); cf->buffer = xmalloc (cf->buffer_size); memset (cf->buffer, 0, cf->buffer_size); - for (iter = cf->head; iter != NULL; iter = next) + case_bytes -= cf->case_cnt * cf->case_acct_size; + for (idx = 0; idx < cf->case_cnt; idx++) { - next = iter->next; - write_case_to_disk (cf, &iter->c); - workspace_free (iter, cf->case_list_size); + size_t block_idx = idx / CASES_PER_BLOCK; + size_t case_idx = idx % CASES_PER_BLOCK; + struct ccase *c = &cf->cases[block_idx][case_idx]; + write_case_to_disk (cf, c); + case_destroy (c); } - flush_buffer (cf); - cf->head = cf->tail = NULL; + + block_cnt = DIV_RND_UP (cf->case_cnt, CASES_PER_BLOCK); + for (idx = 0; idx < block_cnt; idx++) + free (cf->cases[idx]); + + free (cf->cases); + cf->cases = NULL; + + if (cf->mode == READ) + flush_buffer (cf); for (reader = cf->readers; reader != NULL; reader = reader->next) reader_open_file (reader); } } -/* Merges lists A and B into a single list, which is returned. Cases are - compared according to comparison function COMPARE, which receives auxiliary - data AUX. */ -static struct case_list * -merge (struct case_list *a, struct case_list *b, - int (*compare) (const struct ccase *, - const struct ccase *, void *aux), - void *aux) -{ - struct case_list head; - struct case_list *tail = &head; - - while (a != NULL && b != NULL) - if (compare (&a->c, &b->c, aux) < 0) - { - tail->next = a; - tail = a; - a = a->next; - } - else - { - tail->next = b; - tail = b; - b = b->next; - } - - tail->next = a == NULL ? b : a; - - return head.next; -} - -/* Sorts the list beginning at FIRST, returning the new first case. Cases are - compared according to comparison function COMPARE, which receives auxiliary - data AUX. */ -static struct case_list * -merge_sort (struct case_list *first, - int (*compare) (const struct ccase *, - const struct ccase *, void *aux), - void *aux) -{ - /* FIXME: we should use a "natural" merge sort to take - advantage of the natural order of the data. */ - struct case_list *middle, *last, *tmp; - - /* A list of zero or one elements is already sorted. */ - if (first == NULL || first->next == NULL) - return first; - - middle = first; - last = first->next; - while (last != NULL && last->next != NULL) - { - middle = middle->next; - last = last->next->next; - } - tmp = middle; - middle = middle->next; - tmp->next = NULL; - return merge (merge_sort (first, compare, aux), - merge_sort (middle, compare, aux), - compare, aux); -} - -/* Tries to sort casefile CF according to comparison function - COMPARE, which is passes auxiliary data AUX. If successful, - returns nonzero. Currently only sorting of in-memory - casefiles is implemented. */ -int -casefile_sort (struct casefile *cf, - int (*compare) (const struct ccase *, - const struct ccase *, void *aux), - void *aux) +/* Changes CF to reader mode, ensuring that no more cases may be + added. Creating a casereader for CF has the same effect. */ +void +casefile_mode_reader (struct casefile *cf) { assert (cf != NULL); - assert (compare != NULL); - - cf->mode = WRITE; - - if (cf->case_cnt < 2) - return 1; - else if (cf->storage == DISK) - return 0; - else - { - cf->head = cf->tail = merge_sort (cf->head, compare, aux); - while (cf->tail->next != NULL) - cf->tail = cf->tail->next; - - return 1; - } + cf->mode = READ; } /* Creates and returns a casereader for CF. A casereader can be used to @@ -439,6 +428,9 @@ casefile_get_reader (const struct casefile *cf_) struct casefile *cf = (struct casefile *) cf_; struct casereader *reader; + assert (cf != NULL); + assert (!cf->being_destroyed); + /* Flush the buffer to disk if it's not empty. */ if (cf->mode == WRITE && cf->storage == DISK) flush_buffer (cf); @@ -453,19 +445,35 @@ casefile_get_reader (const struct casefile *cf_) reader->prev = NULL; cf->readers = reader; reader->case_idx = 0; - reader->cur = NULL; reader->fd = -1; reader->buffer = NULL; reader->buffer_pos = 0; + case_nullify (&reader->c); - if (reader->cf->storage == MEMORY) - reader->cur = cf->head; - else + if (reader->cf->storage == DISK) reader_open_file (reader); return reader; } +/* Creates and returns a destructive casereader for CF. Like a + normal casereader, a destructive casereader sequentially reads + the cases in a casefile. Unlike a normal casereader, a + destructive reader cannot operate concurrently with any other + reader. (This restriction could be relaxed in a few ways, but + it is so far unnecessary for other code.) */ +struct casereader * +casefile_get_destructive_reader (struct casefile *cf) +{ + struct casereader *reader; + + assert (cf->readers == NULL); + reader = casefile_get_reader (cf); + reader->destructive = 1; + cf->being_destroyed = 1; + return reader; +} + /* Opens a disk file for READER and seeks to the current position as indicated by case_idx. Normally the current position is the beginning of the file, but casefile_to_disk may cause the file to be opened at a different @@ -473,53 +481,55 @@ casefile_get_reader (const struct casefile *cf_) static void reader_open_file (struct casereader *reader) { + struct casefile *cf = reader->cf; size_t buffer_case_cnt; off_t file_ofs; - if (reader->case_idx >= reader->cf->case_cnt) + if (reader->case_idx >= cf->case_cnt) return; - if (reader->cf->fd != -1) + if (cf->fd != -1) { - reader->fd = reader->cf->fd; - reader->cf->fd = -1; + reader->fd = cf->fd; + cf->fd = -1; } else { - reader->fd = safe_open (reader->cf->filename, O_RDONLY); + reader->fd = safe_open (cf->filename, O_RDONLY); if (reader->fd < 0) msg (FE, _("%s: Opening temporary file: %s."), - reader->cf->filename, strerror (errno)); + cf->filename, strerror (errno)); } - if (reader->cf->buffer != NULL) + if (cf->buffer != NULL) { - reader->buffer = reader->cf->buffer; - reader->cf->buffer = NULL; + reader->buffer = cf->buffer; + cf->buffer = NULL; } else { - reader->buffer = xmalloc (reader->cf->buffer_size); - memset (reader->buffer, 0, reader->cf->buffer_size); + reader->buffer = xmalloc (cf->buffer_size); + memset (reader->buffer, 0, cf->buffer_size); } - if (reader->cf->case_size != 0) + if (cf->case_size != 0) { - buffer_case_cnt = reader->cf->buffer_size / reader->cf->case_size; + buffer_case_cnt = cf->buffer_size / cf->case_size; file_ofs = ((off_t) reader->case_idx - / buffer_case_cnt * reader->cf->buffer_size); + / buffer_case_cnt * cf->buffer_size); reader->buffer_pos = (reader->case_idx % buffer_case_cnt - * reader->cf->case_size); + * cf->case_size); } else file_ofs = 0; - call_posix_fadvise (reader->fd, file_ofs, 0, POSIX_FADV_SEQUENTIAL); if (lseek (reader->fd, file_ofs, SEEK_SET) != file_ofs) msg (FE, _("%s: Seeking temporary file: %s."), - reader->cf->filename, strerror (errno)); + cf->filename, strerror (errno)); - if (reader->cf->case_cnt > 0 && reader->cf->case_size > 0) + if (cf->case_cnt > 0 && cf->case_size > 0) fill_buffer (reader); + + case_create (&reader->c, cf->value_cnt); } /* Fills READER's buffer by reading a block from disk. */ @@ -535,23 +545,32 @@ fill_buffer (struct casereader *reader) reader->cf->filename); } +/* Returns the casefile that READER reads. */ +const struct casefile * +casereader_get_casefile (const struct casereader *reader) +{ + assert (reader != NULL); + + return reader->cf; +} + +/* Reads a copy of the next case from READER into C. + Caller is responsible for destroying C. */ int -casereader_read (struct casereader *reader, const struct ccase **c) +casereader_read (struct casereader *reader, struct ccase *c) { assert (reader != NULL); if (reader->case_idx >= reader->cf->case_cnt) - { - *c = NULL; - return 0; - } + return 0; - reader->case_idx++; if (reader->cf->storage == MEMORY) { - assert (reader->cur != NULL); - *c = &reader->cur->c; - reader->cur = reader->cur->next; + size_t block_idx = reader->case_idx / CASES_PER_BLOCK; + size_t case_idx = reader->case_idx % CASES_PER_BLOCK; + + case_clone (c, &reader->cf->cases[block_idx][case_idx]); + reader->case_idx++; return 1; } else @@ -562,12 +581,40 @@ casereader_read (struct casereader *reader, const struct ccase **c) reader->buffer_pos = 0; } - *c = (struct ccase *) (reader->buffer + reader->buffer_pos); + case_unserialize (&reader->c, reader->buffer + reader->buffer_pos, + reader->cf->case_size); reader->buffer_pos += reader->cf->case_size; + reader->case_idx++; + + case_clone (c, &reader->c); + return 1; + } +} + +/* Reads the next case from READER into C and transfers ownership + to the caller. Caller is responsible for destroying C. */ +int +casereader_read_xfer (struct casereader *reader, struct ccase *c) +{ + assert (reader != NULL); + + if (reader->destructive == 0 + || reader->case_idx >= reader->cf->case_cnt + || reader->cf->storage == DISK) + return casereader_read (reader, c); + else + { + size_t block_idx = reader->case_idx / CASES_PER_BLOCK; + size_t case_idx = reader->case_idx % CASES_PER_BLOCK; + struct ccase *read_case = &reader->cf->cases[block_idx][case_idx]; + + case_move (c, read_case); + reader->case_idx++; return 1; } } +/* Destroys READER. */ void casereader_destroy (struct casereader *reader) { @@ -593,9 +640,13 @@ casereader_destroy (struct casereader *reader) safe_close (reader->fd); } + case_destroy (&reader->c); + free (reader); } +/* Calls open(), passing FILENAME and FLAGS, repeating as necessary + to deal with interrupted calls. */ static int safe_open (const char *filename, int flags) { @@ -610,6 +661,8 @@ safe_open (const char *filename, int flags) return fd; } +/* Calls close(), passing FD, repeating as necessary to deal with + interrupted calls. */ static int safe_close (int fd) { int retval; @@ -623,6 +676,8 @@ static int safe_close (int fd) return retval; } +/* Calls read(), passing FD, BUFFER, and SIZE, repeating as + necessary to deal with interrupted calls. */ static int full_read (int fd, char *buffer, size_t size) { @@ -642,6 +697,8 @@ full_read (int fd, char *buffer, size_t size) return bytes_read; } +/* Calls write(), passing FD, BUFFER, and SIZE, repeating as + necessary to deal with interrupted calls. */ static int full_write (int fd, const char *buffer, size_t size) { @@ -659,6 +716,8 @@ full_write (int fd, const char *buffer, size_t size) return bytes_written; } +/* Registers our exit handler with atexit() if it has not already + been registered. */ static void register_atexit (void) { @@ -670,6 +729,8 @@ register_atexit (void) } } +/* atexit() handler that closes and deletes our temporary + files. */ static void exit_handler (void) { @@ -682,8 +743,9 @@ exit_handler (void) #include "random.h" #include "lexer.h" -static void test_casefile (int pattern, size_t case_size, size_t case_cnt); -static struct ccase *get_random_case (size_t case_size, size_t case_idx); +static void test_casefile (int pattern, size_t value_cnt, size_t case_cnt); +static void get_random_case (struct ccase *, size_t value_cnt, + size_t case_idx); static void write_random_case (struct casefile *cf, size_t case_idx); static void read_and_verify_random_case (struct casefile *cf, struct casereader *reader, @@ -723,7 +785,7 @@ cmd_debug_casefile (void) for (case_cnt = 0; case_cnt <= case_max; case_cnt = (case_cnt * 2) + 1) - test_casefile (pattern, *size * sizeof (union value), case_cnt); + test_casefile (pattern, *size, case_cnt); } } printf ("Casefile tests succeeded.\n"); @@ -731,18 +793,18 @@ cmd_debug_casefile (void) } static void -test_casefile (int pattern, size_t case_size, size_t case_cnt) +test_casefile (int pattern, size_t value_cnt, size_t case_cnt) { int zero = 0; struct casefile *cf; struct casereader *r1, *r2; - const struct ccase *c; + struct ccase c; struct rng *rng; size_t i, j; rng = rng_create (); rng_seed (rng, &zero, sizeof zero); - cf = casefile_create (case_size); + cf = casefile_create (value_cnt); for (i = 0; i < case_cnt; i++) write_random_case (cf, i); r1 = casefile_get_reader (cf); @@ -785,41 +847,70 @@ test_casefile (int pattern, size_t case_size, size_t case_cnt) casereader_destroy (r1); if (pattern != 2) casereader_destroy (r2); + if (pattern > 2) + { + r1 = casefile_get_destructive_reader (cf); + for (i = 0; i < case_cnt; i++) + { + struct ccase read_case, expected_case; + + get_random_case (&expected_case, value_cnt, i); + if (!casereader_read_xfer (r1, &read_case)) + fail_test ("Premature end of casefile."); + for (j = 0; j < value_cnt; j++) + { + double a = case_num (&read_case, j); + double b = case_num (&expected_case, j); + if (a != b) + fail_test ("Case %lu fails comparison.", (unsigned long) i); + } + case_destroy (&expected_case); + case_destroy (&read_case); + } + casereader_destroy (r1); + } casefile_destroy (cf); rng_destroy (rng); } -static struct ccase * -get_random_case (size_t case_size, size_t case_idx) +static void +get_random_case (struct ccase *c, size_t value_cnt, size_t case_idx) { - struct ccase *c = xmalloc (case_size); - memset (c, case_idx % 257, case_size); - return c; + int i; + case_create (c, value_cnt); + for (i = 0; i < value_cnt; i++) + case_data_rw (c, i)->f = case_idx % 257 + i; } static void write_random_case (struct casefile *cf, size_t case_idx) { - struct ccase *c = get_random_case (casefile_get_case_size (cf), case_idx); - casefile_append (cf, c); - free (c); + struct ccase c; + get_random_case (&c, casefile_get_value_cnt (cf), case_idx); + casefile_append_xfer (cf, &c); } static void read_and_verify_random_case (struct casefile *cf, struct casereader *reader, size_t case_idx) { - const struct ccase *read_case; - struct ccase *expected_case; - size_t case_size; - - case_size = casefile_get_case_size (cf); - expected_case = get_random_case (case_size, case_idx); + struct ccase read_case, expected_case; + size_t value_cnt; + size_t i; + + value_cnt = casefile_get_value_cnt (cf); + get_random_case (&expected_case, value_cnt, case_idx); if (!casereader_read (reader, &read_case)) fail_test ("Premature end of casefile."); - if (memcmp (read_case, expected_case, case_size)) - fail_test ("Case %lu fails comparison.", (unsigned long) case_idx); - free (expected_case); + for (i = 0; i < value_cnt; i++) + { + double a = case_num (&read_case, i); + double b = case_num (&expected_case, i); + if (a != b) + fail_test ("Case %lu fails comparison.", (unsigned long) case_idx); + } + case_destroy (&read_case); + case_destroy (&expected_case); } static void diff --git a/src/casefile.h b/src/casefile.h index 65ab4915..5674de0a 100644 --- a/src/casefile.h +++ b/src/casefile.h @@ -26,23 +26,26 @@ struct ccase; struct casefile; struct casereader; -struct casefile *casefile_create (size_t case_size); +struct casefile *casefile_create (size_t value_cnt); void casefile_destroy (struct casefile *); int casefile_in_core (const struct casefile *); -size_t casefile_get_case_size (const struct casefile *); +void casefile_to_disk (const struct casefile *); +void casefile_sleep (const struct casefile *); + +size_t casefile_get_value_cnt (const struct casefile *); unsigned long casefile_get_case_cnt (const struct casefile *); void casefile_append (struct casefile *, const struct ccase *); -void casefile_to_disk (struct casefile *); - -int casefile_sort (struct casefile *, - int (*compare) (const struct ccase *, - const struct ccase *, void *aux), - void *aux); +void casefile_append_xfer (struct casefile *, struct ccase *); +void casefile_mode_reader (struct casefile *); struct casereader *casefile_get_reader (const struct casefile *); -int casereader_read (struct casereader *, const struct ccase **); +struct casereader *casefile_get_destructive_reader (struct casefile *); + +const struct casefile *casereader_get_casefile (const struct casereader *); +int casereader_read (struct casereader *, struct ccase *); +int casereader_read_xfer (struct casereader *, struct ccase *); void casereader_destroy (struct casereader *); #endif /* casefile.h */ diff --git a/src/compute.c b/src/compute.c index 4cc9a187..f30d7b1f 100644 --- a/src/compute.c +++ b/src/compute.c @@ -21,6 +21,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "expr.h" @@ -112,7 +113,8 @@ compute_num (struct trns_header *compute_, struct ccase *c, if (compute->test == NULL || expr_evaluate (compute->test, c, case_num, NULL) == 1.0) { - expr_evaluate (compute->rvalue, c, case_num, &c->data[compute->fv]); + expr_evaluate (compute->rvalue, c, case_num, + case_data_rw (c, compute->fv)); } return -1; @@ -149,7 +151,7 @@ compute_num_vec (struct trns_header *compute_, struct ccase *c, return -1; } expr_evaluate (compute->rvalue, c, case_num, - &c->data[compute->vector->var[rindx - 1]->fv]); + case_data_rw (c, compute->vector->var[rindx - 1]->fv)); } return -1; @@ -169,8 +171,8 @@ compute_str (struct trns_header *compute_, struct ccase *c, union value v; expr_evaluate (compute->rvalue, c, case_num, &v); - st_bare_pad_len_copy (c->data[compute->fv].s, &v.c[1], compute->width, - v.c[0]); + st_bare_pad_len_copy (case_data_rw (c, compute->fv)->s, + &v.c[1], compute->width, v.c[0]); } return -1; @@ -216,7 +218,8 @@ compute_str_vec (struct trns_header *compute_, struct ccase *c, expr_evaluate (compute->rvalue, c, case_num, &v); vr = compute->vector->var[rindx - 1]; - st_bare_pad_len_copy (c->data[vr->fv].s, &v.c[1], vr->width, v.c[0]); + st_bare_pad_len_copy (case_data_rw (c, vr->fv)->s, + &v.c[1], vr->width, v.c[0]); } return -1; diff --git a/src/count.c b/src/count.c index dd88e01a..a6e70616 100644 --- a/src/count.c +++ b/src/count.c @@ -21,6 +21,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "lexer.h" @@ -366,16 +367,14 @@ static inline int count_numeric (struct counting * cnt, struct ccase * c) { int counter = 0; - - struct cnt_num *num; - - double cmp; int i; for (i = 0; i < cnt->n; i++) { + struct cnt_num *num; + /* Extract the variable value and eliminate missing values. */ - cmp = c->data[cnt->v[i]->fv].f; + double cmp = case_num (c, cnt->v[i]->fv); if (cmp == SYSMIS) { if (cnt->missing >= 1) @@ -433,27 +432,21 @@ static inline int count_string (struct counting * cnt, struct ccase * c) { int counter = 0; - - struct cnt_str *str; - - char *cmp; - int len; - int i; for (i = 0; i < cnt->n; i++) { - /* Extract the variable value, variable width. */ - cmp = c->data[cnt->v[i]->fv].s; - len = cnt->v[i]->width; + struct cnt_str *str; + /* Extract the variable value, variable width. */ for (str = cnt->crit.s;; str++) switch (str->type) { case CNT_ERROR: assert (0); case CNT_SINGLE: - if (memcmp (cmp, str->s, len)) + if (memcmp (case_str (c, cnt->v[i]->fv), str->s, + cnt->v[i]->width)) break; counter++; goto done; @@ -484,7 +477,7 @@ count_trns_proc (struct trns_header * trns, struct ccase * c, counter += count_numeric (cnt, c); else counter += count_string (cnt, c); - c->data[info->d->fv].f = counter; + case_data_rw (c, info->d->fv)->f = counter; } return -1; } diff --git a/src/crosstabs.q b/src/crosstabs.q index 4a58d10c..0a1e891f 100644 --- a/src/crosstabs.q +++ b/src/crosstabs.q @@ -37,6 +37,7 @@ #include #include "algorithm.h" #include "alloc.h" +#include "case.h" #include "hash.h" #include "pool.h" #include "command.h" @@ -156,11 +157,6 @@ static void submit (struct tab_table *); static void format_short (char *s, const struct fmt_spec *fp, const union value *v); -#if DEBUGGING -static void debug_print (void); -static void print_table_entries (struct table_entry **tab); -#endif - /* Parse and execute CROSSTABS, then clean up. */ int cmd_crosstabs (void) @@ -190,11 +186,6 @@ internal_cmd_crosstabs (void) if (!parse_crosstabs (&cmd)) return CMD_FAILURE; -#if DEBUGGING - /* Needs variables. */ - debug_print (); -#endif - mode = variables ? INTEGER : GENERAL; /* CELLS. */ @@ -453,58 +444,6 @@ crs_custom_variables (struct cmd_crosstabs *cmd UNUSED) variables = NULL; return 0; } - -#if DEBUGGING -static void -debug_print (void) -{ - printf ("CROSSTABS\n"); - - if (variables != NULL) - { - int i; - - printf ("\t/VARIABLES="); - for (i = 0; i < variables_cnt; i++) - { - struct variable *v = variables[i]; - - printf ("%s ", v->name); - if (i < variables_cnt - 1) - { - struct variable *nv = variables[i + 1]; - - if (v->p.crs.min == nv->p.crs.min - && v->p.crs.max == nv->p.crs.max) - continue; - } - printf ("(%d,%d) ", v->p.crs.min, v->p.crs.max - 1); - } - printf ("\n"); - } - - { - int i; - - printf ("\t/TABLES="); - for (i = 0; i < nxtab; i++) - { - struct crosstab *x = xtab[i]; - int j; - - if (i) - printf("\t\t"); - for (j = 0; j < x->nvar; j++) - { - if (j) - printf (" BY "); - printf ("%s", x->v[j]->name); - } - printf ("\n"); - } - } -} -#endif /* DEBUGGING */ /* Data file processing. */ @@ -608,9 +547,9 @@ calc_general (struct ccase *c, void *aux UNUSED) for (j = 0; j < x->nvar; j++) { if ((cmd.miss == CRS_TABLE - && is_missing (&c->data[x->vars[j]->fv], x->vars[j])) + && is_missing (case_data (c, x->vars[j]->fv), x->vars[j])) || (cmd.miss == CRS_INCLUDE - && is_system_missing (&c->data[x->vars[j]->fv], + && is_system_missing (case_data (c, x->vars[j]->fv), x->vars[j]))) { x->missing += weight; @@ -618,10 +557,10 @@ calc_general (struct ccase *c, void *aux UNUSED) } if (x->vars[j]->type == NUMERIC) - te->values[j].f = c->data[x->vars[j]->fv].f; + te->values[j].f = case_num (c, x->vars[j]->fv); else { - memcpy (te->values[j].s, c->data[x->vars[j]->fv].s, + memcpy (te->values[j].s, case_str (c, x->vars[j]->fv), x->vars[j]->width); /* Necessary in order to simplify comparisons. */ @@ -676,7 +615,7 @@ calc_integer (struct ccase *c, void *aux UNUSED) for (i = 0; i < x->nvar; i++) { struct variable *const v = x->vars[i]; - double value = c->data[v->fv].f; + double value = case_num (c, v->fv); /* Note that the first test also rules out SYSMIS. */ if ((value < v->p.crs.min || value >= v->p.crs.max) @@ -694,8 +633,8 @@ calc_integer (struct ccase *c, void *aux UNUSED) } { - const int row = c->data[x->vars[ROW_VAR]->fv].f - x->vars[ROW_VAR]->p.crs.min; - const int col = c->data[x->vars[COL_VAR]->fv].f - x->vars[COL_VAR]->p.crs.min; + const int row = case_num (c, x->vars[ROW_VAR]->fv) - x->vars[ROW_VAR]->p.crs.min; + const int col = case_num (c, x->vars[COL_VAR]->fv) - x->vars[COL_VAR]->p.crs.min; const int col_dim = x->vars[COL_VAR]->p.crs.count; sorted_tab[ofs]->u.data[col + row * col_dim] += weight; @@ -707,36 +646,6 @@ calc_integer (struct ccase *c, void *aux UNUSED) return 1; } -#if DEBUGGING -/* Print out all table entries in NULL-terminated TAB for use by a - debugger (a person, not a program). */ -static void -print_table_entries (struct table_entry **tab) -{ - printf ("raw crosstabulation data:\n"); - for (; *tab; tab++) - { - const struct crosstab *x = xtab[(*tab)->table]; - int i; - - printf ("(%g) table:%d ", (*tab)->u.freq, (*tab)->table); - for (i = 0; i < x->nvar; i++) - { - if (i) - printf (", "); - printf ("%s:", x->v[i]->name); - - if (x->v[i]->type == NUMERIC) - printf ("%g", (*tab)->v[i].f); - else - printf ("%.*s", x->v[i]->width, (*tab)->v[i].s); - } - printf ("\n"); - } - fflush (stdout); -} -#endif - /* Compare the table_entry's at A and B and return a strcmp()-type result. */ static int @@ -812,9 +721,6 @@ postcalc (void *aux UNUSED) { n_sorted_tab = hsh_count (gen_tab); sorted_tab = (struct table_entry **) hsh_sort (gen_tab); -#if DEBUGGING - print_table_entries (sorted_tab); -#endif } make_summary_table (); @@ -1398,35 +1304,6 @@ output_pivot_table (struct table_entry **pb, struct table_entry **pe, W = cum; } -#if DEBUGGING - /* Print the matrix. */ - { - int i, r, c; - - printf ("%s by %s for", x->v[0]->name, x->v[1]->name); - for (i = 2; i < nvar; i++) - printf (" %s=%g", x->v[i]->name, tb[0]->v[i].f); - printf ("\n"); - printf (" "); - for (c = 0; c < n_cols; c++) - printf ("%4g", cols[c].f); - printf ("\n"); - for (r = 0; r < n_rows; r++) - { - printf ("%4g:", rows[r].f); - for (c = 0; c < n_cols; c++) - printf ("%4g", mat[c + r * n_cols]); - printf ("%4g", row_tot[r]); - printf ("\n"); - } - printf (" "); - for (c = 0; c < n_cols; c++) - printf ("%4g", col_tot[c]); - printf ("%4g", W); - printf ("\n\n"); - } -#endif - /* Find the first variable that differs from the last subtable, then display the values of the dimensioning variables for each table that needs it. */ diff --git a/src/data-list.c b/src/data-list.c index c9203122..a3e6c67a 100644 --- a/src/data-list.c +++ b/src/data-list.c @@ -25,6 +25,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "command.h" #include "data-in.h" #include "debug-print.h" @@ -280,6 +281,7 @@ cmd_data_list (void) error: destroy_dls_var_spec (dls->first); + free (dls->delims); free (dls); return CMD_FAILURE; } @@ -1119,7 +1121,7 @@ read_from_data_list_fixed (const struct data_list_pgm *dls, data_in_finite_line (&di, ls_c_str (&line), ls_length (&line), var_spec->fc, var_spec->lc); - di.v = &c->data[var_spec->fv]; + di.v = case_data_rw (c, var_spec->fv); di.flags = 0; di.f1 = var_spec->fc; di.format = var_spec->input; @@ -1171,7 +1173,7 @@ read_from_data_list_free (const struct data_list_pgm *dls, di.s = ls_c_str (&field); di.e = ls_end (&field); - di.v = &c->data[var_spec->fv]; + di.v = case_data_rw (c, var_spec->fv); di.flags = 0; di.f1 = column; di.format = var_spec->input; @@ -1212,9 +1214,9 @@ read_from_data_list_list (const struct data_list_pgm *dls, { int width = get_format_var_width (&var_spec->input); if (width == 0) - c->data[var_spec->fv].f = SYSMIS; + case_data_rw (c, var_spec->fv)->f = SYSMIS; else - memset (c->data[var_spec->fv].s, ' ', width); + memset (case_data_rw (c, var_spec->fv)->s, ' ', width); } break; } @@ -1224,7 +1226,7 @@ read_from_data_list_list (const struct data_list_pgm *dls, di.s = ls_c_str (&field); di.e = ls_end (&field); - di.v = &c->data[var_spec->fv]; + di.v = case_data_rw (c, var_spec->fv); di.flags = 0; di.f1 = column; di.format = var_spec->input; @@ -1255,6 +1257,7 @@ static void data_list_trns_free (struct trns_header *pgm) { struct data_list_pgm *dls = (struct data_list_pgm *) pgm; + free (dls->delims); destroy_dls_var_spec (dls->first); fh_close_handle (dls->handle); free (pgm); @@ -1298,11 +1301,11 @@ data_list_trns_proc (struct trns_header *t, struct ccase *c, { if (retval == -2) { - c->data[dls->end->fv].f = 1.0; + case_data_rw (c, dls->end->fv)->f = 1.0; retval = -1; } else - c->data[dls->end->fv].f = 0.0; + case_data_rw (c, dls->end->fv)->f = 0.0; } dfm_pop (dls->handle); @@ -1784,7 +1787,7 @@ realize_value (struct rpd_num_or_var *n, struct ccase *c) assert (n->num == 0); if (n->var != NULL) { - double v = c->data[n->var->fv].f; + double v = case_num (c, n->var->fv); if (v == SYSMIS || v <= INT_MIN || v >= INT_MAX) return -1; @@ -1891,7 +1894,7 @@ rpd_parse_record (const struct rpd_parse_info *info) struct data_in di; data_in_finite_line (&di, info->line, info->len, fc, lc); - di.v = &info->c->data[var_spec->fv]; + di.v = case_data_rw (info->c, var_spec->fv); di.flags = 0; di.f1 = fc + 1; di.format = var_spec->input; diff --git a/src/descript.c b/src/descript.c index 93c31717..9acd21e6 100644 --- a/src/descript.c +++ b/src/descript.c @@ -26,6 +26,7 @@ #include #include "algorithm.h" #include "alloc.h" +#include "case.h" #include "casefile.h" #include "command.h" #include "lexer.h" @@ -569,7 +570,7 @@ dump_z_table (struct dsc_proc *dsc) */ static int descriptives_trns_proc (struct trns_header *trns, struct ccase * c, - int case_num UNUSED) + int case_idx UNUSED) { struct dsc_trns *t = (struct dsc_trns *) trns; struct dsc_z_score *z; @@ -581,7 +582,7 @@ descriptives_trns_proc (struct trns_header *trns, struct ccase * c, assert(t->vars); for (vars = t->vars; vars < t->vars + t->var_cnt; vars++) { - double score = c->data[(*vars)->fv].f; + double score = case_num (c, (*vars)->fv); if ( score == SYSMIS || (!t->include_user_missing && is_num_user_missing(score, *vars)) ) { @@ -593,14 +594,15 @@ descriptives_trns_proc (struct trns_header *trns, struct ccase * c, for (z = t->z_scores; z < t->z_scores + t->z_score_cnt; z++) { - double score = c->data[z->src_idx].f; + double input = case_num (c, z->src_idx); + double *output = &case_data_rw (c, z->dst_idx)->f; if (z->mean == SYSMIS || z->std_dev == SYSMIS - || all_sysmis || score == SYSMIS - || (!t->include_user_missing && is_num_user_missing(score, z->v))) - c->data[z->dst_idx].f = SYSMIS; + || all_sysmis || input == SYSMIS + || (!t->include_user_missing && is_num_user_missing(input, z->v))) + *output = SYSMIS; else - c->data[z->dst_idx].f = (score - z->mean) / z->std_dev; + *output = (input - z->mean) / z->std_dev; } return -1; } @@ -695,7 +697,7 @@ calc_descriptives (const struct casefile *cf, void *dsc_) { struct dsc_proc *dsc = dsc_; struct casereader *reader; - const struct ccase *c; + struct ccase c; int i; for (i = 0; i < dsc->var_cnt; i++) @@ -712,15 +714,16 @@ calc_descriptives (const struct casefile *cf, void *dsc_) dsc->valid = 0.; /* First pass to handle most of the work. */ - reader = casefile_get_reader (cf); - while (casereader_read (reader, &c)) + for (reader = casefile_get_reader (cf); + casereader_read (reader, &c); + case_destroy (&c)) { - double weight = dict_get_case_weight (default_dict, c, &dsc->bad_warn); + double weight = dict_get_case_weight (default_dict, &c, &dsc->bad_warn); if (weight <= 0.0) - continue; + continue; /* Check for missing values. */ - if (listwise_missing (dsc, c)) + if (listwise_missing (dsc, &c)) { dsc->missing_listwise += weight; if (dsc->missing_type == DSC_LISTWISE) @@ -731,7 +734,7 @@ calc_descriptives (const struct casefile *cf, void *dsc_) for (i = 0; i < dsc->var_cnt; i++) { struct dsc_var *dv = &dsc->vars[i]; - double x = c->data[dv->v->fv].f; + double x = case_num (&c, dv->v->fv); if (dsc->missing_type != DSC_LISTWISE && (x == SYSMIS @@ -756,23 +759,24 @@ calc_descriptives (const struct casefile *cf, void *dsc_) /* Second pass for higher-order moments. */ if (dsc->max_moment > MOMENT_MEAN) { - reader = casefile_get_reader (cf); - while (casereader_read (reader, &c)) + for (reader = casefile_get_reader (cf); + casereader_read (reader, &c); + case_destroy (&c)) { - double weight = dict_get_case_weight (default_dict, c, + double weight = dict_get_case_weight (default_dict, &c, &dsc->bad_warn); if (weight <= 0.0) continue; /* Check for missing values. */ - if (listwise_missing (dsc, c) + if (listwise_missing (dsc, &c) && dsc->missing_type == DSC_LISTWISE) continue; for (i = 0; i < dsc->var_cnt; i++) { struct dsc_var *dv = &dsc->vars[i]; - double x = c->data[dv->v->fv].f; + double x = case_num (&c, dv->v->fv); if (dsc->missing_type != DSC_LISTWISE && (x == SYSMIS @@ -837,7 +841,7 @@ listwise_missing (struct dsc_proc *dsc, const struct ccase *c) for (i = 0; i < dsc->var_cnt; i++) { struct dsc_var *dv = &dsc->vars[i]; - double x = c->data[dv->v->fv].f; + double x = case_num (c, dv->v->fv); if (x == SYSMIS || (!dsc->include_user_missing && is_num_user_missing (x, dv->v))) diff --git a/src/dfm.c b/src/dfm.c index 786c33e4..8216079b 100644 --- a/src/dfm.c +++ b/src/dfm.c @@ -593,8 +593,7 @@ cmd_begin_data (void) /* FIXME: figure out the *exact* conditions, not these really lenient conditions. */ if (vfm_source == NULL - || case_source_is_class (vfm_source, &storage_source_class) - || case_source_is_class (vfm_source, &sort_source_class)) + || case_source_is_class (vfm_source, &storage_source_class)) { msg (SE, _("This command is not valid here since the current " "input program does not access the inline file.")); diff --git a/src/dictionary.c b/src/dictionary.c index 8cceaf1a..20d64bc5 100644 --- a/src/dictionary.c +++ b/src/dictionary.c @@ -22,6 +22,7 @@ #include #include "algorithm.h" #include "alloc.h" +#include "case.h" #include "hash.h" #include "misc.h" #include "str.h" @@ -574,7 +575,7 @@ dict_get_case_weight (const struct dictionary *d, const struct ccase *c, return 1.0; else { - double w = c->data[d->weight->fv].f; + double w = case_num (c, d->weight->fv); if ( w < 0.0 || w == SYSMIS || is_num_user_missing(w, d->weight) ) w = 0.0; if ( w == 0.0 && *warn_on_invalid ) { diff --git a/src/do-if.c b/src/do-if.c index 6f9aa507..73fc057c 100644 --- a/src/do-if.c +++ b/src/do-if.c @@ -31,10 +31,6 @@ #include "debug-print.h" -#if DEBUGGING -#include -#endif - /* *INDENT-OFF* */ /* Description of DO IF transformations: diff --git a/src/expr-evl.c b/src/expr-evl.c index 3c743375..2ddefec7 100644 --- a/src/expr-evl.c +++ b/src/expr-evl.c @@ -37,6 +37,7 @@ #include #include #include +#include "case.h" #include "data-in.h" #include "error.h" #include "julcal/julcal.h" @@ -51,7 +52,7 @@ #include "vfmP.h" double -expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, +expr_evaluate (const struct expression *e, const struct ccase *c, int case_idx, union value *v) { unsigned char *op = e->op; @@ -1148,7 +1149,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, break; } assert (c != NULL); - sp->f = c->data[v->var[rindx - 1]->fv].f; + sp->f = case_num (c, v->var[rindx - 1]->fv); } break; case OP_VEC_ELEM_STR: @@ -1177,7 +1178,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, sp->c = pool_alloc (e->pool, v->width + 1); sp->c[0] = v->width; assert (c != NULL); - memcpy (&sp->c[1], c->data[v->fv].s, v->width); + memcpy (&sp->c[1], case_str (c, v->fv), v->width); } break; @@ -1195,7 +1196,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, case OP_NUM_VAR: sp++; assert (c != NULL); - sp->f = c->data[(*vars)->fv].f; + sp->f = case_num (c, (*vars)->fv); if (is_num_user_missing (sp->f, *vars)) sp->f = SYSMIS; vars++; @@ -1208,7 +1209,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, sp->c = pool_alloc (e->pool, width + 1); sp->c[0] = width; assert (c != NULL); - memcpy (&sp->c[1], &c->data[(*vars)->fv], width); + memcpy (&sp->c[1], case_str (c, (*vars)->fv), width); vars++; } break; @@ -1221,7 +1222,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, sp->f = SYSMIS; else { - sp->f = c->data[(*vars)->fv].f; + sp->f = case_num (c, (*vars)->fv); if (is_num_user_missing (sp->f, *vars)) sp->f = SYSMIS; } @@ -1240,7 +1241,7 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, if (c == NULL) memset (sp->c, ' ', width); else - memcpy (&sp->c[1], &c->data[(*vars)->fv], width); + memcpy (&sp->c[1], case_str (c, (*vars)->fv), width); vars++; } @@ -1248,16 +1249,16 @@ expr_evaluate (const struct expression *e, const struct ccase *c, int case_num, case OP_NUM_SYS: sp++; assert (c != NULL); - sp->f = c->data[*op++].f == SYSMIS; + sp->f = case_num (c, *op++) == SYSMIS; break; case OP_NUM_VAL: sp++; assert (c != NULL); - sp->f = c->data[*op++].f; + sp->f = case_num (c, *op++); break; case OP_CASENUM: sp++; - sp->f = case_num; + sp->f = case_idx; break; case OP_SENTINEL: diff --git a/src/expr-prs.c b/src/expr-prs.c index 05395bc4..da475783 100644 --- a/src/expr-prs.c +++ b/src/expr-prs.c @@ -60,10 +60,6 @@ static int type_check (union any_node **n, static algo_compare_func compare_functions; static void init_func_tab (void); - -#if DEBUGGING -static void debug_print_tree (union any_node *, int); -#endif /* Public functions. */ @@ -1519,84 +1515,6 @@ init_func_tab (void) /* Debug output. */ -#if DEBUGGING -static void -print_type (union any_node * n) -{ - const char *s; - size_t len; - - s = ops[n->type].name; - len = strlen (s); - if (ops[n->type].flags & OP_MIN_ARGS) - printf ("%s.%d\n", s, (int) n->nonterm.arg[n->nonterm.n]); - else if (ops[n->type].flags & OP_FMT_SPEC) - { - struct fmt_spec f; - - f.type = (int) n->nonterm.arg[n->nonterm.n + 0]; - f.w = (int) n->nonterm.arg[n->nonterm.n + 1]; - f.d = (int) n->nonterm.arg[n->nonterm.n + 2]; - printf ("%s(%s)\n", s, fmt_to_string (&f)); - } - else - printf ("%s\n", s); -} - -static void -debug_print_tree (union any_node * n, int level) -{ - int i; - for (i = 0; i < level; i++) - printf (" "); - if (n->type < OP_TERMINAL) - { - print_type (n); - for (i = 0; i < n->nonterm.n; i++) - debug_print_tree (n->nonterm.arg[i], level + 1); - } - else - { - switch (n->type) - { - case OP_TERMINAL: - printf (_("!!TERMINAL!!")); - break; - case OP_NUM_CON: - if (n->num_con.value == SYSMIS) - printf ("SYSMIS"); - else - printf ("%f", n->num_con.value); - break; - case OP_STR_CON: - printf ("\"%.*s\"", n->str_con.len, n->str_con.s); - break; - case OP_NUM_VAR: - case OP_STR_VAR: - printf ("%s", n->var.v->name); - break; - case OP_NUM_LAG: - case OP_STR_LAG: - printf ("LAG(%s,%d)", n->lag.v->name, n->lag.lag); - break; - case OP_NUM_SYS: - printf ("SYSMIS(%s)", n->var.v->name); - break; - case OP_NUM_VAL: - printf ("VALUE(%s)", n->var.v->name); - break; - case OP_SENTINEL: - printf (_("!!SENTINEL!!")); - break; - default: - printf (_("!!ERROR%d!!"), n->type); - assert (0); - } - printf ("\n"); - } -} -#endif /* DEBUGGING */ - void expr_debug_print_postfix (const struct expression *e) { diff --git a/src/expr.h b/src/expr.h index 12912565..2bf19831 100644 --- a/src/expr.h +++ b/src/expr.h @@ -39,7 +39,7 @@ union value; struct expression *expr_parse (enum expr_type); enum expr_type expr_get_type (const struct expression *); double expr_evaluate (const struct expression *, const struct ccase *, - int case_num, union value *); + int case_idx, union value *); void expr_free (struct expression *); void expr_debug_print_postfix (const struct expression *); diff --git a/src/exprP.h b/src/exprP.h index 40033a95..2cbbbabf 100644 --- a/src/exprP.h +++ b/src/exprP.h @@ -20,8 +20,6 @@ #if !exprP_h #define exprP_h 1 -#undef DEBUGGING -/*#define DEBUGGING 1*/ #include "debug-print.h" void debug_print_op (short int *); diff --git a/src/file-type.c b/src/file-type.c index 2b11a4ab..ec076146 100644 --- a/src/file-type.c +++ b/src/file-type.c @@ -21,6 +21,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "data-in.h" #include "dfm.h" @@ -643,7 +644,7 @@ file_type_source_read (struct case_source *source, { struct data_in di; - v.c = c->data[fty->record.v->fv].s; + v.c = case_data_rw (c, fty->record.v->fv)->s; data_in_finite_line (&di, ls_c_str (&line), ls_length (&line), fty->record.fc, fty->record.fc + fty->record.nc); @@ -676,7 +677,7 @@ file_type_source_read (struct case_source *source, di.format = format; data_in (&di); - memcpy (&c->data[fty->record.v->fv].f, &v.f, sizeof v.f); + case_data_rw (c, fty->record.v->fv)->f = v.f; for (iter = fty->recs_head; iter; iter = iter->next) { if (iter->flags & RCT_OTHER) diff --git a/src/flip.c b/src/flip.c index 1349ef1a..f9162522 100644 --- a/src/flip.c +++ b/src/flip.c @@ -25,6 +25,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "lexer.h" @@ -303,7 +304,7 @@ flip_sink_write (struct case_sink *sink, const struct ccase *c) v->next = NULL; if (flip->new_names->type == NUMERIC) { - double f = c->data[sink->idx_to_fv[flip->new_names->index]].f; + double f = case_num (c, sink->idx_to_fv[flip->new_names->index]); if (f == SYSMIS) strcpy (v->name, "VSYSMIS"); @@ -322,7 +323,7 @@ flip_sink_write (struct case_sink *sink, const struct ccase *c) else { int width = min (flip->new_names->width, 8); - memcpy (v->name, c->data[sink->idx_to_fv[flip->new_names->index]].s, + memcpy (v->name, case_str (c, sink->idx_to_fv[flip->new_names->index]), width); v->name[width] = 0; } @@ -336,10 +337,15 @@ flip_sink_write (struct case_sink *sink, const struct ccase *c) /* Write to external file. */ for (i = 0; i < flip->var_cnt; i++) - if (flip->var[i]->type == NUMERIC) - info->output_buf[i].f = c->data[sink->idx_to_fv[flip->var[i]->index]].f; - else - info->output_buf[i].f = SYSMIS; + { + double out; + + if (flip->var[i]->type == NUMERIC) + out = case_num (c, sink->idx_to_fv[flip->var[i]->index]); + else + out = SYSMIS; + info->output_buf[i].f = out; + } if (fwrite (info->output_buf, sizeof *info->output_buf, flip->var_cnt, flip->file) != (size_t) flip->var_cnt) @@ -407,9 +413,14 @@ flip_file (struct flip_pgm *flip) for (j = 0; j < read_cases; j++) output_buf[j] = input_buf[i + j * flip->var_cnt]; - if (fseek (output_file, - sizeof *input_buf * (case_idx + i * flip->case_cnt), - SEEK_SET) != 0) +#ifndef HAVE_FSEEKO +#define fseeko fseek +#endif + + if (fseeko (output_file, + sizeof *input_buf * (case_idx + + (off_t) i * flip->case_cnt), + SEEK_SET) != 0) msg (FE, _("Error seeking FLIP source file: %s."), strerror (errno)); @@ -466,11 +477,15 @@ flip_source_read (struct case_source *source, write_case_func *write_case, write_case_data wc_data) { struct flip_pgm *flip = source->aux; + union value *input_buf; int i; + input_buf = xmalloc (sizeof *input_buf * flip->case_cnt); for (i = 0; i < flip->var_cnt; i++) { - if (fread (c->data, sizeof *c->data, flip->case_cnt, + size_t j; + + if (fread (input_buf, sizeof *input_buf, flip->case_cnt, flip->file) != flip->case_cnt) { if (ferror (flip->file)) @@ -483,9 +498,12 @@ flip_source_read (struct case_source *source, break; } + for (j = 0; j < flip->case_cnt; j++) + case_data_rw (c, j)->f = input_buf[j].f; if (!write_case (wc_data)) break; } + free (input_buf); } /* Destroy internal data in SOURCE. */ diff --git a/src/formats.c b/src/formats.c index 36c019c0..8e8e45af 100644 --- a/src/formats.c +++ b/src/formats.c @@ -30,10 +30,6 @@ #include "debug-print.h" -#if DEBUGGING -static void debug_print (void); -#endif - enum { FORMATS_PRINT = 001, @@ -131,30 +127,9 @@ internal_cmd_formats (int which) free (v); v = NULL; } -#if DEBUGGING - debug_print (); -#endif return CMD_SUCCESS; fail: free (v); return CMD_PART_SUCCESS_MAYBE; } - -#if DEBUGGING -static void -debug_print (void) -{ - int i; - - printf (_("Formats:\n")); - printf (_(" Name Print Write\n")); - printf (" -------- ------------ ------------\n"); - for (i = 0; i < default_dict.nvar; i++) - { - struct variable *v = default_dict.var[i]; - printf (" %-8s %-12s", v->name, fmt_to_string (&v->print)); - printf (" %-12s\n", fmt_to_string (&v->write)); - } -} -#endif /* DEBUGGING */ diff --git a/src/frequencies.q b/src/frequencies.q index 8a415164..a6b82f6f 100644 --- a/src/frequencies.q +++ b/src/frequencies.q @@ -29,6 +29,7 @@ #include #include "alloc.h" #include "bitvector.h" +#include "case.h" #include "hash.h" #include "pool.h" #include "command.h" @@ -377,7 +378,7 @@ calc (struct ccase *c, void *aux UNUSED) for (i = 0; i < n_variables; i++) { struct variable *v = v_variables[i]; - union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); struct freq_tab *ft = &v->p.frq.tab; switch (v->p.frq.tab.mode) @@ -561,7 +562,7 @@ postprocess_freq_tab (struct variable *v) data = hsh_data (ft->data); /* Copy dereferenced data into freqs. */ - freqs = xmalloc (count* sizeof *freqs); + freqs = xmalloc (count * sizeof *freqs); for (i = 0; i < count; i++) { struct freq *f = data[i]; diff --git a/src/get.c b/src/get.c index aa3a4e49..dc16d7a5 100644 --- a/src/get.c +++ b/src/get.c @@ -21,6 +21,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "file-handle.h" @@ -66,10 +67,6 @@ static int save_write_case_func (struct ccase *, void *); static trns_proc_func save_trns_proc; static trns_free_func save_trns_free; -#if DEBUGGING -void dump_dict_variables (struct dictionary *); -#endif - /* Parses the GET command. */ int cmd_get (void) @@ -93,31 +90,14 @@ cmd_get (void) if (dict == NULL) return CMD_FAILURE; -#if DEBUGGING - dump_dict_variables (dict); -#endif if (0 == trim_dictionary (dict, &options)) { fh_close_handle (handle); return CMD_FAILURE; } -#if DEBUGGING - dump_dict_variables (dict); -#endif dict_compact_values (dict); -#if DEBUGGING - printf (_("GET translation table from file to memory:\n")); - for (i = 0; i < dict->nvar; i++) - { - struct variable *v = dict->var[i]; - - printf (_(" %8s from %3d,%3d to %3d,%3d\n"), v->name, - v->get.fv, v->get.nv, v->fv, v->nv); - } -#endif - dict_destroy (default_dict); default_dict = dict; @@ -158,9 +138,6 @@ cmd_save_internal (enum save_cmd save_cmd) return CMD_FAILURE; dict = dict_clone (default_dict); -#if DEBUGGING - dump_dict_variables (dict); -#endif for (i = 0; i < dict_get_var_cnt (dict); i++) dict_get_var (dict, i)->aux = dict_get_var (default_dict, i); if (0 == trim_dictionary (dict, &options)) @@ -169,10 +146,6 @@ cmd_save_internal (enum save_cmd save_cmd) return CMD_FAILURE; } -#if DEBUGGING - dump_dict_variables (dict); -#endif - /* Write dictionary. */ inf.h = handle; inf.dict = dict; @@ -236,7 +209,7 @@ do_write_case (struct save_trns *t, struct ccase *c) struct variable *v = t->var[i]; if (v->type == NUMERIC) { - double src = c->data[v->fv].f; + double src = case_num (c, v->fv); if (src == SYSMIS) *p++ = -FLT64_MAX; else @@ -244,7 +217,7 @@ do_write_case (struct save_trns *t, struct ccase *c) } else { - memcpy (p, c->data[v->fv].s, v->width); + memcpy (p, case_str (c, v->fv), v->width); memset (&((char *) p)[v->width], ' ', REM_RND_UP (v->width, sizeof *p)); p += DIV_RND_UP (v->width, sizeof *p); @@ -470,19 +443,6 @@ done: return success; } - -#if DEBUGGING -void -dump_dict_variables (struct dictionary * dict) -{ - int i; - - printf (_("\nVariables in dictionary:\n")); - for (i = 0; i < dict->nvar; i++) - printf ("%s, ", dict->var[i]->name); - printf ("\n"); -} -#endif /* Clears internal state related to GET input procedure. */ static void @@ -505,7 +465,7 @@ get_source_read (struct case_source *source, { struct get_pgm *pgm = source->aux; - while (sfm_read_case (pgm->handle, c->data, default_dict) + while (sfm_read_case (pgm->handle, c, default_dict) && write_case (wc_data)) ; } @@ -543,7 +503,7 @@ struct mtf_file struct dictionary *dict; /* Dictionary from system file. */ char in[9]; /* Name of the variable from IN=. */ char first[9], last[9]; /* Name of the variables from FIRST=, LAST=. */ - union value *input; /* Input record. */ + struct ccase input; /* Input record. */ }; /* MATCH FILES procedure. */ @@ -623,7 +583,7 @@ cmd_match_files (void) file->in[0] = file->first[0] = file->last[0] = '\0'; file->dict = NULL; file->by = NULL; - file->input = NULL; + case_nullify (&file->input); if (lex_match_id ("FILE")) file->type = MTF_FILE; @@ -703,6 +663,7 @@ cmd_match_files (void) file->dict = sfm_read_dictionary (file->handle, NULL); if (!file->dict) goto lossage; + case_create (&file->input, dict_get_next_value_idx (file->dict)); } else file->dict = default_dict; @@ -831,15 +792,6 @@ cmd_match_files (void) } } -#if DEBUGGING - { - /* From sfm-read.c. */ - extern void dump_dictionary (struct dictionary *); - - dump_dictionary (mtf.dict); - } -#endif - /* MATCH FILES performs an n-way merge on all its input files. Abstract algorithm: @@ -956,7 +908,7 @@ mtf_free_file (struct mtf_file *file) dict_destroy (file->dict); free (file->by); if (file->handle) - free (file->input); + case_destroy (&file->input); free (file); } @@ -1002,11 +954,12 @@ mtf_delete_file_in_place (struct mtf_proc *mtf, struct mtf_file **file) for (i = 0; i < dict_get_var_cnt (f->dict); i++) { struct variable *v = dict_get_var (f->dict, i); + union value *out = case_data_rw (mtf->mtf_case, v->p.mtf.master->fv); if (v->type == NUMERIC) - mtf->mtf_case->data[v->p.mtf.master->fv].f = SYSMIS; + out->f = SYSMIS; else - memset (mtf->mtf_case->data[v->p.mtf.master->fv].s, ' ', v->width); + memset (out->s, ' ', v->width); } } @@ -1024,10 +977,7 @@ mtf_read_nonactive_records (void *mtf_ UNUSED) { if (iter->handle) { - assert (iter->input == NULL); - iter->input = xmalloc (dict_get_case_size (iter->dict)); - - if (!sfm_read_case (iter->handle, iter->input, iter->dict)) + if (!sfm_read_case (iter->handle, &iter->input, iter->dict)) mtf_delete_file_in_place (mtf, &iter); else iter = iter->next; @@ -1044,12 +994,12 @@ mtf_compare_BY_values (struct mtf_proc *mtf, struct mtf_file *a, struct mtf_file *b, struct ccase *c) { - union value *a_input, *b_input; + struct ccase *a_input, *b_input; int i; assert ((a == NULL) + (b == NULL) + (c == NULL) <= 1); - a_input = a->input != NULL ? a->input : c->data; - b_input = b->input != NULL ? b->input : c->data; + a_input = case_is_null (&a->input) ? c : &a->input; + b_input = case_is_null (&b->input) ? c : &b->input; for (i = 0; i < mtf->by_cnt; i++) { assert (a->by[i]->type == b->by[i]->type); @@ -1057,8 +1007,8 @@ mtf_compare_BY_values (struct mtf_proc *mtf, if (a->by[i]->type == NUMERIC) { - double af = a_input[a->by[i]->fv].f; - double bf = b_input[b->by[i]->fv].f; + double af = case_num (a_input, a->by[i]->fv); + double bf = case_num (b_input, b->by[i]->fv); if (af < bf) return -1; @@ -1070,8 +1020,8 @@ mtf_compare_BY_values (struct mtf_proc *mtf, int result; assert (a->by[i]->type == ALPHA); - result = memcmp (a_input[a->by[i]->fv].s, - b_input[b->by[i]->fv].s, + result = memcmp (case_str (a_input, a->by[i]->fv), + case_str (b_input, b->by[i]->fv), a->by[i]->width); if (result < 0) return -1; @@ -1171,7 +1121,7 @@ mtf_processing (struct ccase *c, void *mtf_ UNUSED) case 1: if (iter->handle == NULL) return 1; - if (sfm_read_case (iter->handle, iter->input, iter->dict)) + if (sfm_read_case (iter->handle, &iter->input, iter->dict)) goto again; mtf_delete_file_in_place (mtf, &iter); break; @@ -1196,20 +1146,21 @@ mtf_processing (struct ccase *c, void *mtf_ UNUSED) for (i = 0; i < dict_get_var_cnt (iter->dict); i++) { struct variable *v = dict_get_var (iter->dict, i); - union value *record; + struct ccase *record; + union value *out; if (mtf->seq_nums[v->p.mtf.master->index] == mtf->seq_num) continue; mtf->seq_nums[v->p.mtf.master->index] = mtf->seq_num; - record = iter->input != NULL ? iter->input : c->data; + record = case_is_null (&iter->input) ? c : &iter->input; assert (v->type == NUMERIC || v->type == ALPHA); + out = case_data_rw (mtf->mtf_case, v->p.mtf.master->fv); if (v->type == NUMERIC) - mtf->mtf_case->data[v->p.mtf.master->fv].f = record[v->fv].f; + out->f = case_num (record, v->fv); else - memcpy (mtf->mtf_case->data[v->p.mtf.master->fv].s, - record[v->fv].s, v->width); + memcpy (out->s, case_str (record, v->fv), v->width); } } @@ -1223,22 +1174,17 @@ mtf_processing (struct ccase *c, void *mtf_ UNUSED) for (i = 0; i < dict_get_var_cnt (iter->dict); i++) { struct variable *v = dict_get_var (iter->dict, i); + union value *out; if (mtf->seq_nums[v->p.mtf.master->index] == mtf->seq_num) continue; mtf->seq_nums[v->p.mtf.master->index] = mtf->seq_num; -#if 0 - printf ("%s/%s: dest-fv=%d\n", - fh_handle_name (iter->handle), - v->name, - v->p.mtf.master->fv); -#endif + out = case_data_rw (mtf->mtf_case, v->p.mtf.master->fv); if (v->type == NUMERIC) - mtf->mtf_case->data[v->p.mtf.master->fv].f = SYSMIS; + out->f = SYSMIS; else - memset (mtf->mtf_case->data[v->p.mtf.master->fv].s, ' ', - v->width); + memset (out->s, ' ', v->width); } if (iter->handle == NULL) @@ -1258,9 +1204,7 @@ mtf_processing (struct ccase *c, void *mtf_ UNUSED) if (iter->handle) { - assert (iter->input != NULL); - - if (!sfm_read_case (iter->handle, iter->input, iter->dict)) + if (!sfm_read_case (iter->handle, &iter->input, iter->dict)) mtf_delete_file_in_place (mtf, &iter); } @@ -1401,31 +1345,14 @@ cmd_import (void) if (dict == NULL) return CMD_FAILURE; -#if DEBUGGING - dump_dict_variables (dict); -#endif if (0 == trim_dictionary (dict, &options)) { fh_close_handle (handle); return CMD_FAILURE; } -#if DEBUGGING - dump_dict_variables (dict); -#endif dict_compact_values (dict); -#if DEBUGGING - printf (_("IMPORT translation table from file to memory:\n")); - for (i = 0; i < dict->nvar; i++) - { - struct variable *v = dict->var[i]; - - printf (_(" %8s from %3d,%3d to %3d,%3d\n"), v->name, - v->get.fv, v->get.nv, v->fv, v->nv); - } -#endif - dict_destroy (default_dict); default_dict = dict; @@ -1446,7 +1373,7 @@ import_source_read (struct case_source *source, { struct get_pgm *pgm = source->aux; - while (pfm_read_case (pgm->handle, c->data, default_dict)) + while (pfm_read_case (pgm->handle, c, default_dict)) if (!write_case (wc_data)) break; } @@ -1483,9 +1410,6 @@ cmd_export (void) return CMD_FAILURE; dict = dict_clone (default_dict); -#if DEBUGGING - dump_dict_variables (dict); -#endif for (i = 0; i < dict_get_var_cnt (dict); i++) dict_get_var (dict, i)->aux = dict_get_var (default_dict, i); if (0 == trim_dictionary (dict, &options)) @@ -1494,10 +1418,6 @@ cmd_export (void) return CMD_FAILURE; } -#if DEBUGGING - dump_dict_variables (dict); -#endif - /* Write dictionary. */ if (!pfm_write_dictionary (handle, dict)) { @@ -1537,9 +1457,9 @@ export_write_case_func (struct ccase *c, void *aux) struct variable *v = t->var[i]; if (v->type == NUMERIC) - *p++ = c->data[v->fv]; + (*p++).f = case_num (c, v->fv); else - (*p++).c = c->data[v->fv].s; + (*p++).c = (char *) case_str (c, v->fv); } pfm_write_case (t->f, (union value *) t->case_buf); diff --git a/src/inpt-pgm.c b/src/inpt-pgm.c index 3d8ef88a..0cbf66f3 100644 --- a/src/inpt-pgm.c +++ b/src/inpt-pgm.c @@ -22,6 +22,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "command.h" #include "data-list.h" #include "dfm.h" @@ -130,14 +131,14 @@ init_case (const struct input_program_pgm *inp, struct ccase *c) switch (inp->init[i]) { case INP_NUMERIC | INP_INIT_ONCE: - c->data[i].f = 0.0; + case_data_rw (c, i)->f = 0.0; break; case INP_NUMERIC | INP_REINIT: - c->data[i].f = SYSMIS; + case_data_rw (c, i)->f = SYSMIS; break; case INP_STRING | INP_INIT_ONCE: case INP_STRING | INP_REINIT: - memset (c->data[i].s, ' ', sizeof c->data[i].s); + memset (case_data_rw (c, i)->s, ' ', sizeof case_data_rw (c, i)->s); break; default: assert (0); @@ -156,12 +157,12 @@ clear_case (const struct input_program_pgm *inp, struct ccase *c) case INP_NUMERIC | INP_INIT_ONCE: break; case INP_NUMERIC | INP_REINIT: - c->data[i].f = SYSMIS; + case_data_rw (c, i)->f = SYSMIS; break; case INP_STRING | INP_INIT_ONCE: break; case INP_STRING | INP_REINIT: - memset (c->data[i].s, ' ', sizeof c->data[i].s); + memset (case_data_rw (c, i)->s, ' ', sizeof case_data_rw (c, i)->s); break; default: assert (0); diff --git a/src/levene.c b/src/levene.c index cfbab7be..06f55f4e 100644 --- a/src/levene.c +++ b/src/levene.c @@ -20,9 +20,10 @@ 02111-1307, USA. */ #include +#include "levene.h" #include "error.h" +#include "case.h" #include "casefile.h" -#include "levene.h" #include "hash.h" #include "str.h" #include "var.h" @@ -98,7 +99,7 @@ levene(const struct casefile *cf, enum lev_missing missing, is_missing_func value_is_missing) { struct casereader *r; - const struct ccase *c; + struct ccase c; struct levene_info l; l.n_dep = n_dep; @@ -111,18 +112,20 @@ levene(const struct casefile *cf, levene_precalc(&l); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - levene_calc(c,&l); + levene_calc(&c,&l); } casereader_destroy (r); levene_postcalc(&l); levene2_precalc(&l); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - levene2_calc(c,&l); + levene2_calc(&c,&l); } casereader_destroy (r); levene2_postcalc(&l); @@ -234,7 +237,7 @@ levene_calc (const struct ccase *c, void *_l) int i; int warn = 0; struct levene_info *l = (struct levene_info *) _l; - const union value *gv = &c->data[l->v_indep->fv]; + const union value *gv = case_data (c, l->v_indep->fv); struct group_statistics key; double weight = dict_get_case_weight(default_dict,c,&warn); @@ -245,7 +248,7 @@ levene_calc (const struct ccase *c, void *_l) for (i = 0; i < l->n_dep; ++i) { struct variable *v = l->v_dep[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); if (l->is_missing(val,v) ) { @@ -262,7 +265,7 @@ levene_calc (const struct ccase *c, void *_l) { struct variable *var = l->v_dep[i]; double levene_z; - const union value *v = &c->data[var->fv]; + const union value *v = case_data (c, var->fv); struct group_statistics *gs; gs = get_group(i,&key); if ( 0 == gs ) @@ -334,7 +337,7 @@ levene2_calc (const struct ccase *c, void *_l) double weight = dict_get_case_weight(default_dict,c,&warn); - const union value *gv = &c->data[l->v_indep->fv]; + const union value *gv = case_data (c, l->v_indep->fv); struct group_statistics key; /* Skip the entire case if /MISSING=LISTWISE is set */ @@ -343,7 +346,7 @@ levene2_calc (const struct ccase *c, void *_l) for (i = 0; i < l->n_dep; ++i) { struct variable *v = l->v_dep[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); if (l->is_missing(val,v) ) { @@ -359,7 +362,7 @@ levene2_calc (const struct ccase *c, void *_l) { double levene_z; struct variable *var = l->v_dep[i] ; - const union value *v = &c->data[var->fv]; + const union value *v = case_data (c, var->fv); struct group_statistics *gs; gs = get_group(i,&key); if ( 0 == gs ) diff --git a/src/lexer.c b/src/lexer.c index 97e65101..e72f7f1b 100644 --- a/src/lexer.c +++ b/src/lexer.c @@ -1237,4 +1237,4 @@ dump_token (void) break; } } -#endif /* DEBUGGING */ +#endif /* DUMP_TOKENS */ diff --git a/src/list.q b/src/list.q index 56f45342..0ab91889 100644 --- a/src/list.q +++ b/src/list.q @@ -22,6 +22,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "command.h" #include "devind.h" #include "lexer.h" @@ -38,10 +39,6 @@ #include "debug-print.h" -#if DEBUGGING -static void debug_print (void); -#endif - /* (specification) list (lst_): *variables=varlist("PV_NO_SCRATCH"); @@ -66,7 +63,7 @@ struct list_ext static struct cmd_list cmd; /* Current case number. */ -static int case_num; +static int case_idx; /* Line buffer. */ static char *line_buf; @@ -218,14 +215,9 @@ cmd_list (void) cmd.v_variables[0] = &casenum_var; } -#if DEBUGGING - /* Print out command. */ - debug_print (); -#endif - determine_layout (); - case_num = 0; + case_idx = 0; procedure_with_splits (write_all_headers, list_cases, NULL, NULL); free (line_buf); @@ -621,9 +613,9 @@ list_cases (struct ccase *c, void *aux UNUSED) { struct outp_driver *d; - case_num++; - if (case_num < cmd.first || case_num > cmd.last - || (cmd.step != 1 && (case_num - cmd.first) % cmd.step)) + case_idx++; + if (case_idx < cmd.first || case_idx > cmd.last + || (cmd.step != 1 && (case_idx - cmd.first) % cmd.step)) return 1; for (d = outp_drivers (NULL); d; d = outp_drivers (d)) @@ -670,12 +662,12 @@ list_cases (struct ccase *c, void *aux UNUSED) } if ((formats[v->print.type].cat & FCAT_STRING) || v->fv != -1) - data_out (&line_buf[x], &v->print, &c->data[v->fv]); + data_out (&line_buf[x], &v->print, case_data (c, v->fv)); else { - union value case_num_value; - case_num_value.f = case_num; - data_out (&line_buf[x], &v->print, &case_num_value); + union value case_idx_value; + case_idx_value.f = case_idx; + data_out (&line_buf[x], &v->print, &case_idx_value); } x += v->print.w; @@ -704,12 +696,12 @@ list_cases (struct ccase *c, void *aux UNUSED) char buf[41]; if ((formats[v->print.type].cat & FCAT_STRING) || v->fv != -1) - data_out (buf, &v->print, &c->data[v->fv]); + data_out (buf, &v->print, case_data (c, v->fv)); else { - union value case_num_value; - case_num_value.f = case_num; - data_out (buf, &v->print, &case_num_value); + union value case_idx_value; + case_idx_value.f = case_idx; + data_out (buf, &v->print, &case_idx_value); } buf[v->print.w] = 0; @@ -728,45 +720,6 @@ list_cases (struct ccase *c, void *aux UNUSED) return 1; } - -/* Debugging output. */ - -#if DEBUGGING -/* Prints out the command as parsed by cmd_list(). */ -static void -debug_print (void) -{ - int i; - - puts ("LIST"); - printf (" VARIABLES="); - for (i = 0; i < cmd.n_variables; i++) - { - if (i) - putc (' ', stdout); - fputs (cmd.v_variables[i]->name, stdout); - } - - printf ("\n /CASES=FROM %ld TO %ld BY %ld\n", cmd.first, cmd.last, cmd.step); - - fputs (" /FORMAT=", stdout); - if (cmd.numbering == LST_NUMBERED) - fputs ("NUMBERED", stdout); - else - fputs ("UNNUMBERED", stdout); - putc (' ', stdout); - if (cmd.wrap == LST_WRAP) - fputs ("WRAP", stdout); - else - fputs ("SINGLE", stdout); - putc (' ', stdout); - if (cmd.weight == LST_WEIGHT) - fputs ("WEIGHT", stdout); - else - fputs ("NOWEIGHT", stdout); - puts ("."); -} -#endif /* DEBUGGING */ /* Local Variables: diff --git a/src/loop.c b/src/loop.c index b2685cfa..5e3f1699 100644 --- a/src/loop.c +++ b/src/loop.c @@ -20,6 +20,7 @@ #include #include "error.h" #include "alloc.h" +#include "case.h" #include "command.h" #include "do-ifP.h" #include "error.h" @@ -259,15 +260,6 @@ internal_cmd_loop (void) add_transformation ((struct trns_header *) one); add_transformation ((struct trns_header *) two); -#if DEBUGGING - printf ("LOOP"); - if (two->flags & LPC_INDEX) - printf ("(INDEX)"); - if (two->flags & LPC_COND) - printf ("(IF)"); - printf ("\n"); -#endif - return 1; } @@ -328,13 +320,6 @@ internal_cmd_end_loop (void) /* Pop off the top of stack. */ ctl_stack = ctl_stack->down; -#if DEBUGGING - printf ("END LOOP"); - if (thr->cond) - printf ("(IF)"); - printf ("\n"); -#endif - return 1; } @@ -360,7 +345,7 @@ loop_1_trns_proc (struct trns_header * trns, struct ccase * c, /* Even if the loop is never entered, force the index variable to assume the initial value. */ - c->data[two->index->fv].f = t1.f; + case_data_rw (c, two->index->fv)->f = t1.f; /* Throw out various pathological cases. */ if (!finite (t1.f) || !finite (t2.f) || !finite (t3.f) || t2.f == 0.0) @@ -428,7 +413,7 @@ loop_2_trns_proc (struct trns_header * trns, struct ccase * c, return two->loop_term; /* Set the current value into the case. */ - c->data[two->index->fv].f = two->curr; + case_data_rw (c, two->index->fv)->f = two->curr; /* Decrement the current value. */ two->curr += two->incr; @@ -441,7 +426,7 @@ loop_2_trns_proc (struct trns_header * trns, struct ccase * c, return two->loop_term; /* Set the current value into the case. */ - c->data[two->index->fv].f = two->curr; + case_data_rw (c, two->index->fv)->f = two->curr; /* Increment the current value. */ two->curr += two->incr; diff --git a/src/matrix-data.c b/src/matrix-data.c index 4d7c5d73..e90aef09 100644 --- a/src/matrix-data.c +++ b/src/matrix-data.c @@ -24,6 +24,7 @@ #include #include "algorithm.h" #include "alloc.h" +#include "case.h" #include "command.h" #include "data-in.h" #include "dfm.h" @@ -141,10 +142,6 @@ static void read_matrices_without_rowtype (struct matrix_data_pgm *); static void read_matrices_with_rowtype (struct matrix_data_pgm *); static int string_to_content_type (char *, int *); -#if DEBUGGING -static void debug_print (void); -#endif - int cmd_matrix_data (void) { @@ -593,10 +590,6 @@ cmd_matrix_data (void) goto lossage; } -#if DEBUGGING - debug_print (); -#endif - if (!dfm_open_for_reading (mx->data_file)) goto lossage; @@ -676,106 +669,6 @@ compare_variables_by_mxd_vartype (const void *a_, const void *b_) else return a->subtype < b->subtype ? -1 : a->subtype > b->subtype; } - -#if DEBUGGING -/* Print out the command as input. */ -static void -debug_print (void) -{ - printf ("MATRIX DATA\n\t/VARIABLES="); - - { - int i; - - for (i = 0; i < default_dict.nvar; i++) - printf ("%s ", default_dict.var[i]->name); - } - printf ("\n"); - - printf ("\t/FORMAT="); - if (fmt == LIST) - printf ("LIST"); - else if (fmt == FREE) - printf ("FREE"); - else - assert (0); - if (section == LOWER) - printf (" LOWER"); - else if (section == UPPER) - printf (" UPPER"); - else if (section == FULL) - printf (" FULL"); - else - assert (0); - if (diag == DIAGONAL) - printf (" DIAGONAL\n"); - else if (diag == NODIAGONAL) - printf (" NODIAGONAL\n"); - else - assert (0); - - if (dict_get_split_cnt (default_dict) != 0) - { - int i; - - printf ("\t/SPLIT="); - for (i = 0; i < dict_get_split_cnt (default_dict); i++) - printf ("%s ", dict_get_split_vars (default_dict)[i]->name); - if (single_split) - printf ("\t/* single split"); - printf ("\n"); - } - - if (n_factors) - { - int i; - - printf ("\t/FACTORS="); - for (i = 0; i < n_factors; i++) - printf ("%s ", factors[i]->name); - printf ("\n"); - } - - if (cells != -1) - printf ("\t/CELLS=%d\n", cells); - - if (mx->pop_n != -1) - printf ("\t/N=%d\n", mx->pop_n); - - if (mx->n_contents) - { - int i; - int space = 0; - - printf ("\t/CONTENTS="); - for (i = 0; i < mx->n_contents; i++) - { - if (mx->contents[i] == LPAREN) - { - if (space) - printf (" "); - printf ("("); - space = 0; - } - else if (mx->contents[i] == RPAREN) - { - printf (")"); - space = 1; - } - else - { - - assert (mx->contents[i] >= 0 && mx->contents[i] <= PROX); - if (space) - printf (" "); - printf ("%s", content_names[mx->contents[i]]); - space = 1; - } - } - printf ("\n"); - } -} -#endif /* DEBUGGING */ /* Matrix tokenizer. */ @@ -1035,7 +928,7 @@ read_matrices_without_rowtype (struct matrix_data_pgm *mx) vfm_source = create_case_source (&matrix_data_without_rowtype_source_class, default_dict, &nr); - procedure (NULL, &nr); + procedure (NULL, NULL); free (nr.split_values); free (nr.factor_values); @@ -1416,11 +1309,11 @@ dump_cell_content (struct matrix_data_pgm *mx, int content, double *cp, int type = content_type[content]; { - st_bare_pad_copy (c->data[mx->rowtype_->fv].s, + st_bare_pad_copy (case_data_rw (c, mx->rowtype_->fv)->s, content_names[content], 8); if (type != 1) - memset (&c->data[mx->varname_->fv].s, ' ', 8); + memset (case_data_rw (c, mx->varname_->fv)->s, ' ', 8); } { @@ -1434,11 +1327,11 @@ dump_cell_content (struct matrix_data_pgm *mx, int content, double *cp, for (j = 0; j < mx->n_continuous; j++) { int fv = dict_get_var (default_dict, mx->first_continuous + j)->fv; - c->data[fv].f = *cp; + case_data_rw (c, fv)->f = *cp; cp++; } if (type == 1) - st_bare_pad_copy (c->data[mx->varname_->fv].s, + st_bare_pad_copy (case_data_rw (c, mx->varname_->fv)->s, dict_get_var (default_dict, mx->first_continuous + i)->name, 8); @@ -1462,7 +1355,7 @@ nr_output_data (struct nr_aux_data *nr, struct ccase *c, split_cnt = dict_get_split_cnt (default_dict); split = dict_get_split_vars (default_dict); for (i = 0; i < split_cnt; i++) - c->data[split[i]->fv].f = nr->split_values[i]; + case_data_rw (c, split[i]->fv)->f = nr->split_values[i]; } if (mx->n_factors) @@ -1476,7 +1369,7 @@ nr_output_data (struct nr_aux_data *nr, struct ccase *c, for (factor = 0; factor < mx->n_factors; factor++) { - c->data[mx->factors[factor]->fv].f + case_data_rw (c, mx->factors[factor]->fv)->f = nr->factor_values[factor + cell * mx->n_factors]; debug_printf (("f:%s ", mx->factors[factor]->name)); } @@ -1505,7 +1398,7 @@ nr_output_data (struct nr_aux_data *nr, struct ccase *c, int factor; for (factor = 0; factor < mx->n_factors; factor++) - c->data[mx->factors[factor]->fv].f = SYSMIS; + case_data_rw (c, mx->factors[factor]->fv)->f = SYSMIS; } for (content = 0; content <= PROX; content++) @@ -1565,7 +1458,7 @@ read_matrices_with_rowtype (struct matrix_data_pgm *mx) vfm_source = create_case_source (&matrix_data_with_rowtype_source_class, default_dict, &wr); - procedure (NULL, &wr); + procedure (NULL, NULL); free (wr.split_values); fh_close_handle (mx->data_file); @@ -1703,7 +1596,7 @@ wr_output_data (struct wr_aux_data *wr, split_cnt = dict_get_split_cnt (default_dict); split = dict_get_split_vars (default_dict); for (i = 0; i < split_cnt; i++) - c->data[split[i]->fv].f = wr->split_values[i]; + case_data_rw (c, split[i]->fv)->f = wr->split_values[i]; } /* Sort the wr->data list. */ @@ -1738,7 +1631,7 @@ wr_output_data (struct wr_aux_data *wr, for (factor = 0; factor < mx->n_factors; factor++) { - c->data[mx->factors[factor]->fv].f + case_data_rw (c, mx->factors[factor]->fv)->f = iter->factors[factor]; debug_printf (("f:%s ", factors[factor]->name)); } diff --git a/src/means.q b/src/means.q index 846448b1..483b6dc6 100644 --- a/src/means.q +++ b/src/means.q @@ -48,10 +48,6 @@ /* (declarations) */ /* (functions) */ -#if DEBUGGING -static void debug_print (struct cmd_means *cmd); -#endif - /* TABLES: Variable lists for each dimension. */ int n_dim; /* Number of dimensions. */ int *nv_dim; /* Number of variables in each dimension. */ @@ -106,10 +102,6 @@ cmd_means (void) goto free; } -#if DEBUGGING - debug_print (&cmd); -#endif - success = CMD_SUCCESS; free: @@ -311,61 +303,6 @@ mns_custom_variables (struct cmd_means *cmd) return 1; } -#if DEBUGGING -static void -debug_print (struct cmd_means *cmd) -{ - int i; - - printf ("MEANS"); - - if (cmd->sbc_variables) - { - int j = 0; - - printf (" VARIABLES="); - for (i = 0; i < default_dict.nvar; i++) - { - struct variable *v = default_dict.var[i]; - - if (v->p.mns.min == SYSMIS) - continue; - if (j++) - printf (" "); - printf ("%s(", v->name); - if (v->p.mns.min == LOWEST) - printf ("LO"); - else - printf ("%g", v->p.mns.min); - printf (","); - if (v->p.mns.max == HIGHEST) - printf ("HI"); - else - printf ("%g", v->p.mns.max); - printf (")"); - } - printf ("\n"); - } - - printf (" TABLES="); - for (i = 0; i < n_dim; i++) - { - int j; - - if (i) - printf (" BY"); - - for (j = 0; j < nv_dim[i]; j++) - { - if (i || j) - printf (" "); - printf (v_dim[i][j]->name); - } - } - printf ("\n"); -} -#endif /* DEBUGGING */ - /* Local Variables: mode: c diff --git a/src/mis-val.c b/src/mis-val.c index f388583a..ce326255 100644 --- a/src/mis-val.c +++ b/src/mis-val.c @@ -29,10 +29,6 @@ #include "debug-print.h" -#if DEBUGGING -static void debug_print (); -#endif - /* Variables on MIS VAL. */ static struct variable **v; static int nv; @@ -86,10 +82,6 @@ cmd_missing_values (void) free (v); } -#if 0 && DEBUGGING - debug_print (); -#endif - return lex_end_of_command (); fail: @@ -351,51 +343,3 @@ copy_missing_values (struct variable *dest, const struct variable *src) memcpy (dest->missing[i].s, src->missing[i].s, src->width); } } - - -/* Debug output. */ - -#if 0 && DEBUGGING -static void -debug_print (void) -{ - int i, j; - - puts (_("Missing value:")); - for (i = 0; i < nvar; i++) - { - printf (" %8s: ", var[i]->name); - if (var[i]->type == ALPHA && var[i]->nv > 1) - puts (_("(long string variable)")); - else - switch (var[i]->miss_type) - { - case MISSING_NONE: - printf (_("(no missing values)\n")); - break; - case MISSING_1: - case MISSING_2: - case MISSING_3: - printf ("(MISSING_%d)", var[i]->miss_type); - for (j = 0; j < var[i]->miss_type; j++) - if (var[i]->type == ALPHA) - printf (" \"%.*s\"", var[i]->width, var[i]->missing[j].s); - else - printf (" %.2g", var[i]->missing[j].f); - printf ("\n"); - break; - case MISSING_RANGE: - printf ("(MISSING_RANGE) %.2g THRU %.2g\n", - var[i]->missing[0].f, var[i]->missing[1].f); - break; - case MISSING_RANGE_1: - printf ("(MISSING_RANGE_1) %.2g THRU %.2g, %.2g\n", - var[i]->missing[0].f, var[i]->missing[1].f, - var[i]->missing[2].f); - break; - default: - printf (_("(!!!INTERNAL ERROR--%d!!!)\n"), var[i]->miss_type); - } - } -} -#endif /* DEBUGGING */ diff --git a/src/pfm-read.c b/src/pfm-read.c index ab33e700..7be8ea05 100644 --- a/src/pfm-read.c +++ b/src/pfm-read.c @@ -27,6 +27,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "file-handle.h" #include "format.h" #include "getline.h" @@ -274,9 +275,6 @@ pfm_read_dictionary (struct file_handle *h, struct pfm_read_info *inf) msg (VM (2), _("Read portable-file dictionary successfully.")); -#if DEBUGGING - dump_dictionary (ext->dict); -#endif return ext->dict; lossage: @@ -972,12 +970,13 @@ read_value_label (struct file_handle *h) return 0; } -/* Reads one case from portable file H into the value array PERM +/* Reads one case from portable file H into PERM according to the instuctions given in associated dictionary DICT, which must have the get.fv elements appropriately set. Returns nonzero only if successful. */ int -pfm_read_case (struct file_handle *h, union value *perm, struct dictionary *dict) +pfm_read_case (struct file_handle *h, struct ccase *perm, + struct dictionary *dict) { struct pfm_fhuser_ext *ext = h->ext; @@ -1021,9 +1020,9 @@ pfm_read_case (struct file_handle *h, union value *perm, struct dictionary *dict continue; if (v->type == NUMERIC) - perm[v->fv].f = temp[v->get.fv].f; + case_data_rw (perm, v->fv)->f = temp[v->get.fv].f; else - memcpy (&perm[v->fv].s, &temp[v->get.fv], v->width); + memcpy (case_data_rw (perm, v->fv)->s, &temp[v->get.fv], v->width); } local_free (temp); diff --git a/src/pfm.h b/src/pfm.h index ad506ac3..172a8f53 100644 --- a/src/pfm.h +++ b/src/pfm.h @@ -44,12 +44,12 @@ struct pfm_read_info struct dictionary; struct file_handle; +struct ccase; union value; struct dictionary *pfm_read_dictionary (struct file_handle *, struct pfm_read_info *); -int pfm_read_case (struct file_handle *, - union value *, struct dictionary *); +int pfm_read_case (struct file_handle *, struct ccase *, struct dictionary *); int pfm_write_dictionary (struct file_handle *, struct dictionary *); int pfm_write_case (struct file_handle *, const union value *elem); diff --git a/src/print.c b/src/print.c index b3098d67..1bbc5ae6 100644 --- a/src/print.c +++ b/src/print.c @@ -23,6 +23,7 @@ #include "error.h" #include #include "alloc.h" +#include "case.h" #include "command.h" #include "dfm.h" #include "error.h" @@ -950,7 +951,7 @@ print_trns_proc (struct trns_header * trns, struct ccase * c, break; case PRT_VAR: - data_out (&buf[i->fc], &i->u.v.f, &c->data[i->u.v.v->fv]); + data_out (&buf[i->fc], &i->u.v.f, case_data (c, i->u.v.v->fv)); len = i->fc + i->u.v.f.w; break; diff --git a/src/recode.c b/src/recode.c index a4abc534..089caba1 100644 --- a/src/recode.c +++ b/src/recode.c @@ -23,6 +23,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "command.h" #include "error.h" #include "lexer.h" @@ -105,7 +106,7 @@ static int parse_dest_spec (struct rcd_var * rcd, union value *v, static int parse_src_spec (struct rcd_var * rcd, int type, size_t max_src_width); static trns_proc_func recode_trns_proc; static trns_free_func recode_trns_free; -static double convert_to_double (char *, int); +static double convert_to_double (const char *, int); /* Parser. */ @@ -687,7 +688,7 @@ recode_trns_free (struct trns_header * t) static inline struct coding * find_src_numeric (struct rcd_var * v, struct ccase * c) { - double cmp = c->data[v->src->fv].f; + double cmp = case_num (c, v->src->fv); struct coding *cp; if (cmp == SYSMIS) @@ -695,9 +696,9 @@ find_src_numeric (struct rcd_var * v, struct ccase * c) if (v->sysmis.f != -SYSMIS) { if ((v->flags & RCD_DEST_MASK) == RCD_DEST_NUMERIC) - c->data[v->dest->fv].f = v->sysmis.f; + case_data_rw (c, v->dest->fv)->f = v->sysmis.f; else - memcpy (c->data[v->dest->fv].s, v->sysmis.s, + memcpy (case_data_rw (c, v->dest->fv)->s, v->sysmis.s, v->dest->width); } return NULL; @@ -738,7 +739,7 @@ find_src_numeric (struct rcd_var * v, struct ccase * c) static inline struct coding * find_src_string (struct rcd_var * v, struct ccase * c) { - char *cmp = c->data[v->src->fv].s; + const char *cmp = case_str (c, v->src->fv); int w = v->src->width; struct coding *cp; @@ -758,7 +759,7 @@ find_src_string (struct rcd_var * v, struct ccase * c) double f = convert_to_double (cmp, w); if (f != -SYSMIS) { - c->data[v->dest->fv].f = f; + case_data_rw (c, v->dest->fv)->f = f; return NULL; } break; @@ -770,7 +771,7 @@ find_src_string (struct rcd_var * v, struct ccase * c) static int recode_trns_proc (struct trns_header * t, struct ccase * c, - int case_num UNUSED) + int case_idx UNUSED) { struct rcd_var *v; @@ -797,20 +798,24 @@ recode_trns_proc (struct trns_header * t, struct ccase * c, if ((v->flags & RCD_DEST_MASK) == RCD_DEST_NUMERIC) { double val = cp->t.f; + double *out = &case_data_rw (c, v->dest->fv)->f; if (val == -SYSMIS) - c->data[v->dest->fv].f = c->data[v->src->fv].f; + *out = case_num (c, v->src->fv); else - c->data[v->dest->fv].f = val; + *out = val; } else { char *val = cp->t.c; if (val == NULL) - st_bare_pad_len_copy (c->data[v->dest->fv].s, - c->data[v->src->fv].s, - v->dest->width, v->src->width); + { + if (v->dest->fv != v->src->fv) + st_bare_pad_len_copy (case_data_rw (c, v->dest->fv)->s, + case_str (c, v->src->fv), + v->dest->width, v->src->width); + } else - memmove (c->data[v->dest->fv].s, cp->t.c, v->dest->width); + memcpy (case_data_rw (c, v->dest->fv)->s, cp->t.c, v->dest->width); } } @@ -822,13 +827,13 @@ recode_trns_proc (struct trns_header * t, struct ccase * c, first character after the number into *ENDPTR. From the GNU C library. */ static long int -string_to_long (char *nptr, int width, char **endptr) +string_to_long (const char *nptr, int width, const char **endptr) { int negative; register unsigned long int cutoff; register unsigned int cutlim; register unsigned long int i; - register char *s; + register const char *s; register unsigned char c; const char *save; @@ -898,7 +903,7 @@ string_to_long (char *nptr, int width, char **endptr) found, or -SYSMIS if there was no valid number in s. WIDTH is the length of string S. From the GNU C library. */ static double -convert_to_double (char *s, int width) +convert_to_double (const char *s, int width) { register const char *end = &s[width]; diff --git a/src/repeat.c b/src/repeat.c index e992b29f..2d788e27 100644 --- a/src/repeat.c +++ b/src/repeat.c @@ -60,11 +60,6 @@ static int parse_strings (struct repeat_entry *); static void clean_up (void); static int internal_cmd_do_repeat (void); -#if DEBUGGING -static void debug_print (void); -static void debug_print_lines (void); -#endif - int cmd_do_repeat (void) { @@ -208,10 +203,6 @@ internal_cmd_do_repeat (void) } while (token != '.'); -#if DEBUGGING - debug_print (); -#endif - /* Read all the lines inside the DO REPEAT ... END REPEAT. */ { int nest = 1; @@ -309,11 +300,6 @@ internal_cmd_do_repeat (void) } line_buf_tail->next = NULL; - /* Show the line list. */ -#if DEBUGGING - debug_print_lines (); -#endif - /* Make new variables. */ { struct repeat_entry *iter; @@ -594,45 +580,3 @@ perform_DO_REPEAT_substitutions (void) ds_destroy (&getl_buf); getl_buf = output; } - -/* Debugging code. */ - -#if DEBUGGING -static void -debug_print (void) -{ - struct repeat_entry *iter; - int j; - - printf ("DO REPEAT\n"); - for (iter = repeat_tab; iter; iter = iter->next) - { - printf (" %s%s=", iter->id, iter->type ? "(ids)" : ""); - for (j = 0; j < count; j++) - printf ("%s ", iter->replacement[j]); - putc (iter->next ? '/' : '.', stdout); - printf ("\n"); - } -} - -static void -debug_print_lines (void) -{ - struct getl_line_list *iter; - const char *fn = "(none)"; - int ln = 65536; - - printf ("---begin DO REPEAT lines---\n"); - for (iter = line_buf_head; iter; iter = iter->next) - { - if (iter->len < 0) - { - ln = -iter->len; - fn = iter->line; - } else { - printf ("%s:%d: %s", fn, ln++, iter->line); - } - } - printf ("---end DO REPEAT lines---\n"); -} -#endif /* DEBUGGING */ diff --git a/src/sample.c b/src/sample.c index 67523b3d..8db7c837 100644 --- a/src/sample.c +++ b/src/sample.c @@ -96,13 +96,6 @@ cmd_sample (void) } lex_get (); -#if DEBUGGING - if (type == TYPE_FRACTION) - printf ("SAMPLE %g.\n", frac / (double) UINT_MAX); - else - printf ("SAMPLE %d FROM %d.\n", a, b); -#endif - trns = xmalloc (sizeof *trns); trns->h.proc = sample_trns_proc; trns->h.free = NULL; diff --git a/src/sfm-read.c b/src/sfm-read.c index fad27c94..622f0a3f 100644 --- a/src/sfm-read.c +++ b/src/sfm-read.c @@ -26,6 +26,7 @@ #include #include #include "alloc.h" +#include "case.h" #include "error.h" #include "file-handle.h" #include "filename.h" @@ -77,10 +78,6 @@ struct sfm_fhuser_ext }; static struct fh_ext_class sfm_r_class; - -#if GLOBAL_DEBUGGING -void dump_dictionary (struct dictionary * dict); -#endif /* Utilities. */ @@ -396,9 +393,6 @@ break_out_of_loop: /* Come here on successful completion. */ msg (VM (2), _("Read system-file dictionary successfully.")); -#if DEBUGGING - dump_dictionary (ext->dict); -#endif free (var_by_index); return ext->dict; @@ -1182,86 +1176,6 @@ read_documents (struct file_handle * h) lossage: return 0; } - -#if GLOBAL_DEBUGGING -#include "debug-print.h" -/* Displays dictionary DICT on stdout. */ -void -dump_dictionary (struct dictionary * dict) -{ - int i; - - debug_printf ((_("dictionary:\n"))); - for (i = 0; i < dict->nvar; i++) - { - char print[32]; - struct variable *v = dict->var[i]; - int n, j; - - debug_printf ((" var %s", v->name)); - debug_printf (("(type:%s,%d)", (v->type == NUMERIC ? _("num") - : (v->type == ALPHA ? _("str") : "!!!")), - v->width)); - debug_printf (("(fv:%d,%d)", v->fv, v->nv)); - debug_printf (("(left:%s)(miss:", v->left ? _("left") : _("right"))); - - switch (v->miss_type) - { - case MISSING_NONE: - n = 0; - debug_printf ((_("none"))); - break; - case MISSING_1: - n = 1; - debug_printf ((_("one"))); - break; - case MISSING_2: - n = 2; - debug_printf ((_("two"))); - break; - case MISSING_3: - n = 3; - debug_printf ((_("three"))); - break; - case MISSING_RANGE: - n = 2; - debug_printf ((_("range"))); - break; - case MISSING_LOW: - n = 1; - debug_printf ((_("low"))); - break; - case MISSING_HIGH: - n = 1; - debug_printf ((_("high"))); - break; - case MISSING_RANGE_1: - n = 3; - debug_printf ((_("range+1"))); - break; - case MISSING_LOW_1: - n = 2; - debug_printf ((_("low+1"))); - break; - case MISSING_HIGH_1: - n = 2; - debug_printf ((_("high+1"))); - break; - default: - assert (0); - } - for (j = 0; j < n; j++) - if (v->type == NUMERIC) - debug_printf ((",%g", v->missing[j].f)); - else - debug_printf ((",\"%.*s\"", v->width, v->missing[j].s)); - strcpy (print, fmt_to_string (&v->print)); - debug_printf ((")(fmt:%s,%s)(lbl:%s)\n", - print, fmt_to_string (&v->write), - v->label ? v->label : "nolabel")); - } -} -#endif /* Data reader. */ @@ -1394,12 +1308,12 @@ lossage: return 0; } -/* Reads one case from system file H into the value array PERM +/* Reads one case from system file H into PERM according to the instructions given in associated dictionary DICT, which must have the get.* elements appropriately set. Returns nonzero only if successful. */ int -sfm_read_case (struct file_handle * h, union value * perm, struct dictionary * dict) +sfm_read_case (struct file_handle * h, struct ccase *perm, struct dictionary * dict) { struct sfm_fhuser_ext *ext = h->ext; @@ -1446,10 +1360,10 @@ sfm_read_case (struct file_handle * h, union value * perm, struct dictionary * d flt64 src = temp[v->get.fv]; if (ext->reverse_endian) bswap_flt64 (&src); - perm[v->fv].f = src == ext->sysmis ? SYSMIS : src; + case_data_rw (perm, v->fv)->f = src == ext->sysmis ? SYSMIS : src; } else - memcpy (&perm[v->fv].s, &temp[v->get.fv], v->width); + memcpy (case_data_rw (perm, v->fv)->s, &temp[v->get.fv], v->width); } local_free (temp); diff --git a/src/sfm.h b/src/sfm.h index edd19d9e..e5691649 100644 --- a/src/sfm.h +++ b/src/sfm.h @@ -41,11 +41,11 @@ struct sfm_read_info struct dictionary; struct file_handle; -union value; +struct ccase; struct dictionary *sfm_read_dictionary (struct file_handle *, struct sfm_read_info *); -int sfm_read_case (struct file_handle *, union value *, struct dictionary *); +int sfm_read_case (struct file_handle *, struct ccase *, struct dictionary *); void sfm_maybe_close (struct file_handle *); /* Information needed by sfm_write_dictionary(). */ diff --git a/src/sort.c b/src/sort.c index 5f9cdb42..33c5651d 100644 --- a/src/sort.c +++ b/src/sort.c @@ -25,6 +25,7 @@ #include #include "algorithm.h" #include "alloc.h" +#include "case.h" #include "casefile.h" #include "command.h" #include "error.h" @@ -36,7 +37,6 @@ #include "var.h" #include "vfm.h" #include "vfmP.h" -#include "workspace.h" #if HAVE_UNISTD_H #include @@ -52,69 +52,136 @@ #include "debug-print.h" -/* Other prototypes. */ -static int compare_record (const union value *, const union value *, - const struct sort_cases_pgm *, int *idx_to_fv); -static int compare_cases (const struct ccase *, const struct ccase *, void *); +/* Sort direction. */ +enum sort_direction + { + SRT_ASCEND, /* A, B, C, ..., X, Y, Z. */ + SRT_DESCEND /* Z, Y, X, ..., C, B, A. */ + }; + +/* A sort criterion. */ +struct sort_criterion + { + int fv; /* Variable data index. */ + int width; /* 0=numeric, otherwise string widthe. */ + enum sort_direction dir; /* Sort direction. */ + }; + +/* A set of sort criteria. */ +struct sort_criteria + { + struct sort_criterion *crits; + size_t crit_cnt; + }; + static int compare_case_dblptrs (const void *, const void *, void *); -static struct internal_sort *do_internal_sort (struct sort_cases_pgm *, - int separate); -static void destroy_internal_sort (struct internal_sort *); -static struct external_sort *do_external_sort (struct sort_cases_pgm *, - int separate); -static void destroy_external_sort (struct external_sort *); -struct sort_cases_pgm *parse_sort (void); +static int compare_record (const struct ccase *, const struct ccase *, + const struct sort_criteria *); +static struct casefile *do_internal_sort (struct casereader *, + const struct sort_criteria *); +static struct casefile *do_external_sort (struct casereader *, + const struct sort_criteria *); /* Performs the SORT CASES procedures. */ int cmd_sort_cases (void) { - struct sort_cases_pgm *scp; + struct sort_criteria *criteria; int success; lex_match (T_BY); - scp = parse_sort (); - if (scp == NULL) + criteria = sort_parse_criteria (default_dict, NULL, NULL); + if (criteria == NULL) return CMD_FAILURE; + success = sort_active_file_in_place (criteria); + sort_destroy_criteria (criteria); + return success ? lex_end_of_command () : CMD_FAILURE; +} + +/* Gets ready to sort the active file, either in-place or to a + separate casefile. */ +static void +prepare_to_sort_active_file (void) +{ + /* Cancel temporary transformations and PROCESS IF. */ if (temporary != 0) - { - msg (SE, _("SORT CASES may not be used after TEMPORARY. " - "Temporary transformations will be made permanent.")); - cancel_temporary (); - } + cancel_temporary (); + expr_free (process_if_expr); + process_if_expr = NULL; - success = sort_cases (scp, 0); - destroy_sort_cases_pgm (scp); - if (success) - return lex_end_of_command (); - else - return CMD_FAILURE; + /* Make sure source cases are in a storage source. */ + procedure (NULL, NULL); + assert (case_source_is_class (vfm_source, &storage_source_class)); +} + +/* Sorts the active file in-place according to CRITERIA. + Returns nonzero if successful. */ +int +sort_active_file_in_place (const struct sort_criteria *criteria) +{ + struct casefile *src, *dst; + + prepare_to_sort_active_file (); + + src = storage_source_get_casefile (vfm_source); + dst = sort_execute (casefile_get_destructive_reader (src), criteria); + free_case_source (vfm_source); + vfm_source = NULL; + + if (dst == NULL) + return 0; + + vfm_source = storage_source_create (dst, default_dict); + return 1; +} + +/* Sorts the active file to a separate casefile. If successful, + returns the sorted casefile. Returns a null pointer on + failure. */ +struct casefile * +sort_active_file_to_casefile (const struct sort_criteria *criteria) +{ + struct casefile *src; + + prepare_to_sort_active_file (); + + src = storage_source_get_casefile (vfm_source); + return sort_execute (casefile_get_reader (src), criteria); } /* Parses a list of sort keys and returns a struct sort_cases_pgm based on it. Returns a null pointer on error. */ -struct sort_cases_pgm * -parse_sort (void) +struct sort_criteria * +sort_parse_criteria (const struct dictionary *dict, + struct variable ***vars, int *var_cnt) { - struct sort_cases_pgm *scp; + struct sort_criteria *criteria; + struct variable **local_vars = NULL; + size_t local_var_cnt; + + assert ((vars == NULL) == (var_cnt == NULL)); + if (vars == NULL) + { + vars = &local_vars; + var_cnt = &local_var_cnt; + } + + criteria = xmalloc (sizeof *criteria); + criteria->crits = NULL; + criteria->crit_cnt = 0; - scp = xmalloc (sizeof *scp); - scp->ref_cnt = 1; - scp->vars = NULL; - scp->dirs = NULL; - scp->var_cnt = 0; - scp->isrt = NULL; - scp->xsrt = NULL; + *vars = NULL; + *var_cnt = 0; do { - int prev_var_cnt = scp->var_cnt; - enum sort_direction direction = SRT_ASCEND; + int prev_var_cnt = *var_cnt; + enum sort_direction direction; /* Variables. */ - if (!parse_variables (default_dict, &scp->vars, &scp->var_cnt, + if (!parse_variables (dict, vars, var_cnt, PV_NO_DUPLICATE | PV_APPEND | PV_NO_SCRATCH)) goto error; @@ -123,7 +190,9 @@ parse_sort (void) { if (lex_match_id ("D") || lex_match_id ("DOWN")) direction = SRT_DESCEND; - else if (!lex_match_id ("A") && !lex_match_id ("UP")) + else if (lex_match_id ("A") || lex_match_id ("UP")) + direction = SRT_ASCEND; + else { msg (SE, _("`A' or `D' expected inside parentheses.")); goto error; @@ -134,178 +203,114 @@ parse_sort (void) goto error; } } - scp->dirs = xrealloc (scp->dirs, sizeof *scp->dirs * scp->var_cnt); - for (; prev_var_cnt < scp->var_cnt; prev_var_cnt++) - scp->dirs[prev_var_cnt] = direction; + else + direction = SRT_ASCEND; + + criteria->crits = xrealloc (criteria->crits, + sizeof *criteria->crits * *var_cnt); + criteria->crit_cnt = *var_cnt; + for (; prev_var_cnt < criteria->crit_cnt; prev_var_cnt++) + { + struct sort_criterion *c = &criteria->crits[prev_var_cnt]; + c->fv = (*vars)[prev_var_cnt]->fv; + c->width = (*vars)[prev_var_cnt]->width; + c->dir = direction; + } } while (token != '.' && token != '/'); - - return scp; + + free (local_vars); + return criteria; error: - destroy_sort_cases_pgm (scp); + free (local_vars); + sort_destroy_criteria (criteria); return NULL; } /* Destroys a SORT CASES program. */ void -destroy_sort_cases_pgm (struct sort_cases_pgm *scp) +sort_destroy_criteria (struct sort_criteria *criteria) { - if (scp != NULL) + if (criteria != NULL) { - assert (scp->ref_cnt > 0); - if (--scp->ref_cnt > 0) - return; - - free (scp->vars); - free (scp->dirs); - destroy_internal_sort (scp->isrt); - destroy_external_sort (scp->xsrt); - free (scp); + free (criteria->crits); + free (criteria); } } -/* Sorts the active file based on the key variables specified in - global variables vars and var_cnt. - - If SEPARATE is zero, then output goes to the active file. The - output cases can be read through the usual VFM interfaces. - - If SEPARATE is nonzero, then output goes to a separate file. - The output cases can be read with a call to - read_sort_output(). - - The caller is responsible for freeing SCP. */ -int -sort_cases (struct sort_cases_pgm *scp, int separate) +/* Reads all the cases from READER, which is destroyed. Sorts + the cases according to CRITERIA. Returns the sorted cases in + a newly created casefile. */ +struct casefile * +sort_execute (struct casereader *reader, const struct sort_criteria *criteria) { - scp->case_size - = sizeof (union value) * dict_get_compacted_value_cnt (default_dict); + struct casefile *output; - /* Not sure this is necessary but it's good to be safe. */ - if (separate && case_source_is_class (vfm_source, &sort_source_class)) - procedure (NULL, NULL); - - /* SORT CASES cancels PROCESS IF. */ - expr_free (process_if_expr); - process_if_expr = NULL; - - /* Try an internal sort first. */ - scp->isrt = do_internal_sort (scp, separate); - if (scp->isrt != NULL) - return 1; - - /* Fall back to an external sort. */ - scp->xsrt = do_external_sort (scp, separate); - if (scp->xsrt != NULL) - return 1; - - destroy_sort_cases_pgm (scp); - return 0; + output = do_internal_sort (reader, criteria); + if (output == NULL) + output = do_external_sort (reader, criteria); + casereader_destroy (reader); + return output; } -/* Results of an internal sort. - Used only for sorting to a separate file. */ -struct internal_sort - { - const struct ccase **cases; - size_t case_cnt; - }; - -/* If the data is in memory, do an internal sort. Return - success. */ -static struct internal_sort * -do_internal_sort (struct sort_cases_pgm *scp, int separate) -{ - struct internal_sort *isrt; - - isrt = xmalloc (sizeof *isrt); - isrt->cases = NULL; - isrt->case_cnt = 0; - - if (case_source_is_class (vfm_source, &storage_source_class)) +/* If the data is in memory, do an internal sort and return a new + casefile for the data. */ +static struct casefile * +do_internal_sort (struct casereader *reader, + const struct sort_criteria *criteria) +{ + const struct casefile *src; + struct casefile *dst; + struct ccase *cases, **case_ptrs; + unsigned long case_cnt; + + src = casereader_get_casefile (reader); + if (casefile_get_case_cnt (src) > 1 && !casefile_in_core (src)) + return NULL; + + case_cnt = casefile_get_case_cnt (src); + cases = malloc (sizeof *cases * case_cnt); + case_ptrs = malloc (sizeof *case_ptrs * case_cnt); + if ((cases != NULL && case_ptrs != NULL) || case_cnt == 0) { - struct casefile *casefile = storage_source_get_casefile (vfm_source); - - if (!separate) - { - if (!casefile_sort (casefile, compare_cases, scp)) - goto error; - } - else + unsigned long case_idx; + + for (case_idx = 0; case_idx < case_cnt; case_idx++) { - /* FIXME FIXME FIXME. - This is crap because the casefile could get flushed - to disk between the time we sort it and we use it - later, causing invalid pointer accesses. - The right solution is probably to extend casefiles - to support duplication. */ - struct casereader *reader; - size_t case_idx; - - if (!casefile_in_core (casefile)) - goto error; - - isrt->case_cnt = casefile_get_case_cnt (casefile); - isrt->cases = workspace_malloc (sizeof *isrt->cases - * isrt->case_cnt); - if (isrt->cases == NULL) - goto error; - - reader = casefile_get_reader (casefile); - for (case_idx = 0; case_idx < isrt->case_cnt; case_idx++) - { - casereader_read (reader, &isrt->cases[case_idx]); - assert (isrt->cases[case_idx] != NULL); - } - casereader_destroy (reader); - - sort (isrt->cases, isrt->case_cnt, casefile_get_case_size (casefile), - compare_case_dblptrs, scp); + int success = casereader_read_xfer (reader, &cases[case_idx]); + assert (success); + case_ptrs[case_idx] = &cases[case_idx]; } - return isrt; + sort (case_ptrs, case_cnt, sizeof *case_ptrs, compare_case_dblptrs, + (void *) criteria); + + dst = casefile_create (casefile_get_value_cnt (src)); + for (case_idx = 0; case_idx < case_cnt; case_idx++) + casefile_append_xfer (dst, case_ptrs[case_idx]); } + else + dst = NULL; - error: - free (isrt); - return NULL; -} - -/* Destroys an internal sort result. */ -static void -destroy_internal_sort (struct internal_sort *isrt) -{ - if (isrt != NULL) - { - workspace_free (isrt->cases, sizeof *isrt->cases * isrt->case_cnt); - free (isrt); - } -} - -/* Compares the variables specified by SCP between the cases at A - and B, and returns a strcmp()-type result. */ -static int -compare_cases (const struct ccase *a, const struct ccase *b, - void *scp_) -{ - struct sort_cases_pgm *scp = scp_; + free (case_ptrs); + free (cases); - return compare_record (a->data, b->data, scp, NULL); + return dst; } -/* Compares the variables specified by SCP between the cases at A - and B, and returns a strcmp()-type result. */ +/* Compares the variables specified by CRITERIA between the cases + at A and B, and returns a strcmp()-type result. */ static int -compare_case_dblptrs (const void *a_, const void *b_, void *scp_) +compare_case_dblptrs (const void *a_, const void *b_, void *criteria_) { - struct sort_cases_pgm *scp = scp_; + struct sort_criteria *criteria = criteria_; struct ccase *const *pa = a_; struct ccase *const *pb = b_; struct ccase *a = *pa; struct ccase *b = *pb; - - return compare_record (a->data, b->data, scp, NULL); + + return compare_record (a, b, criteria); } /* External sort. */ @@ -325,74 +330,57 @@ compare_case_dblptrs (const void *a_, const void *b_, void *scp_) #error MIN_BUFFER_SIZE_RECS and MIN_BUFFER_TOTAL_SIZE_RECS do not make sense. #endif -/* An initial run and its length. */ -struct initial_run - { - int file_idx; /* File index. */ - size_t case_cnt; /* Number of cases. */ - }; - /* Sorts initial runs A and B in decending order by length. */ static int compare_initial_runs (const void *a_, const void *b_, void *aux UNUSED) { - const struct initial_run *a = a_; - const struct initial_run *b = b_; + const struct casefile *a = a_; + const struct casefile *b = b_; + unsigned long a_case_cnt = casefile_get_case_cnt (a); + unsigned long b_case_cnt = casefile_get_case_cnt (b); - return a->case_cnt > b->case_cnt ? -1 : a->case_cnt case_cnt; + return a_case_cnt > b_case_cnt ? -1 : a_case_cnt < b_case_cnt; } /* Results of an external sort. */ struct external_sort { - struct sort_cases_pgm *scp; /* SORT CASES info. */ - struct initial_run *initial_runs; /* Array of initial runs. */ + const struct sort_criteria *criteria; /* Sort criteria. */ + size_t value_cnt; /* Size of data in `union value's. */ + struct casefile **initial_runs; /* Array of initial runs. */ size_t run_cnt, run_cap; /* Number of runs, allocated capacity. */ - char *temp_dir; /* Temporary file directory name. */ - char *temp_name; /* Name of a temporary file. */ - int next_file_idx; /* Lowest unused file index. */ }; /* Prototypes for helper functions. */ -static void sort_sink_write (struct case_sink *, const struct ccase *); -static int write_initial_runs (struct external_sort *, int separate); -static int init_external_sort (struct external_sort *); +static int write_initial_runs (struct external_sort *, struct casereader *); static int merge (struct external_sort *); -static void rmdir_temp_dir (struct external_sort *); -static void remove_temp_file (struct external_sort *xsrt, int file_idx); +static void destroy_external_sort (struct external_sort *); /* Performs an external sort of the active file according to the specification in SCP. Forms initial runs using a heap as a reservoir. Determines the optimum merge pattern via Huffman's method (see Knuth vol. 3, 2nd edition, p. 365-366), and merges according to that pattern. */ -static struct external_sort * -do_external_sort (struct sort_cases_pgm *scp, int separate) +static struct casefile * +do_external_sort (struct casereader *reader, + const struct sort_criteria *criteria) { struct external_sort *xsrt; - int success = 0; - if (vfm_source != NULL - && case_source_is_class (vfm_source, &storage_source_class)) - casefile_to_disk (storage_source_get_casefile (vfm_source)); + casefile_to_disk (casereader_get_casefile (reader)); xsrt = xmalloc (sizeof *xsrt); - xsrt->scp = scp; - if (!init_external_sort (xsrt)) - goto done; - if (!write_initial_runs (xsrt, separate)) - goto done; - if (!merge (xsrt)) - goto done; - - success = 1; - - done: - if (success) + xsrt->criteria = criteria; + xsrt->value_cnt = casefile_get_value_cnt (casereader_get_casefile (reader)); + xsrt->run_cap = 512; + xsrt->run_cnt = 0; + xsrt->initial_runs = xmalloc (sizeof *xsrt->initial_runs * xsrt->run_cap); + if (write_initial_runs (xsrt, reader) && merge (xsrt)) { - /* Don't destroy anything because we'll need it for reading - the output. */ - return xsrt; + struct casefile *output = xsrt->initial_runs[0]; + xsrt->initial_runs[0] = NULL; + destroy_external_sort (xsrt); + return output; } else { @@ -410,227 +398,11 @@ destroy_external_sort (struct external_sort *xsrt) int i; for (i = 0; i < xsrt->run_cnt; i++) - remove_temp_file (xsrt, xsrt->initial_runs[i].file_idx); - rmdir_temp_dir (xsrt); - free (xsrt->temp_dir); - free (xsrt->temp_name); + casefile_destroy (xsrt->initial_runs[i]); free (xsrt->initial_runs); free (xsrt); } } - -#ifdef HAVE_MKDTEMP -/* Creates and returns the name of a temporary directory. */ -static char * -make_temp_dir (void) -{ - const char *parent_dir; - char *temp_dir; - - if (getenv ("TMPDIR") != NULL) - parent_dir = getenv ("TMPDIR"); - else - parent_dir = P_tmpdir; - - temp_dir = xmalloc (strlen (parent_dir) + 32); - sprintf (temp_dir, "%s%cpsppXXXXXX", parent_dir, DIR_SEPARATOR); - if (mkdtemp (temp_dir) == NULL) - { - msg (SE, _("%s: Creating temporary directory: %s."), - temp_dir, strerror (errno)); - free (temp_dir); - return NULL; - } - else - return temp_dir; -} -#else /* !HAVE_MKDTEMP */ -/* Creates directory DIR. */ -static int -do_mkdir (const char *dir) -{ -#ifndef __MSDOS__ - return mkdir (dir, S_IRWXU); -#else - return mkdir (dir); -#endif -} - -/* Creates and returns the name of a temporary directory. */ -static char * -make_temp_dir (void) -{ - int i; - - for (i = 0; i < 100; i++) - { - char temp_dir[L_tmpnam + 1]; - if (tmpnam (temp_dir) == NULL) - { - msg (SE, _("Generating temporary directory name failed: %s."), - strerror (errno)); - return NULL; - } - else if (do_mkdir (temp_dir) == 0) - return xstrdup (temp_dir); - } - - msg (SE, _("Creating temporary directory failed: %s."), strerror (errno)); - return NULL; -} -#endif /* !HAVE_MKDTEMP */ - -/* Sets up to open temporary files. */ -static int -init_external_sort (struct external_sort *xsrt) -{ - /* Zero. */ - xsrt->temp_dir = NULL; - xsrt->next_file_idx = 0; - - /* Huffman queue. */ - xsrt->run_cap = 512; - xsrt->run_cnt = 0; - xsrt->initial_runs = xmalloc (sizeof *xsrt->initial_runs * xsrt->run_cap); - - /* Temporary directory. */ - xsrt->temp_dir = make_temp_dir (); - xsrt->temp_name = NULL; - if (xsrt->temp_dir == NULL) - return 0; - xsrt->temp_name = xmalloc (strlen (xsrt->temp_dir) + 64); - - return 1; -} - -/* Returns nonzero if we should return an error even though the - operation succeeded. Useful for testing. */ -static int -simulate_error (void) -{ - static int op_err_cnt = -1; - static int op_cnt; - - if (op_err_cnt == -1 || op_cnt++ < op_err_cnt) - return 0; - else - { - errno = 0; - return 1; - } -} - -/* Removes the directory created for temporary files, if one - exists. */ -static void -rmdir_temp_dir (struct external_sort *xsrt) -{ - if (xsrt->temp_dir != NULL && rmdir (xsrt->temp_dir) == -1) - { - msg (SW, _("%s: Error removing directory for temporary files: %s."), - xsrt->temp_dir, strerror (errno)); - xsrt->temp_dir = NULL; - } -} - -/* Returns the name of temporary file number FILE_IDX for XSRT. - The name is written into a static buffer, so be careful. */ -static char * -get_temp_file_name (struct external_sort *xsrt, int file_idx) -{ - assert (xsrt->temp_dir != NULL); - sprintf (xsrt->temp_name, "%s%c%04d", - xsrt->temp_dir, DIR_SEPARATOR, file_idx); - return xsrt->temp_name; -} - -/* Opens temporary file numbered FILE_IDX for XSRT with mode MODE - and returns the FILE *. */ -static FILE * -open_temp_file (struct external_sort *xsrt, int file_idx, const char *mode) -{ - char *temp_file; - FILE *file; - - temp_file = get_temp_file_name (xsrt, file_idx); - - file = fopen (temp_file, mode); - if (simulate_error () || file == NULL) - msg (SE, _("%s: Error opening temporary file for %s: %s."), - temp_file, mode[0] == 'r' ? "reading" : "writing", - strerror (errno)); - - return file; -} - -/* Closes FILE, which is the temporary file numbered FILE_IDX - under XSRT. Returns nonzero only if successful. */ -static int -close_temp_file (struct external_sort *xsrt, int file_idx, FILE *file) -{ - if (file != NULL) - { - char *temp_file = get_temp_file_name (xsrt, file_idx); - if (simulate_error () || fclose (file) == EOF) - { - msg (SE, _("%s: Error closing temporary file: %s."), - temp_file, strerror (errno)); - return 0; - } - } - return 1; -} - -/* Delete temporary file numbered FILE_IDX for XSRT. */ -static void -remove_temp_file (struct external_sort *xsrt, int file_idx) -{ - if (file_idx != -1) - { - char *temp_file = get_temp_file_name (xsrt, file_idx); - if (simulate_error () || remove (temp_file) != 0) - msg (SW, _("%s: Error removing temporary file: %s."), - temp_file, strerror (errno)); - } -} - -/* Writes SIZE bytes from buffer DATA into FILE, which is - temporary file numbered FILE_IDX from XSRT. */ -static int -write_temp_file (struct external_sort *xsrt, int file_idx, - FILE *file, const void *data, size_t size) -{ - if (!simulate_error () && fwrite (data, size, 1, file) == 1) - return 1; - else - { - char *temp_file = get_temp_file_name (xsrt, file_idx); - msg (SE, _("%s: Error writing temporary file: %s."), - temp_file, strerror (errno)); - return 0; - } -} - -/* Reads SIZE bytes into buffer DATA into FILE, which is - temporary file numbered FILE_IDX from XSRT. */ -static int -read_temp_file (struct external_sort *xsrt, int file_idx, - FILE *file, void *data, size_t size) -{ - if (!simulate_error () && fread (data, size, 1, file) == 1) - return 1; - else - { - char *temp_file = get_temp_file_name (xsrt, file_idx); - if (ferror (file)) - msg (SE, _("%s: Error reading temporary file: %s."), - temp_file, strerror (errno)); - else - msg (SE, _("%s: Unexpected end of temporary file."), - temp_file); - return 0; - } -} /* Replacement selection. */ @@ -638,7 +410,7 @@ read_temp_file (struct external_sort *xsrt, int file_idx, struct record_run { int run; /* Run number of case. */ - struct case_list *record; /* Case data. */ + struct ccase record; /* Case data. */ }; /* Represents a set of initial runs during an external sort. */ @@ -646,43 +418,39 @@ struct initial_run_state { struct external_sort *xsrt; - int *idx_to_fv; /* Translation table copied from sink. */ - /* Reservoir. */ struct record_run *records; /* Records arranged as a heap. */ size_t record_cnt; /* Current number of records. */ size_t record_cap; /* Capacity for records. */ - struct case_list *free_list;/* Cases not in heap. */ /* Run currently being output. */ - int file_idx; /* Temporary file number. */ + int run; /* Run number. */ size_t case_cnt; /* Number of cases so far. */ - FILE *output_file; /* Output file. */ - struct case_list *last_output;/* Record last output. */ + struct casefile *casefile; /* Output file. */ + struct ccase last_output; /* Record last output. */ int okay; /* Zero if an error has been encountered. */ }; static const struct case_sink_class sort_sink_class; -static void destroy_initial_run_state (struct initial_run_state *irs); +static void destroy_initial_run_state (struct initial_run_state *); +static void process_case (struct initial_run_state *, const struct ccase *); static int allocate_cases (struct initial_run_state *); -static struct case_list *grab_case (struct initial_run_state *); -static void release_case (struct initial_run_state *, struct case_list *); -static void output_record (struct initial_run_state *irs); -static void start_run (struct initial_run_state *irs); -static void end_run (struct initial_run_state *irs); +static void output_record (struct initial_run_state *); +static void start_run (struct initial_run_state *); +static void end_run (struct initial_run_state *); static int compare_record_run (const struct record_run *, const struct record_run *, struct initial_run_state *); static int compare_record_run_minheap (const void *, const void *, void *); -/* Writes initial runs for XSRT, sending them to a separate file - if SEPARATE is nonzero. */ +/* Reads cases from READER and composes initial runs in XSRT. */ static int -write_initial_runs (struct external_sort *xsrt, int separate) +write_initial_runs (struct external_sort *xsrt, struct casereader *reader) { struct initial_run_state *irs; + struct ccase c; int success = 0; /* Allocate memory for cases. */ @@ -690,29 +458,19 @@ write_initial_runs (struct external_sort *xsrt, int separate) irs->xsrt = xsrt; irs->records = NULL; irs->record_cnt = irs->record_cap = 0; - irs->free_list = NULL; - irs->output_file = NULL; - irs->last_output = NULL; - irs->file_idx = 0; + irs->run = 0; irs->case_cnt = 0; + irs->casefile = NULL; + case_nullify (&irs->last_output); irs->okay = 1; if (!allocate_cases (irs)) goto done; - /* Create case sink. */ - if (!separate) - { - if (vfm_sink != NULL && vfm_sink->class->destroy != NULL) - vfm_sink->class->destroy (vfm_sink); - vfm_sink = create_case_sink (&sort_sink_class, default_dict, irs); - xsrt->scp->ref_cnt++; - } - /* Create initial runs. */ start_run (irs); - procedure (NULL, NULL); - irs->idx_to_fv = NULL; - while (irs->record_cnt > 0 && irs->okay) + for (; irs->okay && casereader_read (reader, &c); case_destroy (&c)) + process_case (irs, &c); + while (irs->okay && irs->record_cnt > 0) output_record (irs); end_run (irs); @@ -726,31 +484,23 @@ write_initial_runs (struct external_sort *xsrt, int separate) /* Add a single case to an initial run. */ static void -sort_sink_write (struct case_sink *sink, const struct ccase *c) +process_case (struct initial_run_state *irs, const struct ccase *c) { - struct initial_run_state *irs = sink->aux; struct record_run *new_record_run; - if (!irs->okay) - return; - - irs->idx_to_fv = sink->idx_to_fv; - /* Compose record_run for this run and add to heap. */ - assert (irs->record_cnt < irs->record_cap); + assert (irs->record_cnt < irs->record_cap - 1); new_record_run = irs->records + irs->record_cnt++; - new_record_run->record = grab_case (irs); - memcpy (new_record_run->record->c.data, c->data, irs->xsrt->scp->case_size); - new_record_run->run = irs->file_idx; - if (irs->last_output != NULL - && compare_record (c->data, irs->last_output->c.data, - irs->xsrt->scp, sink->idx_to_fv) < 0) - new_record_run->run = irs->xsrt->next_file_idx; + case_copy (&new_record_run->record, 0, c, 0, irs->xsrt->value_cnt); + new_record_run->run = irs->run; + if (!case_is_null (&irs->last_output) + && compare_record (c, &irs->last_output, irs->xsrt->criteria) < 0) + new_record_run->run = irs->run + 1; push_heap (irs->records, irs->record_cnt, sizeof *irs->records, compare_record_run_minheap, irs); /* Output a record if the reservoir is full. */ - if (irs->record_cnt == irs->record_cap && irs->okay) + if (irs->record_cnt == irs->record_cap - 1 && irs->okay) output_record (irs); } @@ -758,28 +508,17 @@ sort_sink_write (struct case_sink *sink, const struct ccase *c) static void destroy_initial_run_state (struct initial_run_state *irs) { - struct case_list *iter, *next; int i; if (irs == NULL) return; - /* Release cases to free list. */ - for (i = 0; i < irs->record_cnt; i++) - release_case (irs, irs->records[i].record); - if (irs->last_output != NULL) - release_case (irs, irs->last_output); - - /* Free cases in free list. */ - for (iter = irs->free_list; iter != NULL; iter = next) - { - next = iter->next; - free (iter); - } - + for (i = 0; i < irs->record_cap; i++) + case_destroy (&irs->records[i].record); free (irs->records); - if (irs->output_file != NULL) - close_temp_file (irs->xsrt, irs->file_idx, irs->output_file); + + if (irs->casefile != NULL) + casefile_sleep (irs->casefile); free (irs); } @@ -795,28 +534,17 @@ allocate_cases (struct initial_run_state *irs) /* Allocate as many cases as we can within the workspace limit. */ approx_case_cost = (sizeof *irs->records - + sizeof *irs->free_list - + irs->xsrt->scp->case_size + + irs->xsrt->value_cnt * sizeof (union value) + 4 * sizeof (void *)); max_cases = get_max_workspace() / approx_case_cost; irs->records = malloc (sizeof *irs->records * max_cases); for (i = 0; i < max_cases; i++) - { - struct case_list *c; - c = malloc (sizeof *c - + irs->xsrt->scp->case_size - - sizeof (union value)); - if (c == NULL) - { - max_cases = i; - break; - } - release_case (irs, c); - } - - /* irs->records gets all but one of the allocated cases. - The extra is used for last_output. */ - irs->record_cap = max_cases - 1; + if (!case_try_create (&irs->records[i].record, irs->xsrt->value_cnt)) + { + max_cases = i; + break; + } + irs->record_cap = max_cases; /* Fail if we didn't allocate an acceptable number of cases. */ if (irs->records == NULL || max_cases < MIN_BUFFER_TOTAL_SIZE_RECS) @@ -833,42 +561,31 @@ allocate_cases (struct initial_run_state *irs) /* Compares the VAR_CNT variables in VARS[] between the `value's at A and B, and returns a strcmp()-type result. */ static int -compare_record (const union value *a, const union value *b, - const struct sort_cases_pgm *scp, - int *idx_to_fv) +compare_record (const struct ccase *a, const struct ccase *b, + const struct sort_criteria *criteria) { int i; assert (a != NULL); assert (b != NULL); - for (i = 0; i < scp->var_cnt; i++) + for (i = 0; i < criteria->crit_cnt; i++) { - struct variable *v = scp->vars[i]; - int fv; + const struct sort_criterion *c = &criteria->crits[i]; int result; - - if (idx_to_fv != NULL) - fv = idx_to_fv[v->index]; - else - fv = v->fv; - if (v->type == NUMERIC) + if (c->width == 0) { - double af = a[fv].f; - double bf = b[fv].f; + double af = case_num (a, c->fv); + double bf = case_num (b, c->fv); result = af < bf ? -1 : af > bf; } else - result = memcmp (a[fv].s, b[fv].s, v->width); + result = memcmp (case_str (a, c->fv), case_str (b, c->fv), c->width); - if (result != 0) - { - if (scp->dirs[i] == SRT_DESCEND) - result = -result; - return result; - } + if (result != 0) + return c->dir == SRT_ASCEND ? result : -result; } return 0; @@ -884,8 +601,7 @@ compare_record_run (const struct record_run *a, if (a->run != b->run) return a->run > b->run ? 1 : -1; else - return compare_record (a->record->c.data, b->record->c.data, - irs->xsrt->scp, irs->idx_to_fv); + return compare_record (&a->record, &b->record, irs->xsrt->criteria); } /* Compares record-run tuples A and B on run number first, then @@ -901,16 +617,11 @@ compare_record_run_minheap (const void *a, const void *b, void *irs) static void start_run (struct initial_run_state *irs) { - irs->file_idx = irs->xsrt->next_file_idx++; + irs->run++; irs->case_cnt = 0; - irs->output_file = open_temp_file (irs->xsrt, irs->file_idx, "wb"); - if (irs->output_file == NULL) - irs->okay = 0; - if (irs->last_output != NULL) - { - release_case (irs, irs->last_output); - irs->last_output = NULL; - } + irs->casefile = casefile_create (irs->xsrt->value_cnt); + casefile_to_disk (irs->casefile); + case_nullify (&irs->last_output); } /* Ends the current initial run. */ @@ -918,24 +629,20 @@ static void end_run (struct initial_run_state *irs) { struct external_sort *xsrt = irs->xsrt; - + /* Record initial run. */ - if (xsrt->run_cnt >= xsrt->run_cap) + if (irs->casefile != NULL) { - xsrt->run_cap *= 2; - xsrt->initial_runs - = xrealloc (xsrt->initial_runs, - sizeof *xsrt->initial_runs * xsrt->run_cap); + if (xsrt->run_cnt >= xsrt->run_cap) + { + xsrt->run_cap *= 2; + xsrt->initial_runs + = xrealloc (xsrt->initial_runs, + sizeof *xsrt->initial_runs * xsrt->run_cap); + } + xsrt->initial_runs[xsrt->run_cnt++] = irs->casefile; + irs->casefile = NULL; } - xsrt->initial_runs[xsrt->run_cnt].file_idx = irs->file_idx; - xsrt->initial_runs[xsrt->run_cnt].case_cnt = irs->case_cnt; - xsrt->run_cnt++; - - /* Close file handle. */ - if (irs->output_file != NULL - && !close_temp_file (irs->xsrt, irs->file_idx, irs->output_file)) - irs->okay = 0; - irs->output_file = NULL; } /* Writes a record to the current initial run. */ @@ -943,6 +650,7 @@ static void output_record (struct initial_run_state *irs) { struct record_run *record_run; + struct ccase case_tmp; /* Extract minimum case from heap. */ assert (irs->record_cnt > 0); @@ -955,52 +663,24 @@ output_record (struct initial_run_state *irs) return; /* Start new run if necessary. */ - assert (record_run->run == irs->file_idx - || record_run->run == irs->xsrt->next_file_idx); - if (record_run->run != irs->file_idx) + assert (record_run->run == irs->run + || record_run->run == irs->run + 1); + if (record_run->run != irs->run) { end_run (irs); start_run (irs); } - assert (record_run->run == irs->file_idx); + assert (record_run->run == irs->run); irs->case_cnt++; /* Write to disk. */ - if (irs->output_file != NULL - && !write_temp_file (irs->xsrt, irs->file_idx, irs->output_file, - &record_run->record->c, irs->xsrt->scp->case_size)) - irs->okay = 0; + if (irs->casefile != NULL) + casefile_append (irs->casefile, &record_run->record); /* This record becomes last_output. */ - if (irs->last_output != NULL) - release_case (irs, irs->last_output); - irs->last_output = record_run->record; -} - -/* Gets a case from the free list in IRS. It is an error to call - this function if the free list is empty. */ -static struct case_list * -grab_case (struct initial_run_state *irs) -{ - struct case_list *c; - - assert (irs != NULL); - assert (irs->free_list != NULL); - - c = irs->free_list; - irs->free_list = c->next; - return c; -} - -/* Returns C to the free list in IRS. */ -static void -release_case (struct initial_run_state *irs, struct case_list *c) -{ - assert (irs != NULL); - assert (c != NULL); - - c->next = irs->free_list; - irs->free_list = c; + irs->last_output = case_tmp = record_run->record; + record_run->record = irs->records[irs->record_cap - 1].record; + irs->records[irs->record_cap - 1].record = case_tmp; } /* Merging. */ @@ -1009,19 +689,16 @@ release_case (struct initial_run_state *irs, struct case_list *c) struct merge_state { struct external_sort *xsrt; /* External sort state. */ - struct ccase **cases; /* Buffers. */ + struct ccase *cases; /* Buffers. */ size_t case_cnt; /* Number of buffers. */ }; struct run; -static int merge_once (struct merge_state *, - const struct initial_run[], size_t, - struct initial_run *); -static int fill_run_buffer (struct merge_state *, struct run *); +static struct casefile *merge_once (struct merge_state *, + struct casefile *[], size_t); static int mod (int, int); -/* Performs a series of P-way merges of initial runs - method. */ +/* Performs a series of P-way merges of initial runs. */ static int merge (struct external_sort *xsrt) { @@ -1036,20 +713,18 @@ merge (struct external_sort *xsrt) /* Allocate as many cases as possible into cases. */ approx_case_cost = (sizeof *mrg.cases - + xsrt->scp->case_size + 4 * sizeof (void *)); + + xsrt->value_cnt * sizeof (union value) + + 4 * sizeof (void *)); mrg.case_cnt = get_max_workspace() / approx_case_cost; mrg.cases = malloc (sizeof *mrg.cases * mrg.case_cnt); if (mrg.cases == NULL) goto done; for (i = 0; i < mrg.case_cnt; i++) - { - mrg.cases[i] = malloc (xsrt->scp->case_size); - if (mrg.cases[i] == NULL) - { - mrg.case_cnt = i; - break; - } - } + if (!case_try_create (&mrg.cases[i], xsrt->value_cnt)) + { + mrg.case_cnt = i; + break; + } if (mrg.case_cnt < MIN_BUFFER_TOTAL_SIZE_RECS) { msg (SE, _("Out of memory. Could not allocate room for minimum of %d " @@ -1063,9 +738,10 @@ merge (struct external_sort *xsrt) max_order = MAX_MERGE_ORDER; if (mrg.case_cnt / max_order < MIN_BUFFER_SIZE_RECS) max_order = mrg.case_cnt / MIN_BUFFER_SIZE_RECS; - else if (mrg.case_cnt / max_order * xsrt->scp->case_size + else if (mrg.case_cnt / max_order * xsrt->value_cnt * sizeof (union value) < MIN_BUFFER_SIZE_BYTES) - max_order = mrg.case_cnt / (MIN_BUFFER_SIZE_BYTES / xsrt->scp->case_size); + max_order = mrg.case_cnt / (MIN_BUFFER_SIZE_BYTES + / (xsrt->value_cnt * sizeof (union value))); if (max_order < 2) max_order = 2; if (max_order > xsrt->run_cnt) @@ -1080,7 +756,7 @@ merge (struct external_sort *xsrt) || (xsrt->run_cnt + dummy_run_cnt) % (max_order - 1) == 1); while (xsrt->run_cnt > 1) { - struct initial_run output_run; + struct casefile *output_run; int order; int i; @@ -1096,10 +772,11 @@ merge (struct external_sort *xsrt) compare_initial_runs, NULL); /* Merge runs. */ - if (!merge_once (&mrg, xsrt->initial_runs + xsrt->run_cnt, order, - &output_run)) + output_run = merge_once (&mrg, + xsrt->initial_runs + xsrt->run_cnt, order); + if (output_run == NULL) goto done; - + /* Add output run to heap. */ xsrt->initial_runs[xsrt->run_cnt++] = output_run; push_heap (xsrt->initial_runs, xsrt->run_cnt, sizeof *xsrt->initial_runs, @@ -1114,7 +791,7 @@ merge (struct external_sort *xsrt) done: for (i = 0; i < mrg.case_cnt; i++) - free (mrg.cases[i]); + case_destroy (&mrg.cases[i]); free (mrg.cases); return success; @@ -1133,291 +810,65 @@ mod (int x, int y) else if (x < 0 && y > 0) return y - (-x) % y; - assert (0); abort (); } -/* A run of data for use in merging. */ -struct run - { - FILE *file; /* File that contains run. */ - int file_idx; /* Index of file that contains run. */ - struct ccase **buffer; /* Case buffer. */ - struct ccase **buffer_head; /* First unconsumed case in buffer. */ - struct ccase **buffer_tail; /* One past last unconsumed case in buffer. */ - size_t buffer_cap; /* Number of cases buffer can hold. */ - size_t unread_case_cnt; /* Number of cases not yet read. */ - }; - /* Merges the RUN_CNT initial runs specified in INPUT_RUNS into a new run. Returns nonzero only if successful. Adds an entry to MRG->xsrt->runs for the output file if and only if the output file is actually created. Always deletes all the input files. */ -static int +static struct casefile * merge_once (struct merge_state *mrg, - const struct initial_run input_runs[], - size_t run_cnt, - struct initial_run *output_run) + struct casefile *input_runs[], + size_t run_cnt) { - struct run runs[MAX_MERGE_ORDER]; - FILE *output_file = NULL; - int success = 0; + struct casereader *input_readers[MAX_MERGE_ORDER]; + struct ccase input_cases[MAX_MERGE_ORDER]; + struct casefile *output_casefile = NULL; int i; - /* Initialize runs[]. */ - for (i = 0; i < run_cnt; i++) - { - runs[i].file = NULL; - runs[i].file_idx = input_runs[i].file_idx; - runs[i].buffer = mrg->cases + mrg->case_cnt / run_cnt * i; - runs[i].buffer_head = runs[i].buffer; - runs[i].buffer_tail = runs[i].buffer; - runs[i].buffer_cap = mrg->case_cnt / run_cnt; - runs[i].unread_case_cnt = input_runs[i].case_cnt; - } - - /* Open input files. */ for (i = 0; i < run_cnt; i++) { - runs[i].file = open_temp_file (mrg->xsrt, runs[i].file_idx, "rb"); - if (runs[i].file == NULL) - goto error; + input_readers[i] = casefile_get_reader (input_runs[i]); + if (!casereader_read_xfer (input_readers[i], &input_cases[i])) + { + run_cnt--; + i--; + } } - /* Create output file and count cases to be output. */ - output_run->file_idx = mrg->xsrt->next_file_idx++; - output_run->case_cnt = 0; - for (i = 0; i < run_cnt; i++) - output_run->case_cnt += input_runs[i].case_cnt; - output_file = open_temp_file (mrg->xsrt, output_run->file_idx, "wb"); - if (output_file == NULL) - goto error; - - /* Prime buffers. */ - for (i = 0; i < run_cnt; i++) - if (!fill_run_buffer (mrg, runs + i)) - goto error; + output_casefile = casefile_create (mrg->xsrt->value_cnt); + casefile_to_disk (output_casefile); /* Merge. */ while (run_cnt > 0) { - struct run *min_run; + size_t min_idx; /* Find minimum. */ - min_run = runs; + min_idx = 0; for (i = 1; i < run_cnt; i++) - if (compare_record ((*runs[i].buffer_head)->data, - (*min_run->buffer_head)->data, - mrg->xsrt->scp, NULL) < 0) - min_run = runs + i; + if (compare_record (&input_cases[i], &input_cases[min_idx], + mrg->xsrt->criteria) < 0) + min_idx = i; /* Write minimum to output file. */ - if (!write_temp_file (mrg->xsrt, min_run->file_idx, output_file, - (*min_run->buffer_head)->data, - mrg->xsrt->scp->case_size)) - goto error; + casefile_append_xfer (output_casefile, &input_cases[i]); - /* Remove case from buffer. */ - if (++min_run->buffer_head >= min_run->buffer_tail) + if (!casereader_read_xfer (input_readers[i], &input_cases[i])) { - /* Buffer is empty. Fill from file. */ - if (!fill_run_buffer (mrg, min_run)) - goto error; - - /* If buffer is still empty, delete its run. */ - if (min_run->buffer_head >= min_run->buffer_tail) - { - close_temp_file (mrg->xsrt, min_run->file_idx, min_run->file); - remove_temp_file (mrg->xsrt, min_run->file_idx); - *min_run = runs[--run_cnt]; - - /* We could donate the now-unused buffer space to - other runs. */ - } - } - } - - /* Close output file. */ - close_temp_file (mrg->xsrt, output_run->file_idx, output_file); - - return 1; - - error: - /* Close and remove output file. */ - if (output_file != NULL) - { - close_temp_file (mrg->xsrt, output_run->file_idx, output_file); - remove_temp_file (mrg->xsrt, output_run->file_idx); - } - - /* Close and remove any remaining input runs. */ - for (i = 0; i < run_cnt; i++) - { - close_temp_file (mrg->xsrt, runs[i].file_idx, runs[i].file); - remove_temp_file (mrg->xsrt, runs[i].file_idx); - } - - return success; -} - -/* Reads as many cases as possible into RUN's buffer. - Reads nonzero unless a disk error occurs. */ -static int -fill_run_buffer (struct merge_state *mrg, struct run *run) -{ - run->buffer_head = run->buffer_tail = run->buffer; - while (run->unread_case_cnt > 0 - && run->buffer_tail < run->buffer + run->buffer_cap) - { - if (!read_temp_file (mrg->xsrt, run->file_idx, run->file, - (*run->buffer_tail)->data, - mrg->xsrt->scp->case_size)) - return 0; - - run->unread_case_cnt--; - run->buffer_tail++; - } - - return 1; -} - -static struct case_source * -sort_sink_make_source (struct case_sink *sink) -{ - struct initial_run_state *irs = sink->aux; - - return create_case_source (&sort_source_class, default_dict, - irs->xsrt->scp); -} - -static const struct case_sink_class sort_sink_class = - { - "SORT CASES", - NULL, - sort_sink_write, - NULL, - sort_sink_make_source, - }; - -struct sort_source_aux - { - struct sort_cases_pgm *scp; - struct ccase *dst; - write_case_func *write_case; - write_case_data wc_data; - }; - -/* Passes C to the write_case function. */ -static int -sort_source_read_helper (const struct ccase *src, void *aux_) -{ - struct sort_source_aux *aux = aux_; - - memcpy (aux->dst, src, aux->scp->case_size); - return aux->write_case (aux->wc_data); -} - -/* Reads all the records from the source stream and passes them - to write_case(). */ -static void -sort_source_read (struct case_source *source, - struct ccase *c, - write_case_func *write_case, write_case_data wc_data) -{ - struct sort_cases_pgm *scp = source->aux; - struct sort_source_aux aux; - - aux.scp = scp; - aux.dst = c; - aux.write_case = write_case; - aux.wc_data = wc_data; - - read_sort_output (scp, sort_source_read_helper, &aux); -} - -static void read_internal_sort_output (struct internal_sort *isrt, - read_sort_output_func *, void *aux); -static void read_external_sort_output (struct external_sort *xsrt, - read_sort_output_func *, void *aux); - -/* Reads all the records from the output stream and passes them to the - function provided, which must have an interface identical to - write_case(). */ -void -read_sort_output (struct sort_cases_pgm *scp, - read_sort_output_func *output_func, void *aux) -{ - assert ((scp->isrt != NULL) + (scp->xsrt != NULL) <= 1); - if (scp->isrt != NULL) - read_internal_sort_output (scp->isrt, output_func, aux); - else if (scp->xsrt != NULL) - read_external_sort_output (scp->xsrt, output_func, aux); - else - { - /* No results. Probably an external sort that failed. */ - } -} - -static void -read_internal_sort_output (struct internal_sort *isrt, - read_sort_output_func *output_func, - void *aux) -{ - size_t case_idx; - - for (case_idx = 0; case_idx < isrt->case_cnt; case_idx++) - if (!output_func (isrt->cases[case_idx], aux)) - break; -} + casereader_destroy (input_readers[i]); + casefile_destroy (input_runs[i]); -static void -read_external_sort_output (struct external_sort *xsrt, - read_sort_output_func *output_func, void *aux) -{ - FILE *file; - int file_idx; - size_t i; - struct ccase *c; - - assert (xsrt->run_cnt == 1); - file_idx = xsrt->initial_runs[0].file_idx; - - file = open_temp_file (xsrt, file_idx, "rb"); - if (file == NULL) - { - err_failure (); - return; + run_cnt--; + input_runs[i] = input_runs[run_cnt--]; + input_readers[i] = input_readers[run_cnt--]; + input_cases[i] = input_cases[run_cnt--]; + } } - c = xmalloc (xsrt->scp->case_size); - for (i = 0; i < xsrt->initial_runs[0].case_cnt; i++) - { - if (!read_temp_file (xsrt, file_idx, file, c, xsrt->scp->case_size)) - { - err_failure (); - break; - } - - if (!output_func (c, aux)) - break; - } - free (c); - close_temp_file (xsrt, file_idx, file); -} + casefile_sleep (output_casefile); -static void -sort_source_destroy (struct case_source *source) -{ - struct sort_cases_pgm *scp = source->aux; - - destroy_sort_cases_pgm (scp); + return output_casefile; } - -const struct case_source_class sort_source_class = - { - "SORT CASES", - NULL, /* FIXME */ - sort_source_read, - sort_source_destroy, - }; diff --git a/src/sort.h b/src/sort.h index 4f8659e6..63054413 100644 --- a/src/sort.h +++ b/src/sort.h @@ -20,37 +20,19 @@ #if !sort_h #define sort_h 1 -#include "vfm.h" - -/* Sort direction. */ -enum sort_direction - { - SRT_ASCEND, /* A, B, C, ..., X, Y, Z. */ - SRT_DESCEND /* Z, Y, X, ..., C, B, A. */ - }; - -/* SORT CASES input program. */ -struct sort_cases_pgm - { - int ref_cnt; /* Reference count. */ - - struct variable **vars; /* Variables to sort. */ - enum sort_direction *dirs; /* Sort directions. */ - int var_cnt; /* Number of variables to sort. */ - - struct internal_sort *isrt; /* Internal sort output. */ - struct external_sort *xsrt; /* External sort output. */ - size_t case_size; /* Number of bytes in case. */ - }; - -/* SORT CASES programmatic interface. */ - -typedef int read_sort_output_func (const struct ccase *, void *aux); - -struct sort_cases_pgm *parse_sort (void); -int sort_cases (struct sort_cases_pgm *, int separate); -void read_sort_output (struct sort_cases_pgm *, - read_sort_output_func, void *aux); -void destroy_sort_cases_pgm (struct sort_cases_pgm *); +#include + +struct casereader; +struct dictionary; +struct variable; + +struct sort_criteria *sort_parse_criteria (const struct dictionary *, + struct variable ***, int *); +void sort_destroy_criteria (struct sort_criteria *); + +struct casefile *sort_execute (struct casereader *, + const struct sort_criteria *); +int sort_active_file_in_place (const struct sort_criteria *); +struct casefile *sort_active_file_to_casefile (const struct sort_criteria *); #endif /* !sort_h */ diff --git a/src/t-test.q b/src/t-test.q index 376477c3..e8da73a9 100644 --- a/src/t-test.q +++ b/src/t-test.q @@ -27,6 +27,7 @@ #include #include "alloc.h" #include "str.h" +#include "case.h" #include "command.h" #include "lexer.h" #include "error.h" @@ -514,6 +515,7 @@ tts_custom_pairs (struct cmd_t_test *cmd UNUSED) n_pairs+=n_pairs_local; + free (vars); return 1; } @@ -629,7 +631,7 @@ ssbox_one_sample_init(struct ssbox *this, ssbox_base_init(this, hsize,vsize); tab_title (this->t, 0, _("One-Sample Statistics")); - tab_vline(this->t, TAL_2, 1,0,vsize); + tab_vline(this->t, TAL_2, 1,0,vsize - 1); tab_text (this->t, 1, 0, TAB_CENTER | TAT_TITLE, _("N")); tab_text (this->t, 2, 0, TAB_CENTER | TAT_TITLE, _("Mean")); tab_text (this->t, 3, 0, TAB_CENTER | TAT_TITLE, _("Std. Deviation")); @@ -651,7 +653,7 @@ ssbox_independent_samples_init(struct ssbox *this, ssbox_base_init(this, hsize,vsize); tab_title (this->t, 0, _("Group Statistics")); - tab_vline(this->t,0,1,0,vsize); + tab_vline(this->t,0,1,0,vsize - 1); tab_text (this->t, 1, 0, TAB_CENTER | TAT_TITLE, indep_var->name); tab_text (this->t, 2, 0, TAB_CENTER | TAT_TITLE, _("N")); tab_text (this->t, 3, 0, TAB_CENTER | TAT_TITLE, _("Mean")); @@ -1047,7 +1049,7 @@ trbox_paired_init(struct trbox *self, trbox_base_init(self,n_pairs,hsize); tab_title (self->t, 0, _("Paired Samples Test")); tab_hline(self->t,TAL_1,2,6,1); - tab_vline(self->t,TAL_2,2,0,vsize); + tab_vline(self->t,TAL_2,2,0,vsize - 1); tab_joint_text(self->t,2,0,6,0,TAB_CENTER,_("Paired Differences")); tab_box(self->t,-1,-1,-1,TAL_1, 2,1,6,vsize-1); tab_box(self->t,-1,-1,-1,TAL_1, 6,0,hsize-1,vsize-1); @@ -1141,7 +1143,7 @@ trbox_one_sample_init(struct trbox *self, struct cmd_t_test *cmd ) trbox_base_init(self, cmd->n_variables,hsize); tab_title (self->t, 0, _("One-Sample Test")); tab_hline(self->t, TAL_1, 1, hsize - 1, 1); - tab_vline(self->t, TAL_2, 1, 0, vsize); + tab_vline(self->t, TAL_2, 1, 0, vsize - 1); tab_joint_text(self->t, 1, 0, hsize-1,0, TAB_CENTER | TAT_PRINTF, _("Test Value = %f"),cmd->n_testval); @@ -1314,7 +1316,7 @@ common_calc (const struct ccase *c, void *_cmd) for(i=0; i< cmd->n_variables ; ++i) { struct variable *v = cmd->v_variables[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); if (value_is_missing(val,v) ) { @@ -1326,7 +1328,7 @@ common_calc (const struct ccase *c, void *_cmd) /* Listwise has to be implicit if the independent variable is missing ?? */ if ( cmd->sbc_groups ) { - const union value *gv = &c->data[indep_var->fv]; + const union value *gv = case_data (c, indep_var->fv); if ( value_is_missing(gv,indep_var) ) { return 0; @@ -1338,7 +1340,7 @@ common_calc (const struct ccase *c, void *_cmd) { struct group_statistics *gs; struct variable *v = cmd->v_variables[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); gs= &cmd->v_variables[i]->p.t_t.ugs; @@ -1413,7 +1415,7 @@ one_sample_calc (const struct ccase *c, void *cmd_) for(i=0; i< cmd->n_variables ; ++i) { struct variable *v = cmd->v_variables[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); if (value_is_missing(val,v) ) { @@ -1426,7 +1428,7 @@ one_sample_calc (const struct ccase *c, void *cmd_) { struct group_statistics *gs; struct variable *v = cmd->v_variables[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); gs= &cmd->v_variables[i]->p.t_t.ugs; @@ -1525,8 +1527,8 @@ paired_calc (const struct ccase *c, void *cmd_) struct variable *v0 = pairs[i].v[0]; struct variable *v1 = pairs[i].v[1]; - const union value *val0 = &c->data[v0->fv]; - const union value *val1 = &c->data[v1->fv]; + const union value *val0 = case_data (c, v0->fv); + const union value *val1 = case_data (c, v1->fv); if ( value_is_missing(val0,v0) || value_is_missing(val1,v1) ) @@ -1541,8 +1543,8 @@ paired_calc (const struct ccase *c, void *cmd_) struct variable *v0 = pairs[i].v[0]; struct variable *v1 = pairs[i].v[1]; - const union value *val0 = &c->data[v0->fv]; - const union value *val1 = &c->data[v1->fv]; + const union value *val0 = case_data (c, v0->fv); + const union value *val1 = case_data (c, v1->fv); if ( ( !value_is_missing(val0,v0) && !value_is_missing(val1,v1) ) ) { @@ -1676,7 +1678,7 @@ group_calc (const struct ccase *c, struct cmd_t_test *cmd) int i; int g; - const union value *gv = &c->data[indep_var->fv]; + const union value *gv = case_data (c, indep_var->fv); const double weight = dict_get_case_weight(default_dict,c,&bad_weight_warn); @@ -1690,7 +1692,7 @@ group_calc (const struct ccase *c, struct cmd_t_test *cmd) for(i=0; i< cmd->n_variables ; ++i) { struct variable *v = cmd->v_variables[i]; - const union value *val = &c->data[v->fv]; + const union value *val = case_data (c, v->fv); if (value_is_missing(val,v) ) { @@ -1700,7 +1702,7 @@ group_calc (const struct ccase *c, struct cmd_t_test *cmd) } - gv = &c->data[indep_var->fv]; + gv = case_data (c, indep_var->fv); g = get_group(gv,indep_var); @@ -1716,7 +1718,7 @@ group_calc (const struct ccase *c, struct cmd_t_test *cmd) struct group_statistics *gs = &var->p.t_t.gs[g]; - const union value *val=&c->data[var->fv]; + const union value *val = case_data (c, var->fv); if ( !value_is_missing(val,var) ) { @@ -1768,15 +1770,16 @@ calculate(const struct casefile *cf, void *cmd_) struct trbox test_results_box; struct casereader *r; - const struct ccase *c; + struct ccase c; struct cmd_t_test *cmd = (struct cmd_t_test *) cmd_; common_precalc(cmd); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - common_calc(c,cmd); + common_calc(&c,cmd); } casereader_destroy (r); common_postcalc(cmd); @@ -1786,9 +1789,10 @@ calculate(const struct casefile *cf, void *cmd_) case T_1_SAMPLE: one_sample_precalc(cmd); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - one_sample_calc(c,cmd); + one_sample_calc(&c,cmd); } casereader_destroy (r); one_sample_postcalc(cmd); @@ -1797,9 +1801,10 @@ calculate(const struct casefile *cf, void *cmd_) case T_PAIRED: paired_precalc(cmd); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - paired_calc(c,cmd); + paired_calc(&c,cmd); } casereader_destroy (r); paired_postcalc(cmd); @@ -1809,9 +1814,10 @@ calculate(const struct casefile *cf, void *cmd_) group_precalc(cmd); for(r = casefile_get_reader (cf); - casereader_read (r, &c) ; ) + casereader_read (r, &c) ; + case_destroy (&c)) { - group_calc(c,cmd); + group_calc(&c,cmd); } casereader_destroy (r); group_postcalc(cmd); diff --git a/src/tab.c b/src/tab.c index 084c5873..8afb8f0b 100644 --- a/src/tab.c +++ b/src/tab.c @@ -38,19 +38,6 @@ struct som_table_class tab_table_class; -#if DEBUGGING -#define DEFFIRST(NAME, LABEL) LABEL, -#define DEFTAB(NAME, LABEL) LABEL, -/* -static const char *tab_names[] = - { -#include "tab.def" - }; -*/ -#undef DEFFIRST -#undef DEFTAB -#endif - /* Creates a table with NC columns and NR rows. If REALLOCABLE is nonzero then the table's size can be increased later; otherwise, its size can only be reduced. */ diff --git a/src/var-labs.c b/src/var-labs.c index a62ec95e..87916977 100644 --- a/src/var-labs.c +++ b/src/var-labs.c @@ -29,10 +29,6 @@ #include "debug-print.h" -#if DEBUGGING -static void debug_print (void); -#endif - int cmd_variable_labels (void) { @@ -70,27 +66,5 @@ cmd_variable_labels (void) free (v); } while (token != '.'); -#if 0 && DEBUGGING - debug_print (); -#endif return CMD_SUCCESS; } - -#if 0 && DEBUGGING -static void -debug_print (void) -{ - int i; - - printf (_("Variable labels:\n")); - for (i = 0; i < nvar; i++) - { - printf (" %8s: ", var[i]->name); - if (var[i]->label) - printf ("`%s'", var[i]->label); - else - printf (_("(no variable label)")); - printf ("\n"); - } -} -#endif /* DEBUGGING */ diff --git a/src/var.h b/src/var.h index 6b826243..a07772a3 100644 --- a/src/var.h +++ b/src/var.h @@ -247,22 +247,6 @@ struct vector int cnt; /* Number of variables. */ }; -/* Cases. */ - -/* A single case. (This doesn't need to be a struct anymore, but it - remains so for hysterical raisins.) */ -struct ccase - { - union value data[1]; - }; - -/* Linked list of cases. */ -struct case_list - { - struct case_list *next; - struct ccase c; - }; - /* Dictionary. */ /* Complete dictionary state. */ @@ -300,6 +284,7 @@ int dict_rename_vars (struct dictionary *, struct variable **, char **new_names, size_t count, char **err_name); +struct ccase; struct variable *dict_get_weight (const struct dictionary *); double dict_get_case_weight (const struct dictionary *, const struct ccase *, int *); @@ -414,12 +399,14 @@ void cancel_transformations (void); struct var_set; -struct var_set *var_set_create_from_dict (struct dictionary *d); -struct var_set *var_set_create_from_array (struct variable **var, size_t); +struct var_set *var_set_create_from_dict (const struct dictionary *d); +struct var_set *var_set_create_from_array (struct variable *const *var, + size_t); -size_t var_set_get_cnt (struct var_set *vs); -struct variable *var_set_get_var (struct var_set *vs, size_t idx); -struct variable *var_set_lookup_var (struct var_set *vs, const char *name); +size_t var_set_get_cnt (const struct var_set *vs); +struct variable *var_set_get_var (const struct var_set *vs, size_t idx); +struct variable *var_set_lookup_var (const struct var_set *vs, + const char *name); void var_set_destroy (struct var_set *vs); /* Variable parsers. */ @@ -438,10 +425,10 @@ enum }; struct variable *parse_variable (void); -struct variable *parse_dict_variable (struct dictionary *); -int parse_variables (struct dictionary *, struct variable ***, int *, +struct variable *parse_dict_variable (const struct dictionary *); +int parse_variables (const struct dictionary *, struct variable ***, int *, int opts); -int parse_var_set_vars (struct var_set *, struct variable ***, int *, +int parse_var_set_vars (const struct var_set *, struct variable ***, int *, int opts); int parse_DATA_LIST_vars (char ***names, int *cnt, int opts); int parse_mixed_vars (char ***names, int *cnt, int opts); diff --git a/src/vars-atr.c b/src/vars-atr.c index c0730022..cf4ef4e3 100644 --- a/src/vars-atr.c +++ b/src/vars-atr.c @@ -57,9 +57,7 @@ discard_variables (void) if (vfm_source != NULL) { - if (vfm_source->class->destroy != NULL) - vfm_source->class->destroy (vfm_source); - free (vfm_source); + free_case_source (vfm_source); vfm_source = NULL; } diff --git a/src/vars-prs.c b/src/vars-prs.c index 21340ef0..91376a5a 100644 --- a/src/vars-prs.c +++ b/src/vars-prs.c @@ -33,7 +33,7 @@ if successful. On failure emits an error message and returns a null pointer. */ static struct variable * -parse_vs_variable (struct var_set *vs) +parse_vs_variable (const struct var_set *vs) { struct variable *vp; @@ -55,7 +55,7 @@ parse_vs_variable (struct var_set *vs) variable if successful. On failure emits an error message and returns a null pointer. */ struct variable * -parse_dict_variable (struct dictionary *d) +parse_dict_variable (const struct dictionary *d) { struct var_set *vs = var_set_create_from_dict (d); struct variable *var = parse_vs_variable (vs); @@ -113,8 +113,8 @@ dict_class_to_name (enum dict_class dict_class) number of variables into *CNT. Returns nonzero only if successful. */ int -parse_variables (struct dictionary *d, struct variable ***var, int *cnt, - int opts) +parse_variables (const struct dictionary *d, struct variable ***var, + int *cnt, int opts) { struct var_set *vs; int success; @@ -133,7 +133,7 @@ parse_variables (struct dictionary *d, struct variable ***var, int *cnt, Conversely, if parse_variables() returns non-zero, then *nv is nonzero and *v is non-NULL. */ int -parse_var_set_vars (struct var_set *vs, +parse_var_set_vars (const struct var_set *vs, struct variable ***v, int *nv, int pv_opts) { @@ -524,16 +524,16 @@ fail: /* A set of variables. */ struct var_set { - size_t (*get_cnt) (struct var_set *); - struct variable *(*get_var) (struct var_set *, size_t idx); - struct variable *(*lookup_var) (struct var_set *, const char *); + size_t (*get_cnt) (const struct var_set *); + struct variable *(*get_var) (const struct var_set *, size_t idx); + struct variable *(*lookup_var) (const struct var_set *, const char *); void (*destroy) (struct var_set *); void *aux; }; /* Returns the number of variables in VS. */ size_t -var_set_get_cnt (struct var_set *vs) +var_set_get_cnt (const struct var_set *vs) { assert (vs != NULL); @@ -543,7 +543,7 @@ var_set_get_cnt (struct var_set *vs) /* Return variable with index IDX in VS. IDX must be less than the number of variables in VS. */ struct variable * -var_set_get_var (struct var_set *vs, size_t idx) +var_set_get_var (const struct var_set *vs, size_t idx) { assert (vs != NULL); assert (idx < var_set_get_cnt (vs)); @@ -554,7 +554,7 @@ var_set_get_var (struct var_set *vs, size_t idx) /* Returns the variable in VS named NAME, or a null pointer if VS contains no variable with that name. */ struct variable * -var_set_lookup_var (struct var_set *vs, const char *name) +var_set_lookup_var (const struct var_set *vs, const char *name) { assert (vs != NULL); assert (name != NULL); @@ -573,7 +573,7 @@ var_set_destroy (struct var_set *vs) /* Returns the number of variables in VS. */ static size_t -dict_var_set_get_cnt (struct var_set *vs) +dict_var_set_get_cnt (const struct var_set *vs) { struct dictionary *d = vs->aux; @@ -583,7 +583,7 @@ dict_var_set_get_cnt (struct var_set *vs) /* Return variable with index IDX in VS. IDX must be less than the number of variables in VS. */ static struct variable * -dict_var_set_get_var (struct var_set *vs, size_t idx) +dict_var_set_get_var (const struct var_set *vs, size_t idx) { struct dictionary *d = vs->aux; @@ -593,7 +593,7 @@ dict_var_set_get_var (struct var_set *vs, size_t idx) /* Returns the variable in VS named NAME, or a null pointer if VS contains no variable with that name. */ static struct variable * -dict_var_set_lookup_var (struct var_set *vs, const char *name) +dict_var_set_lookup_var (const struct var_set *vs, const char *name) { struct dictionary *d = vs->aux; @@ -609,28 +609,28 @@ dict_var_set_destroy (struct var_set *vs) /* Returns a variable set based on D. */ struct var_set * -var_set_create_from_dict (struct dictionary *d) +var_set_create_from_dict (const struct dictionary *d) { struct var_set *vs = xmalloc (sizeof *vs); vs->get_cnt = dict_var_set_get_cnt; vs->get_var = dict_var_set_get_var; vs->lookup_var = dict_var_set_lookup_var; vs->destroy = dict_var_set_destroy; - vs->aux = d; + vs->aux = (void *) d; return vs; } /* A variable set based on an array. */ struct array_var_set { - struct variable **var; /* Array of variables. */ + struct variable *const *var;/* Array of variables. */ size_t var_cnt; /* Number of elements in var. */ struct hsh_table *name_tab; /* Hash from variable names to variables. */ }; /* Returns the number of variables in VS. */ static size_t -array_var_set_get_cnt (struct var_set *vs) +array_var_set_get_cnt (const struct var_set *vs) { struct array_var_set *avs = vs->aux; @@ -640,17 +640,17 @@ array_var_set_get_cnt (struct var_set *vs) /* Return variable with index IDX in VS. IDX must be less than the number of variables in VS. */ static struct variable * -array_var_set_get_var (struct var_set *vs, size_t idx) +array_var_set_get_var (const struct var_set *vs, size_t idx) { struct array_var_set *avs = vs->aux; - return avs->var[idx]; + return (struct variable *) avs->var[idx]; } /* Returns the variable in VS named NAME, or a null pointer if VS contains no variable with that name. */ static struct variable * -array_var_set_lookup_var (struct var_set *vs, const char *name) +array_var_set_lookup_var (const struct var_set *vs, const char *name) { struct array_var_set *avs = vs->aux; struct variable v; @@ -674,7 +674,7 @@ array_var_set_destroy (struct var_set *vs) /* Returns a variable set based on the VAR_CNT variables in VAR. */ struct var_set * -var_set_create_from_array (struct variable **var, size_t var_cnt) +var_set_create_from_array (struct variable *const *var, size_t var_cnt) { struct var_set *vs; struct array_var_set *avs; @@ -692,7 +692,7 @@ var_set_create_from_array (struct variable **var, size_t var_cnt) compare_variables, hash_variable, NULL, NULL); for (i = 0; i < var_cnt; i++) - if (hsh_insert (avs->name_tab, var[i]) != NULL) + if (hsh_insert (avs->name_tab, (void *) var[i]) != NULL) { var_set_destroy (vs); return NULL; diff --git a/src/vfm.c b/src/vfm.c index 0d42ff5c..1d3d2a5c 100644 --- a/src/vfm.c +++ b/src/vfm.c @@ -28,6 +28,7 @@ #include /* Required by SunOS4. */ #endif #include "alloc.h" +#include "case.h" #include "casefile.h" #include "do-ifP.h" #include "error.h" @@ -57,8 +58,8 @@ struct write_case_data int (*proc_func) (struct ccase *, void *); /* Function. */ void *aux; /* Auxiliary data. */ - struct ccase *trns_case; /* Case used for transformations. */ - struct ccase *sink_case; /* Case written to sink, if + struct ccase trns_case; /* Case used for transformations. */ + struct ccase sink_case; /* Case written to sink, if compaction is necessary. */ size_t cases_written; /* Cases output so far. */ size_t cases_analyzed; /* Cases passed to procedure so far. */ @@ -81,11 +82,11 @@ time_t last_vfm_invocation; int n_lag; /* Number of cases to lag. */ static int lag_count; /* Number of cases in lag_queue so far. */ static int lag_head; /* Index where next case will be added. */ -static struct ccase **lag_queue; /* Array of n_lag ccase * elements. */ +static struct ccase *lag_queue; /* Array of n_lag ccase * elements. */ static void internal_procedure (int (*proc_func) (struct ccase *, void *), void *aux); -static struct ccase *create_trns_case (struct dictionary *); +static void create_trns_case (struct ccase *, struct dictionary *); static void open_active_file (void); static int write_case (struct write_case_data *wc_data); static int execute_transformations (struct ccase *c, @@ -125,6 +126,16 @@ static void close_active_file (void); void procedure (int (*proc_func) (struct ccase *, void *), void *aux) { + if (proc_func == NULL + && case_source_is_class (vfm_source, &storage_source_class) + && vfm_sink == NULL + && !temporary + && n_trns == 0) + { + /* Nothing to do. */ + return; + } + open_active_file (); internal_procedure (proc_func, aux); close_active_file (); @@ -144,19 +155,19 @@ internal_procedure (int (*proc_func) (struct ccase *, void *), void *aux) wc_data.proc_func = proc_func; wc_data.aux = aux; - wc_data.trns_case = create_trns_case (default_dict); - wc_data.sink_case = xmalloc (dict_get_case_size (default_dict)); + create_trns_case (&wc_data.trns_case, default_dict); + case_create (&wc_data.sink_case, dict_get_next_value_idx (default_dict)); wc_data.cases_written = 0; last_vfm_invocation = time (NULL); if (vfm_source != NULL) vfm_source->class->read (vfm_source, - wc_data.trns_case, + &wc_data.trns_case, write_case, &wc_data); - free (wc_data.sink_case); - free (wc_data.trns_case); + case_destroy (&wc_data.sink_case); + case_destroy (&wc_data.trns_case); assert (--recursive_call == 0); } @@ -164,28 +175,23 @@ internal_procedure (int (*proc_func) (struct ccase *, void *), void *aux) /* Creates and returns a case, initializing it from the vectors that say which `value's need to be initialized just once, and which ones need to be re-initialized before every case. */ -static struct ccase * -create_trns_case (struct dictionary *dict) +static void +create_trns_case (struct ccase *trns_case, struct dictionary *dict) { - struct ccase *c = xmalloc (dict_get_case_size (dict)); size_t var_cnt = dict_get_var_cnt (dict); size_t i; + case_create (trns_case, dict_get_next_value_idx (dict)); for (i = 0; i < var_cnt; i++) { struct variable *v = dict_get_var (dict, i); + union value *value = case_data_rw (trns_case, v->fv); - if (v->type == NUMERIC) - { - if (v->reinit) - c->data[v->fv].f = 0.0; - else - c->data[v->fv].f = SYSMIS; - } + if (v->type == NUMERIC) + value->f = v->reinit ? 0.0 : SYSMIS; else - memset (c->data[v->fv].s, ' ', v->width); + memset (value->s, ' ', v->width); } - return c; } /* Makes all preparations for reading from the data source and writing @@ -220,7 +226,7 @@ open_active_file (void) lag_head = 0; lag_queue = xmalloc (n_lag * sizeof *lag_queue); for (i = 0; i < n_lag; i++) - lag_queue[i] = xmalloc (dict_get_case_size (temp_dict)); + case_nullify (&lag_queue[i]); } /* Close any unclosed DO IF or LOOP constructs. */ @@ -235,7 +241,7 @@ static int write_case (struct write_case_data *wc_data) { /* Execute permanent transformations. */ - if (!execute_transformations (wc_data->trns_case, t_trns, f_trns, temp_trns, + if (!execute_transformations (&wc_data->trns_case, t_trns, f_trns, temp_trns, wc_data->cases_written + 1)) goto done; @@ -247,27 +253,27 @@ write_case (struct write_case_data *wc_data) /* Write case to LAG queue. */ if (n_lag) - lag_case (wc_data->trns_case); + lag_case (&wc_data->trns_case); /* Write case to replacement active file. */ if (vfm_sink->class->write != NULL) { if (compaction_necessary) { - compact_case (wc_data->sink_case, wc_data->trns_case); - vfm_sink->class->write (vfm_sink, wc_data->sink_case); + compact_case (&wc_data->sink_case, &wc_data->trns_case); + vfm_sink->class->write (vfm_sink, &wc_data->sink_case); } else - vfm_sink->class->write (vfm_sink, wc_data->trns_case); + vfm_sink->class->write (vfm_sink, &wc_data->trns_case); } /* Execute temporary transformations. */ - if (!execute_transformations (wc_data->trns_case, t_trns, temp_trns, n_trns, + if (!execute_transformations (&wc_data->trns_case, t_trns, temp_trns, n_trns, wc_data->cases_written)) goto done; /* FILTER, PROCESS IF, post-TEMPORARY N OF CASES. */ - if (filter_case (wc_data->trns_case, wc_data->cases_written) + if (filter_case (&wc_data->trns_case, wc_data->cases_written) || (dict_get_case_limit (temp_dict) && wc_data->cases_analyzed >= dict_get_case_limit (temp_dict))) goto done; @@ -275,10 +281,10 @@ write_case (struct write_case_data *wc_data) /* Pass case to procedure. */ if (wc_data->proc_func != NULL) - wc_data->proc_func (wc_data->trns_case, wc_data->aux); + wc_data->proc_func (&wc_data->trns_case, wc_data->aux); done: - clear_case (wc_data->trns_case); + clear_case (&wc_data->trns_case); return 1; } @@ -320,20 +326,20 @@ execute_transformations (struct ccase *c, exclude as specified on FILTER or PROCESS IF, otherwise zero. */ static int -filter_case (const struct ccase *c, int case_num) +filter_case (const struct ccase *c, int case_idx) { /* FILTER. */ struct variable *filter_var = dict_get_filter (default_dict); if (filter_var != NULL) { - double f = c->data[filter_var->fv].f; + double f = case_num (c, filter_var->fv); if (f == 0.0 || f == SYSMIS || is_num_user_missing (f, filter_var)) return 1; } /* PROCESS IF. */ if (process_if_expr != NULL - && expr_evaluate (process_if_expr, c, case_num, NULL) != 1.0) + && expr_evaluate (process_if_expr, c, case_idx, NULL) != 1.0) return 1; return 0; @@ -345,7 +351,8 @@ lag_case (const struct ccase *c) { if (lag_count < n_lag) lag_count++; - memcpy (lag_queue[lag_head], c, dict_get_case_size (temp_dict)); + case_destroy (&lag_queue[lag_head]); + case_clone (&lag_queue[lag_head], c); if (++lag_head >= n_lag) lag_head = 0; } @@ -371,13 +378,17 @@ compact_case (struct ccase *dest, const struct ccase *src) if (dict_class_from_id (v->name) == DC_SCRATCH) continue; - if (v->type == NUMERIC) - dest->data[nval++] = src->data[v->fv]; + if (v->type == NUMERIC) + { + case_data_rw (dest, nval)->f = case_num (src, v->fv); + nval++; + } else { int w = DIV_RND_UP (v->width, sizeof (union value)); - memcpy (&dest->data[nval], &src->data[v->fv], w * sizeof (union value)); + memcpy (case_data_rw (dest, nval), case_str (src, v->fv), + w * sizeof (union value)); nval += w; } } @@ -396,10 +407,10 @@ clear_case (struct ccase *c) struct variable *v = dict_get_var (default_dict, i); if (v->init && v->reinit) { - if (v->type == NUMERIC) - c->data[v->fv].f = SYSMIS; + if (v->type == NUMERIC) + case_data_rw (c, v->fv)->f = SYSMIS; else - memset (c->data[v->fv].s, ' ', v->width); + memset (case_data_rw (c, v->fv)->s, ' ', v->width); } } } @@ -414,7 +425,7 @@ close_active_file (void) int i; for (i = 0; i < n_lag; i++) - free (lag_queue[i]); + case_destroy (&lag_queue[i]); free (lag_queue); n_lag = 0; } @@ -434,20 +445,13 @@ close_active_file (void) /* Free data source. */ if (vfm_source != NULL) { - if (vfm_source->class->destroy != NULL) - vfm_source->class->destroy (vfm_source); - free (vfm_source); + free_case_source (vfm_source); + vfm_source = NULL; } /* Old data sink becomes new data source. */ if (vfm_sink->class->make_source != NULL) vfm_source = vfm_sink->class->make_source (vfm_sink); - else - { - if (vfm_sink->class->destroy != NULL) - vfm_sink->class->destroy (vfm_sink); - vfm_source = NULL; - } free_case_sink (vfm_sink); vfm_sink = NULL; @@ -478,15 +482,18 @@ storage_sink_open (struct case_sink *sink) struct storage_stream_info *info; sink->aux = info = xmalloc (sizeof *info); - info->casefile = casefile_create (sink->value_cnt * sizeof (union value)); + info->casefile = casefile_create (sink->value_cnt); } /* Destroys storage stream represented by INFO. */ static void destroy_storage_stream_info (struct storage_stream_info *info) { - casefile_destroy (info->casefile); - free (info); + if (info != NULL) + { + casefile_destroy (info->casefile); + free (info); + } } /* Writes case C to the storage sink SINK. */ @@ -505,12 +512,15 @@ storage_sink_destroy (struct case_sink *sink) destroy_storage_stream_info (sink->aux); } -/* Closes and destroys the sink and returns a storage source to - read back the written data. */ +/* Closes the sink and returns a storage source to read back the + written data. */ static struct case_source * storage_sink_make_source (struct case_sink *sink) { - return create_case_source (&storage_source_class, sink->dict, sink->aux); + struct case_source *source + = create_case_source (&storage_source_class, sink->dict, sink->aux); + sink->aux = NULL; + return source; } /* Storage sink. */ @@ -543,14 +553,16 @@ storage_source_read (struct case_source *source, write_case_func *write_case, write_case_data wc_data) { struct storage_stream_info *info = source->aux; - const struct ccase *casefile_case; + struct ccase casefile_case; struct casereader *reader; - reader = casefile_get_reader (info->casefile); - while (casereader_read (reader, &casefile_case)) + for (reader = casefile_get_reader (info->casefile); + casereader_read (reader, &casefile_case); + case_destroy (&casefile_case)) { - memcpy (output_case, casefile_case, - casefile_get_case_size (info->casefile)); + case_copy (output_case, 0, + &casefile_case, 0, + casefile_get_value_cnt (info->casefile)); write_case (wc_data); } casereader_destroy (reader); @@ -580,6 +592,17 @@ storage_source_get_casefile (struct case_source *source) assert (source->class == &storage_source_class); return info->casefile; } + +struct case_source * +storage_source_create (struct casefile *cf, const struct dictionary *dict) +{ + struct storage_stream_info *info; + + info = xmalloc (sizeof *info); + info->casefile = cf; + + return create_case_source (&storage_source_class, dict, info); +} /* Null sink. Used by a few procedures that keep track of output themselves and would throw away anything that the sink @@ -600,15 +623,15 @@ struct ccase * lagged_case (int n_before) { assert (n_before <= n_lag); - if (n_before > lag_count) + if (n_before <= lag_count) + { + int index = lag_head - n_before; + if (index < 0) + index += n_lag; + return &lag_queue[index]; + } + else return NULL; - - { - int index = lag_head - n_before; - if (index < 0) - index += n_lag; - return lag_queue[index]; - } } /* Appends TRNS to t_trns[], the list of all transformations to be @@ -660,6 +683,19 @@ create_case_source (const struct case_source_class *class, return source; } +/* Destroys case source SOURCE. It is the caller's responsible to + call the source's destroy function, if any. */ +void +free_case_source (struct case_source *source) +{ + if (source != NULL) + { + if (source->class->destroy != NULL) + source->class->destroy (source); + free (source); + } +} + /* Returns nonzero if a case source is "complex". */ int case_source_is_complex (const struct case_source *source) @@ -692,20 +728,24 @@ create_case_sink (const struct case_sink_class *class, return sink; } -/* Destroys case sink SINK. It is the caller's responsible to - call the sink's destroy function, if any. */ +/* Destroys case sink SINK. */ void free_case_sink (struct case_sink *sink) { - free (sink->idx_to_fv); - free (sink); + if (sink != NULL) + { + if (sink->class->destroy != NULL) + sink->class->destroy (sink); + free (sink->idx_to_fv); + free (sink); + } } /* Represents auxiliary data for handling SPLIT FILE. */ struct split_aux_data { size_t case_count; /* Number of cases so far. */ - struct ccase *prev_case; /* Data in previous case. */ + struct ccase prev_case; /* Data in previous case. */ /* Functions to call... */ void (*begin_func) (void *); /* ...before data. */ @@ -740,7 +780,7 @@ procedure_with_splits (void (*begin_func) (void *aux), struct split_aux_data split_aux; split_aux.case_count = 0; - split_aux.prev_case = xmalloc (dict_get_case_size (default_dict)); + case_nullify (&split_aux.prev_case); split_aux.begin_func = begin_func; split_aux.proc_func = proc_func; split_aux.end_func = end_func; @@ -750,7 +790,7 @@ procedure_with_splits (void (*begin_func) (void *aux), if (split_aux.case_count > 0 && end_func != NULL) end_func (func_aux); - free (split_aux.prev_case); + case_destroy (&split_aux.prev_case); } /* procedure() callback used by procedure_with_splits(). */ @@ -761,13 +801,14 @@ procedure_with_splits_callback (struct ccase *c, void *split_aux_) /* Start a new series if needed. */ if (split_aux->case_count == 0 - || !equal_splits (c, split_aux->prev_case)) + || !equal_splits (c, &split_aux->prev_case)) { if (split_aux->case_count > 0 && split_aux->end_func != NULL) split_aux->end_func (split_aux->func_aux); dump_splits (c); - memcpy (split_aux->prev_case, c, dict_get_case_size (default_dict)); + case_destroy (&split_aux->prev_case); + case_clone (&split_aux->prev_case, c); if (split_aux->begin_func != NULL) split_aux->begin_func (split_aux->func_aux); @@ -798,11 +839,11 @@ equal_splits (const struct ccase *a, const struct ccase *b) switch (v->type) { case NUMERIC: - if (a->data[v->fv].f != b->data[v->fv].f) + if (case_num (a, v->fv) != case_num (b, v->fv)) return 0; break; case ALPHA: - if (memcmp (a->data[v->fv].s, b->data[v->fv].s, v->width)) + if (memcmp (case_str (a, v->fv), case_str (b, v->fv), v->width)) return 0; break; default: @@ -843,12 +884,12 @@ dump_splits (struct ccase *c) assert (v->type == NUMERIC || v->type == ALPHA); tab_text (t, 0, i + 1, TAB_LEFT | TAT_PRINTF, "%s", v->name); - data_out (temp_buf, &v->print, &c->data[v->fv]); + data_out (temp_buf, &v->print, case_data (c, v->fv)); temp_buf[v->print.w] = 0; tab_text (t, 1, i + 1, TAT_PRINTF, "%.*s", v->print.w, temp_buf); - val_lab = val_labs_find (v->val_labs, c->data[v->fv]); + val_lab = val_labs_find (v->val_labs, *case_data (c, v->fv)); if (val_lab) tab_text (t, 2, i + 1, TAB_LEFT, val_lab); } @@ -860,7 +901,7 @@ dump_splits (struct ccase *c) multipass procedure. */ struct multipass_split_aux_data { - struct ccase *prev_case; /* Data in previous case. */ + struct ccase prev_case; /* Data in previous case. */ struct casefile *casefile; /* Accumulates data for a split. */ /* Function to call with the accumulated data. */ @@ -882,7 +923,7 @@ multipass_procedure_with_splits (void (*split_func) (const struct casefile *, open_active_file (); - aux.prev_case = xmalloc (dict_get_case_size (default_dict)); + case_nullify (&aux.prev_case); aux.casefile = NULL; aux.split_func = split_func; aux.func_aux = func_aux; @@ -890,7 +931,7 @@ multipass_procedure_with_splits (void (*split_func) (const struct casefile *, internal_procedure (multipass_split_callback, &aux); if (aux.casefile != NULL) multipass_split_output (&aux); - free (aux.prev_case); + case_destroy (&aux.prev_case); close_active_file (); } @@ -902,18 +943,19 @@ multipass_split_callback (struct ccase *c, void *aux_) struct multipass_split_aux_data *aux = aux_; /* Start a new series if needed. */ - if (aux->casefile == NULL || !equal_splits (c, aux->prev_case)) + if (aux->casefile == NULL || !equal_splits (c, &aux->prev_case)) { /* Pass any cases to split_func. */ if (aux->casefile != NULL) multipass_split_output (aux); /* Start a new casefile. */ - aux->casefile = casefile_create (dict_get_case_size (default_dict)); + aux->casefile = casefile_create (dict_get_next_value_idx (default_dict)); /* Record split values. */ dump_splits (c); - memcpy (aux->prev_case, c, dict_get_case_size (default_dict)); + case_destroy (&aux->prev_case); + case_clone (&aux->prev_case, c); } casefile_append (aux->casefile, c); diff --git a/src/vfm.h b/src/vfm.h index 5d9403bb..9b577ed8 100644 --- a/src/vfm.h +++ b/src/vfm.h @@ -71,11 +71,15 @@ struct dictionary; struct case_source *create_case_source (const struct case_source_class *, const struct dictionary *, void *); +void free_case_source (struct case_source *); + int case_source_is_complex (const struct case_source *); int case_source_is_class (const struct case_source *, const struct case_source_class *); struct casefile *storage_source_get_casefile (struct case_source *); +struct case_source *storage_source_create (struct casefile *, + const struct dictionary *); /* The replacement active file, to which cases are written. */ extern struct case_sink *vfm_sink; @@ -107,9 +111,10 @@ struct case_sink_class /* Closes and destroys the sink. */ void (*destroy) (struct case_sink *); - /* Closes and destroys the sink and returns a source that can - read back the cases that were written, perhaps transformed - in some way. */ + /* Closes the sink and returns a source that can read back + the cases that were written, perhaps transformed in some + way. The sink must still be separately destroyed by + calling destroy(). */ struct case_source *(*make_source) (struct case_sink *); }; diff --git a/tests/ChangeLog b/tests/ChangeLog index cf91b148..ce99b90f 100644 --- a/tests/ChangeLog +++ b/tests/ChangeLog @@ -1,3 +1,27 @@ +Mon May 31 21:49:19 2004 Ben Pfaff + + * bugs/t-test-with-temp.sh: Use $SUPERVISOR. + + * bugs/t-test-1-indep-val.sh: Ditto. + + * bugs/t-test-1-sample-missing-anal.sh: Ditto. + + * bugs/t-test-1-sample-missing-list.sh: Ditto. + + * bugs/t-test-1s.sh: Ditto. + + * bugs/t-test-groups.sh: Ditto. + + * bugs/t-test-indep-missing-anal.sh: Ditto. + + * bugs/t-test-indep-missing-list.sh: Ditto. + + * bugs/t-test-paired-missing-anal.sh: Ditto. + + * bugs/t-test-paired-missing-list.sh: Ditto. + + * bugs/t-test-pairs.sh: Ditto. + Sun May 30 19:18:26 2004 Ben Pfaff * command/tabs.sh: Default tab width is now 4. diff --git a/tests/bugs/t-test-with-temp.sh b/tests/bugs/t-test-with-temp.sh index e77ca799..b7803d20 100755 --- a/tests/bugs/t-test-with-temp.sh +++ b/tests/bugs/t-test-with-temp.sh @@ -64,7 +64,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" diff --git a/tests/command/t-test-1-indep-val.sh b/tests/command/t-test-1-indep-val.sh index 972818ac..89f2a038 100755 --- a/tests/command/t-test-1-indep-val.sh +++ b/tests/command/t-test-1-indep-val.sh @@ -79,7 +79,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi diff --git a/tests/command/t-test-1-sample-missing-anal.sh b/tests/command/t-test-1-sample-missing-anal.sh index bd6a2249..53d463be 100755 --- a/tests/command/t-test-1-sample-missing-anal.sh +++ b/tests/command/t-test-1-sample-missing-anal.sh @@ -64,7 +64,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -90,7 +90,7 @@ EOF if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="compare outputs" diff --git a/tests/command/t-test-1-sample-missing-list.sh b/tests/command/t-test-1-sample-missing-list.sh index 437725dd..2e2f1d01 100755 --- a/tests/command/t-test-1-sample-missing-list.sh +++ b/tests/command/t-test-1-sample-missing-list.sh @@ -63,7 +63,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -90,7 +90,7 @@ EOF if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="compare outputs" diff --git a/tests/command/t-test-1s.sh b/tests/command/t-test-1s.sh index cd1b4ad5..b440d555 100755 --- a/tests/command/t-test-1s.sh +++ b/tests/command/t-test-1s.sh @@ -63,7 +63,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="compare output" diff --git a/tests/command/t-test-groups.sh b/tests/command/t-test-groups.sh index 487ff30b..1173dcc2 100755 --- a/tests/command/t-test-groups.sh +++ b/tests/command/t-test-groups.sh @@ -70,7 +70,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="compare output" diff --git a/tests/command/t-test-indep-missing-anal.sh b/tests/command/t-test-indep-missing-anal.sh index 4c38822e..30dc2b7b 100755 --- a/tests/command/t-test-indep-missing-anal.sh +++ b/tests/command/t-test-indep-missing-anal.sh @@ -63,7 +63,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -91,7 +91,7 @@ EOF if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi diff --git a/tests/command/t-test-indep-missing-list.sh b/tests/command/t-test-indep-missing-list.sh index 0a807cc2..1f122300 100755 --- a/tests/command/t-test-indep-missing-list.sh +++ b/tests/command/t-test-indep-missing-list.sh @@ -64,7 +64,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -89,7 +89,7 @@ EOF if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi diff --git a/tests/command/t-test-paired-missing-anal.sh b/tests/command/t-test-paired-missing-anal.sh index a4ac3b9e..2829385e 100755 --- a/tests/command/t-test-paired-missing-anal.sh +++ b/tests/command/t-test-paired-missing-anal.sh @@ -63,7 +63,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -89,7 +89,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi diff --git a/tests/command/t-test-paired-missing-list.sh b/tests/command/t-test-paired-missing-list.sh index 21dbc376..5158a6a1 100755 --- a/tests/command/t-test-paired-missing-list.sh +++ b/tests/command/t-test-paired-missing-list.sh @@ -63,7 +63,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 1" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="copy output" @@ -89,7 +89,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program 2" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi diff --git a/tests/command/t-test-pairs.sh b/tests/command/t-test-pairs.sh index 6f2c42a1..80920a6a 100755 --- a/tests/command/t-test-pairs.sh +++ b/tests/command/t-test-pairs.sh @@ -62,7 +62,7 @@ if [ $? -ne 0 ] ; then no_result ; fi activity="run program" -$here/../src/pspp -o raw-ascii $TEMPDIR/out.stat +$SUPERVISOR $here/../src/pspp -o raw-ascii $TEMPDIR/out.stat if [ $? -ne 0 ] ; then no_result ; fi activity="compare output"