X-Git-Url: https://pintos-os.org/cgi-bin/gitweb.cgi?a=blobdiff_plain;f=src%2Fdata%2FChangeLog;h=ef91902bc486efa312eef48f046d57c4d5ea5ea2;hb=aab828a13b340ac7f3302e3a009274ed49d43ef1;hp=56976e375c6e92a916773fc02be92c1722d29886;hpb=e21ed65bab28549cd4bb5e8761e5632567ac4cb2;p=pspp-builds.git diff --git a/src/data/ChangeLog b/src/data/ChangeLog index 56976e37..ef91902b 100644 --- a/src/data/ChangeLog +++ b/src/data/ChangeLog @@ -1,3 +1,1004 @@ +2008-07-25 Ben Pfaff + + * casereader-translator.c (struct casereader_arithmetic_sequence): + New struct. + (casereader_create_arithmetic_sequence): New function. + (cas_translate): New function. + (cas_destroy): New function. + +2008-07-26 John Darrington + + * case-ordering.c case-ordering.h: Removed the value_cnt + associated with the ordering structure. + +2008-07-16 Jason Stover + + * category.c (cat_get_category_count): New function. + +2008-05-15 Ben Pfaff + + Patch #6512. + + * data-out.c (output_infinite): Use gsl_isnan instead of isnan, + and gsl_isinf instead of isinf, as a stopgap measure for + portability until appropriate gnulib modules are available. + + * por-file-writer.c (format_trig_double): Similarly, use + gsl_finite instead of finite. + +2008-03-18 John Darrington + + * data-in.c: If category is custom currency, then use + numeric format for input. + +2008-03-06 Ben Pfaff + + * dictionary.c (var_name_is_insertable): New function. + (make_hinted_name): Don't accept variable names that match PSPP + keywords. Thanks to Jason Stover for reporting the problem. + +2008-03-06 Ben Pfaff + + * format-guesser.c (syntax): Require month names to be spelled out + as English words, so that single characters that happen to be + Roman numerals don't get detected as months. Thanks to John + Darrington for reporting this bug. + +2008-03-04 Ben Pfaff + + Patch #6441. Reviewed by John Darrington. + + * format.c (fmt_fix): New function. + (fmt_fix_input): New function. + (fmt_fix_output): New function. + + * format.def: Correct minimum width for DATETIME format. It was + 7, should have been 17. + + * automake.mk: Add new files. + + * format-guesser.c: New file. + + * format-guesser.h: New file. + +2008-02-18 Ben Pfaff + + Patch #6426. Reviewed by John Darrington. + + * format.c (min_width): Renamed fmt_min_width and made public. + Updated all references. + (max_width): Renamed fmt_max_width and made public. Updated all + references. + (max_decimals): Renamed fmt_max_decimals and made public. Updated + all references. + (var_create): Use the new functions for default variable + attributes below. + (var_default_formats): New function. + (var_default_measure): New function. + (var_default_alignment): New function. + + * format.h (macro FMT_MAX_NUMERIC_WIDTH): New macro. + +2008-02-09 Ben Pfaff + + Add a couple of extensions to GET DATA TYPE=TXT. Patch #6412. + Thanks to John Darrington for review. + + * data-in.c (data_in): Add new argument to designate the last + column of the data field being parsed, for use in error messages. + Update all callers. + + * data-parser (struct data_parser): New member `quote_escape'. + (data_parser_create): Initialize quote_escape. + (data_parser_set_quotes): New function. + (cut_field): Support escaped quotes. + (parse_delimited_span): Ditto. + (parse_delimited_no_span): Ditto. + + * get-data.c (parse_get_txt): Support ESCAPE extension subcommand + in enhanced mode. Only support multiple quote characters in + enhanced mode. + +2008-02-06 John Darrington + + psql-reader.c psql-reader.h: Read more than one tuple at + once. Fix bug reading a query which returns no data. Fix bug + when transformation followed a reader. + Ask the server for the number of records in the query, for the + benefit of the gui. + +2008-02-05 John Darrington + + psql-reader.c: So yesterday they release postgresql 8.3.0 + which has money represented with 64 bits. They must get + paid more than me. + +2008-02-02 John Darrington + + psql-reader.c psql-reader.h: New files. Thanks to Ben Pfaff + for reviewing this code. + +2008-02-02 Ben Pfaff + + Patch #6347. + + * sys-file-reader.c (read_variable_record): Allow missing values + to be specified on long string variables, but warn about them + because PSPP does not yet support them. + (read_extension_record): Ignore extension records 20 and 21, which + PSPP does not yet support. + (read_header): Fix error message when floating-point format cannot + be identified. + +2008-02-01 Ben Pfaff + + Patch #6386. Thanks to John Darrington for review and for the + updates to gnumeric-reader.c. + + * dictionary.c (make_hinted_name): New function. + (make_numeric_name): New function. + (dict_make_unique_var_name): New function. + + * gnumeric-reader.c (devise_name): Removed. + (munge_name): Removed. + (gnumeric_open_reader): Use new function + dict_make_unique_var_name. + + * short-names.c (set_var_short_name_suffix): Use new function + str_format_26adic. + +2008-01-19 John Darrington + + * settings.c settings.h: Moved static variables into a + single struct. Renamed functions to have a settings_ prefix. + +2008-01-14 John Darrington + + * data-out.c (output_AHEX): Corrected number of bytes in + call to output_hex. Closes bug #22011 + +2008-01-02 John Darrington + + * variable.c variable.h: Replaced var_get_value_name with + var_append_value_name which doesn't use any static data. + Thanks to Ben for review. + +2007-12-07 Ben Pfaff + + Patch #6302. + + * casegrouper.c (casegrouper_get_next_group): Cause a casegrouper + made from an empty casereader produce a casegrouper with no + groups, instead of one with one group that has no cases. + + * casereader.c (casereader_is_empty): New function. + +2007-12-06 Ben Pfaff + + Patch #6303. + + * sys-file-reader.c (read_display_parameters): Handle variable + display parameters record with only 2 data items per variable. + Reported by Guido Gay . + +2007-12-04 Ben Pfaff + + * identifier.c (lex_id_match_n): New function. + (lex_id_match): Reimplement in terms of lex_id_match_n. + +2007-11-24 Ben Pfaff + + * automake.mk (src_data_libdata_a_SOURCES): Add val-type.h, to fix + make distcheck. + +2007-11-24 Ben Pfaff + + Fix warning reported when reading back system files that include + very long string variables. Thanks to Guido Gay for + reporting this bug. + + * short-names.c (short_names_assign): Fix dumb typo, in which `i' + was written where `j' was meant. + +2007-11-18 Ben Pfaff + + Properly write variables that include a range of missing values to + system files. Thanks to Guido Gay for reporting + this bug. + + * sys-file-writer.c (write_variable): Correctly calculate + n_missing_values field when writing variables that include a range + of missing values. + +2007-11-10 Ben Pfaff + + Cleanups and bug fixes devised while writing up documentation. + Patch #6262. + + * automake.mk (src_data_libdata_a_SOURCES): Add new files. + + * dict-class.c: New file. + (dict_class_from_id): Move here. + (dict_class_to_name): Move here. + + * dict-class.h: New file. + (enum dict_class): Move here. Change from consecutive integers to + consecutive bits, to make testing for any of multiple values + easier. Add new DC_ALL constant. + + * dictionary.c (struct dictionary): Change `case_limit' from + size_t to casenumber. + (dict_get_vars): Make final argument an enum dict_class. + (dict_get_vars_mutable): Ditto. + (dict_get_case_limit): Change return value to casenumber. + (dict_set_case_limit): Change final argument to a casenumber. + (dict_unset_split_var): Add assertion. Rephrase slightly. + (dict_set_label): Use xstrndup to simplify. + + * format.c (fmt_step_width): AHEX format also needs 2-byte + stepping. + (fmt_set_style): Simplify assertions. + + * missing-values.c (mv_add_num_range): Rename mv_add_range. + Simplify implementation. + (mv_has_value): Simplify implementation. + (mv_pop_value): Remove the first value, not the last, to avoid + having GET followed by SAVE reverse the order of missing values. + (mv_peek_value): Rename mv_get_value. Simplify assertion. + (mv_has_range): Simplify implementation. + (mv_peek_range): Rename mv_get_range. + (can_resize_string): Removed. + (mv_is_resizable): Use value_is_resizable. + (mv_resize): Use value_resize. + + * short-names.h (SHORT_NAME_LEN): Move here. + + * val-type.h: New file, for definitions related to type and width + of abstract values. Before, these definitions were mixed among + those related to "union value" and those related to variables. + (macro SYSMIS): Move here. + (macro LOWEST): Move here. + (macro HIGHEST): Move here. + (macro MAX_STRING); Move here. + (enum val_type): New enum with values VAL_NUMERIC and VAL_STRING. + Replaces enum var_type that had values VAR_NUMERIC and VAR_STRING. + All references updated. + (val_type_is_valid): New function. Replaces var_type_is_valid. + All references updated. + (val_type_from_width): New function. Replaces + var_type_from_width. All references updated. + + * value-labels.c (val_labs_copy): Renamed val_labs_clone. All + references updated. + (val_labs_can_set_width): Use value_is_resizable. + (val_labs_add): Simply return false if the value labels set is too + wide, instead of having undefined behavior. + (val_labs_replace): Ditto. + (val_labs_replace): Ditto. + (val_labs_first): Set iterator to null if iteration is complete. + (val_labs_first_sorted): Ditto. + (val_labs_done): Become a no-op if the iterator is null. + + * value.c (value_is_resizable): New function. + (value_resize): New function. + + * variable.c (var_get_dict_class): New function. + + * variable.h (macro LONG_NAME_LEN): Rename VAR_NAME_LEN. Update + all references. + +2007-11-08 Ben Pfaff + + * data-in.c: Make formatted data parsing locale-independent. + (parse_number): Use c_strtod instead of strtod, to avoid + locale-specific behavior. + (parse_Z): Ditto. + +2007-11-06 Ben Pfaff + + Patch #6256: add support for binary, 360 file formats. Reviewed + by John Darrington. + + * data-in.c (struct data_in): Add `encoding' member. + (data_in): Add `encoding' parameter, and re-encode the data passed + in where appropriate. Update all callers to pass it in. + (parse_A): Implement EBCDIC recoding wart described in manual. + (parse_AHEX): Implement EBCDIC recoding. + + * data-out.c (data_out_legacy): New function. + (data_out): Make into a wrapper around data_out_legacy. + + * file-handle-def.c (struct file_handle): New member `encoding'. + (fh_create_file): Set encoding. + (fh_default_properties): Set default encoding. + (fh_get_legacy_encoding): New function. + + * file-handle-def.h (enum fh_mode): New modes FH_MODE_FIXED + (that replaces FH_MODE_BINARY), FH_MODE_VARIABLE, + FH_MODE_360_VARIABLE, FH_MODE_360_SPANNED. + (struct fh_properties): New member `encoding'. + +2007-11-05 Ben Pfaff + + Patch #6258. Reviewed by John Darrington. + + * file-handle-def.c (fh_lock): Add comment that TYPE should be + marked with N_() in the caller. Added these markings to each + caller too. Should make i18n easier. + Suggested by Chusslove Illich . + +2007-11-03 Ben Pfaff + + Allow output files to overwrite input files (bug #21280). + + * file-handle-def.c: Separate locking of files for input and for + output, to allow both to take place at once. Also, distinguish a + file handle from the identity of the underlying file, because the + identity of a file changes over time and the file handle can't + represent two different identities. + (struct file_handle): Remove `next', `open_cnt', `deleted', + `type', `open_mode', `aux', `identity' members. Change `id' from + char array to char *. Add `ref_cnt' member. + (file_handle_from_ll) New function. + (file_handles) Removed. + (named_handles) New variable. + (fh_init) Initialize named_handles. + (fh_done) Remove name from all named_handles. + (free_handle) Updated for modified struct file_handle. + (unname_handle) New function. + (fh_ref) New function. + (fh_from_file_name) Removed. + (fh_unref) New function. + (fh_unname) New function. + (fh_from_id) Rewritten. + (create_handle) Updated for modified struct file_handle. + (fh_inline_file) Increment the handle's ref count. + (fh_create_file) Updated for modified struct file_handle. + (fh_create_scratch) Ditto. + (fh_free) Removed. + (mode_name) Removed. + (fh_open) Removed. + (fh_close) Removed. + (fh_is_open) Removed. + (fh_get_id) Updated for modified struct file_handle. + (fh_get_default_handle) Increment the handle's ref count. + (fh_set_default_handle) Handle ref counts. + (struct fh_lock) New structure. + (locks) New static var. + (fh_lock) New function. + (fh_unlock) New function. + (fh_lock_get_aux) New function. + (fh_lock_set_aux) New function. + (fh_is_locked) New function. + (make_key) New function. + (free_key) New function. + (compare_fh_locks) New function. + (hash_fh_lock) New function. + + * file-handle-def.h (enum fh_access) New enum. + + * file-name.c: Made file_identity the same in all supported + environments. + (struct file_identity): New `name' member. + (fn_get_identity): For a file that doesn't exist, get the + dev/inode of its directory plus its name. If even the directory + doesn't exist, just use its name. Merge the Windows + implementation into the Unix one. + (fn_compare_file_identities): Rewritten. Merge the Windows + implementation into the Unix one. + (fn_hash_identity): New function. + + * make-file.c (struct replace_file): New structure. + (all_files): New static var. + (replace_file_start): New function. + (replace_file_commit): New function. + (replace_file_abort): New function. + (free_replace_file): New function. + (unlink_replace_files): New function. + + * por-file-reader.c (struct pfm_reader): Add `lock' member. + (close_reader): Unlock file. + (pfm_open_reader): Lock file. + + * por-file-writer.c (struct pfm_writer): Add fh_lock, replace_file + members. + (pfm_open_writer): Lock file and prepare for its replacement. + (close_writer): Unlock file. + + * scratch-handle.h (struct scratch_handle): Add unique_id so that + different generations of a scratch file can be distinguished. + + * scratch-reader.c (scratch_reader_open): Verify that the file is + a scratch file. + + * scratch-writer.c (struct scratch_writer): Add `lock' and `dict', + remove scratch_handle member. + (scratch_writer_open): Lock handle. Prepare to replace handle + data, instead of doing it immediately. + (scratch_writer_casewriter_destroy): Replace handle data and + unlock handle. + + * sys-file-reader.c (struct sfm_reader): Add `lock' member. + (sfm_open_reader): Lock file. + (close_reader): Unlock file. + + * sys-file-writer.c (struct sfm_writer): Add fh_lock, replace_file + members. + (sfm_open_writer): Lock file and prepare for its replacement. + (close_writer): Unlock file. + +2007-11-02 Ben Pfaff + + * data-out.c (output_number): Use isfinite (from C99) instead of + gsl_isfinite, since we now have universal support for it in + gnulib. + (output_infinite): Ditto. + [!HAVE_ROUND] (round): Remove definition, since we now have a + replacement in gnulib. + +2007-10-27 John Darrington + + * dictionary.c: Fixed bug in dict_clone, where the vardict.dict member + wasn't initialised correctly. Closes bugs #21330 and 21397 + + +2007-10-08 Ben Pfaff + + Fix bug #21280. Thanks to John Darrington for review. + + * file-name.c (create_stream): New function. + + * por-file-writer.c (pfm_open_writer): Use fh_open to open the + file handle before creating the file, to ensure that we don't + truncate a file that we're reading. Make code easier to read by + using create_stream. + + * sys-file-write.c (sfm_open_writer): Ditto. + +2007-10-01 Ben Pfaff + + Fix bug #21192. Thanks to John Darrington for review. + + * casereader.c (casereader_read): Decrement case_cnt before + calling the casereader's "read" member function, so that we + interact properly with lazy_casereader. + + * datasheet.c: Add regression test for above bug fix. + (clone_datasheet): New function. + (lazy_callback): New function. + (check_datasheet_casereader): New function. + (check_datasheet): Check datasheet contents are reported correctly + through an ordinary casereader and a lazy casereader. + (clone_model): Use clone_datasheet. + +2007-09-24 Ben Pfaff + + Patch #6210. Reviewed by John Darrington. + * settings.c: Drop "long view", which is not needed any longer. + (static var long_view): Removed. + (force_long_view): Removed. + (get_viewwidth): Removed. + (init_viewport): Removed long_view reference. + +2007-09-19 John Darrington + + * settings.c settings.h: Changed viewport's length and width to be + owned by the user interface which uses the data library. This allows + better abstraction, and makes dynamically adjustable dimensions easier. + +2007-09-18 Ben Pfaff + + * procedure.c (proc_extract_active_file_data): New function. + + * lazy-casereader.h: New file. + + * lazy-casereader.c: New file. + + * casereader.c (casereader_dynamic_cast): New function. + +2007-09-14 Ben Pfaff + + * dictionary.c (dict_clone): Copy case indexes from cloned + dictionary. Fixes bug #21061. Reviewed by John Darrington. + +2007-09-13 John Darrington + + * value.c value.h (create_value): New function. + +2007-09-12 Ben Pfaff + + Make it clear that translator casereader and casewriter translate + functions are supposed to destroy their input case. + + * casereader-translator.c (struct casereader_translator): Change + input case parameter of translate member function from const + struct ccase * to struct ccase *. + (casereader_create_translator): Ditto, for translate parameter. + + * casewriter-translator.c (struct casewriter_translator): Ditto. + (casewriter_create_translator): Ditto. + +2007-08-27 John Darrington + + * sys-file-reader.c (read_display_parameters): Force display width + to 8, if the sys file says 0 (like SPSS does). + +2007-08-12 Ben Pfaff + + * dictionary.c (dict_dump): New function. + +2007-08-12 Ben Pfaff + + Drop dict_compactor in favor of using the new struct case_map. + + * dictionary.c (struct copy_map): Removed. + (struct dict_compactor): Removed. + (dict_make_compactor): Removed. + (dict_compactor_compact): Removed. + (dict_compactor_destroy): Removed. + + * procedure.c (struct data_set): Change `compactor' member to be a + struct case_map *. + (proc_open): Use case_map_to_compact_dict instead of + dict_make_compactor. + (proc_casereader_read): Use case_map_execute instead of + dict_compactor_compact. + (proc_commit): Use case_map_destroy instead of + dict_compactor_destroy. + + * scratch-writer.c (struct scratch_writer): Change `compactor' + member to be a struct case_map *. + (scratch_writer_open): Use case_map_to_compact_dict instead of + dict_make_compactor. + (scratch_writer_casewriter_write): Use case_map_execute instead of + dict_compactor_compact. + +2007-08-12 Ben Pfaff + + * automake.mk: Add case-map.c, case-map.h. + + * case-map.c: New file. + + * case-map.h: New file. + +2007-08-12 Ben Pfaff + + * dictionary.c (dict_compact_values): Don't delete scratch + variables as well as compacting case indexes. Update all callers. + (dict_get_compacted_value_cnt): Rename dict_count_values and + change interface. Update all callers. + (dict_get_compacted_value_cnt): Remove. + (dict_compacting_would_shrink): Remove. + (dict_compacting_would_change): Remove. + (dict_make_compactor): Add new parameter. Update all callers. + + * procedure.c (proc_casereader_read): Use casewriter_get_value_cnt + instead of dict_count_values, changing an O(N) operation into + O(1). + +2007-08-12 Ben Pfaff + + * casereader.c (casereader_read): Don't require cases read by a + casereader to be exactly the expected size: as long as they're big + enough, it's OK. + +2007-08-12 Ben Pfaff + + Make casewriters keep track of the number of `union value's in + each case. This is useful for two reasons: casewriter_write can + then check that the case being written is large enough, and later + recipients of the casewriter can determine the size of the case. + + * casewriter-translator.c (casewriter_create_translator): Add + value_cnt parameter. + + * casewriter.c (struct casewriter): Add value_cnt member. + (casewriter_write): Check that the case passed in is big enough. + (casewriter_get_value_cnt): New function. + (casewriter_create): Add value_cnt parameter. + +2007-08-09 Ben Pfaff + + Fix bug reported by Jason Stover. + * settings.c: Move get_termcap_viewport and in particular the + #include for to the end of the file. curses.h + redefines bool on some systems (e.g. OpenBSD), which causes + disagreement between uses of bool before its inclusion and + afterward. + Tested by Jason Stover. + +2007-07-29 Ben Pfaff + + Provisional fix for bug #18692 and bug #20161. Reviewed by John + Darrington. + + * file-name.c (fn_open): Only pass "r" or "w" to popen as mode + argument (never "rb" or "wb") because SUSv3 says that only those + modes are defined, and glibc in fact rejects other modes. + + Open portable files with fn_open so that they can be read from + pipes. Fix missing fh_close call to go along with fh_open. + Report an error if the file close reports an error. + * por-file-reader.c (close_reader): New function. + (por_file_casereader_destroy): Use close_reader. + (pfm_open_reader): Open file with fn_open. + +2007-07-28 Ben Pfaff + + Make PSPP able to read all the portable files I could find on the + web. Thanks to John Darrington for review. Bug #17620. + * por-file-reader.c (struct pfm_reader): New member `line_length'. + (error): Print file offset in hexadecimal. + (warning): New function. + (advance): Treat lines less than 80 bytes long as padded to 80 + bytes with spaces. + (pfm_open_reader): Call read_documents if we find an "E" record. + (convert_format): Convert invalid formats to the default format + instead of aborting reading the file. + (read_variables): Rename duplicate variable names instead of + aborting reading the file. + (read_value_label): Allow string variables of different widths to + be assigned value labels in the same record. Replace duplicate + value labels instead of aborting. + (read_documents): New function. + + * por-file-writer.c (pfm_open_writer): Call write_documents if the + dictionary has documents. + (write_documents): New function. + +2007-07-25 Ben Pfaff + + Fix bugs related to bug #17213. + + * settings.c: Use HAVE_LIBNCURSES instead of HAVE_LIBTERMCAP, + since the former is what config.h has. Include the needed ncurses + headers. + (static var echo) Rename to `do_echo' because the original name is + the same as an ncurses identifier. + (get_termcap_viewport) Use error instead of msg. + + * file-name.c (fn_interp_vars): Fix interpolation of $VARS. + (fn_close): Don't close stdin, stdout, stderr. + +2007-07-26 John Darrington + + * procedure.c procedure.h: Added callbacks which get invoked whenever + a dataset's transformation chain changes. + +2007-07-24 Ben Pfaff + + Fix bug #6113. + * sys-file-writer.c (write_variable_display_parameters): Use new + var_default_display_width function to choose display width of + segments after the first one in a given variable. + * variable.c (var_create): Use var_default_display_width to pick + new variable's display width. + (var_default_display_width): New function. + Reviewed by John Darrington. + +2007-07-24 Ben Pfaff + + Fix bug #20427. + * por-file-writer.c (write_variables): Write weight variable. + Reviewed by John Darrington. + +2007-07-23 Ben Pfaff + + Improvements to system file reader and writer. + + First, move all detailed knowledge of very long strings into + sys-file-private.[ch], so that this nasty stuff can be isolated. + + * sys-file-private.c (REAL_VLS_CHUNK): New macro. + (EFFECTIVE_VLS_CHUNK): New macro. + (min_int): New function. + (max_int): New function. + (sfm_width_to_bytes): Rewrite. + (sfm_width_to_octs): New function. + (sfm_segment_alloc_width): New function. + (sfm_segment_alloc_bytes): New function. + (sfm_segment_used_bytes): New function. + (sfm_segment_offset): New function. + (sfm_segment_effective_offset): New function. + (sfm_dictionary_to_sfm_vars): New function. + + * sys-file-private.h (MIN_VERY_LONG_STRING): Removed. + (EFFECTIVE_LONG_STRING_LENGTH): Removed. + (struct sfm_var): New structure. + + Next, improvements to the system file reader. + + * sys-file-reader.h (struct sfm_read_info): Changed `case_cnt' to + type casenumber. Added `version_major', `version_minor', + `version_revision'. + + * sys-file-reader.c (struct sfm_reader): Replaced `flt64_cnt' by + `oct_cnt'. Rename `vars', `var_cnt' to `sfm_vars', `sfm_var_cnt'. + Change `case_cnt' to type casenumber. Removed `has_vls'. + (struct sfm_var): Removed. + (sfm_open_reader): Don't warn on wrong case size if the file was + written by SPSS 13, which tends to get it wrong. Use + sfm_dictionary_to_sfm_vars. + (read_header): Always output system file info. + (read_variable_record): Simplify code for reading missing values. + (read_machine_int32_info): Save version numbers from system file + into info struct passed as new argument. + (read_long_string_map): Restructured to use new sys-file-private + functions. + (read_value_labels): Use size_overflow_p. + (sys_file_casereader_read): Get rid of distinction between fast + and slow paths. Use information provided by sys-file-primate's + struct sfm_var to simplify code. + (skip_whole_strings): New function. + (read_int32): Renamed read_int. Changed return value to int. + Updated all callers. + (read_flt64): Renamed read_float. Changed return value to + double. Updated all callers. + (int32_to_native): Removed. Changed callers to use + integer_convert. + (flt64_to_double): Removed. Changed callers to use float_convert. + + Finally, get rid of int32, flt64 terminology and types in system + file writer. The former wasn't very useful since a POSIX "int" + can hold the whole range of int32 and we generally didn't have a + need for it to be exactly-32-bits, just at-least-32-bits. The + latter was inconvenient because we had to assume that it could be + different from double and thereby convert special values SYSMIS, + HIGHEST, LOWEST to and from it in multiple places. Instead, now + we just use "int" and "double" in most places, and do conversions, + if necessary, very close to where we do I/O. This change meant + that the writer code couldn't represent records in the file as C + structs any longer, but that's no great loss. The code actually + seems to be more readable without them. + + Simplify the compression buffering code: only buffer as much as + necessary, which is no more than eight 8-byte units at any given + time. + + * sys-file-writer.c (typedef flt64): Removed. + (macro second_lowest_flt64): Removed. + (struct sysfile_header): Removed. + (struct sysfile_variable): Removed. + (struct sfm_writer): Removed `needs_translation', `has_vls', + `flt64_cnt'. Changed `compress' to type bool and `case_cnt' to + type casenumber. Renamed `vars' to `sfm_vars', `var_cnt' to + `sfm_var_cnt'. Replaced `buf', `end', `ptr', `x', `y' for + compression buffering by `opcodes', `opcode_cnt', `data', + `data_cnt'. Renamed `var_cnt_vls' as `segment_cnt'. + (sfm_open_writer): Use sfm_dictionary_to_sfm_vars. Use simple + data writer functions instead of structures. + (calc_oct_idx): New function. + (write_header): Use simple data writer functions instead of + structures. + (write_format_spec): Renamed write_format. New argument. + (write_variable_continuation_records): New function. + (write_variable): Use simple data writer functions instead of + structures. Use write_variable_continuation_records. Write + entire very long string instead of requiring caller to understand + them. + (write_value_labels): Use simple data writer functions instead of + structures. + (write_documents): Ditto. + (write_variable_display_parameters): Use sys-file-private + functions to simplify. Use simple data writer functions instead + of structures. + (write_vls_length_table): Use simple data writer functions instead + of structures. + (write_longvar_table): Ditto. + (write_rec_7_34): Break into new functions + write_integer_info_record, write_float_info_record. Use simple + data writer functions instead of structures. + (buf_write): Removed. + (append_string_max): Removed. + (ensure_buf_space): Removed. + (sys_file_casewriter_write): Get rid of the distinction between + fast and slow paths, which didn't seem to be too useful. Use new + functions write_case_uncompressed, write_case_compressed. + (put_instruction): Removed. + (put_element): Removed. + (write_compressed_data): Removed. + (close_writer): Use flush_compressed. Only write case count to + system file if it will fit in the field. + (write_case_compressed): New function. + (write_case_uncompressed): New function. + (flush_compressed): New function. + (put_cmp_opcode): New function. + (put_cmp_number): New function. + (write_int): New function. + (convert_double_to_output_format): New function. + (write_float): New function. + (write_value): New function. + (write_string): New function. + (write_bytes): New function. + (write_zeros): New function. + (write_spaces): New function. + + Reviewed by John Darrington. + +2007-07-22 Ben Pfaff + + Don't try to write very long strings to portable files. The + format does not support it. + + * por-file-writer.c (MAX_POR_WIDTH): New macro. + (pfm_open_writer): Limit output width to MAX_POR_WIDTH. + (write_format): Add arg to take width to resize format to. + (write_value): Limit width of value written to MAX_POR_WIDTH. + (write_variables): Limit width of variable and its output formats + to MAX_POR_WIDTH. + Reviewed by John Darrington. + +2007-07-22 Ben Pfaff + + * sys-file-reader.c (read_variable_to_value_map): Use max_warnings + local variable instead of literal 5. + Reviewed by John Darrington. + +2007-07-22 Ben Pfaff + + Fix problems with uniqueness of short names in system files with + very long string variables. Now a variable may have multiple + short names. + + * automake.mk (src_data_libdata_a_SOURCES): Add new files + short-names.c, short-names.h. + + * dictionary.c (dict_clone): Clone all the short names. + (compare_strings): Move into short-names.c. + (hash_strings): Ditto. + (set_var_short_name_suffix): Ditto. + (dict_assign_short_names): Ditto, rename short_names_assign, + change to assign all short names. + + * por-file-writer.c (write_variables): Use short_names_assign + instead of dict_assign_short_names. + + * short-names.c: New file. + + * short-names.h: New file. + + * sys-file-private.c (sfm_width_to_segments): New function. + + * sys-file-reader.c (read_long_var_name_map): Save and restore all + the short names, not just the first one. + + * sys-file-writer.c (cont_var_name): Removed. + (sfm_open_writer): Use short_names_assign instead of + dict_assign_short_names. Use unique short names assigned by + short_names_assign instead of those generated by cont_var_name. + + * variable.c (struct variable): Remove `short_name' member, + replace by `short_names' and `short_name_cnt'. + (var_create) Initialize new members. + (var_get_short_name_cnt): New function. + (var_get_short_name): Now takes an index argument. Changed most + callers to pass 0. + (var_set_short_name): Ditto. + (var_clear_short_name): Renamed var_clear_short_names, changed to + clear all short names. + + Reviewed by John Darrington. + +2007-07-22 Ben Pfaff + + * variable.c (var_set_width): Use new var_set_width function. + + * missing-values.c (mv_n_values): Drop assertion, which was not + needed. + + * format.c (fmt_default_for_width): New function. + (fmt_resize): New function. + + Reviewed by John Darrington. + +2007-07-18 John Darrington + + * datasheet.c (datasheet_delete_columns): Added assertion to check + we're not deleting outside the range of the sheet. + + + * dictionary.c dictionary.h variable.c: Added the ability for string + variables to be resized. + + * vardict.h: Added some prototypes (moved from dictionary.h) as + these should only be called by variable.c + + +2007-07-14 John Darrington + + * sfm-reader.c: Respect case_cnt field in file header. + +2007-07-01 John Darrington + + * transformation.c transformation.h (trns_chain_execute): Changed the + signature (Patch #6057) + +2007-06-10 Ben Pfaff + + * casereader-filter.c (casereader_filter_destroy): Make sure to + write all the remaining excluded cases to the casewriter, if any. + + * caseinit.c (init_list_destroy): Rewrite. + (init_list_clear): Ditto. + + * casegrouper.c (casegrouper_get_next_group): Always set *reader + to null when returning false. + +2007-06-06 Ben Pfaff + + Actually implement the new procedure code and adapt all of its + clients to match. Also adapt all of the other case sources and + sinks in the tree and their clients to use the + casereader/casewriter infrastructure. + + * automake.mk: Add and remove files. + + * any-reader.c: Change into a casereader. + * por-file-reader.c: Ditto. + * scratch-reader.c: Ditto. + * sys-file-reader.c: Ditto. + + * any-writer.c: Change into a casewriter. + * por-file-writer.c: Ditto. + * scratch-writer.c: Ditto. + * sys-file-writer.c: Ditto. + + * procedure.c: Change to use casereader, casewriter, caseinit, and + other new infrastructure. + + * scratch-handle.c: Adapt to new infrastructure. + + * case-sink.c: Removed, now dead code. + * case-sink.h: Ditto. + * case-source.c: Ditto. + * case-source.h: Ditto. + * casefile-factory.c: Ditto. + * casefile-private.h: Ditto. + * casefile.c: Ditto. + * casefile.h: Ditto. + * casefilter.c: Ditto. + * casefilter.h: Ditto. + * fastfile.c: Ditto. + * fastfile.h: Ditto. + * fastfile-factory.c: Ditto. + * fastfile-factory.h: Ditto. + * storage-stream.c: Ditto. + * storage-stream.h: Ditto. + +2007-06-06 Ben Pfaff + + Add datasheet code. + + * automake.mk: Add new files. + + * datasheet.c: New file. + + * datasheet.h: New file. + +2007-06-06 Ben Pfaff + + Until now, the procedure code has provided a case to the + case_source, which has filled in the data values that come from + the active file. "Left" data values that don't come from the + active file naturally stay the same from case to case, because the + procedure code keeps using that same case. + + One of the compromises that comes with the new procedure code is + that the active file allocates and provides its own case, which + the procedure code then has to resize to provide room for any + other variables that should go in the case and then fill in the + values of "left" variables. Then, when we're done with that case, + we have to save the values of "left" variables to copy into the + next case read from the active file. + + The caseinit code helps with this. + + * automake.mk: Add new files. + + * caseinit.c: New file. + + * caseinit.h: New file. + 2007-06-06 Ben Pfaff * value.h (value_cnt_from_width): New function.