+2008-03-06 Ben Pfaff <blp@gnu.org>
+
+ * dictionary.c (var_name_is_insertable): New function.
+ (make_hinted_name): Don't accept variable names that match PSPP
+ keywords. Thanks to Jason Stover for reporting the problem.
+
+2008-03-06 Ben Pfaff <blp@gnu.org>
+
+ * format-guesser.c (syntax): Require month names to be spelled out
+ as English words, so that single characters that happen to be
+ Roman numerals don't get detected as months. Thanks to John
+ Darrington for reporting this bug.
+
+2008-03-04 Ben Pfaff <blp@gnu.org>
+
+ Patch #6441. Reviewed by John Darrington.
+
+ * format.c (fmt_fix): New function.
+ (fmt_fix_input): New function.
+ (fmt_fix_output): New function.
+
+ * format.def: Correct minimum width for DATETIME format. It was
+ 7, should have been 17.
+
+ * automake.mk: Add new files.
+
+ * format-guesser.c: New file.
+
+ * format-guesser.h: New file.
+
+2008-02-18 Ben Pfaff <blp@gnu.org>
+
+ Patch #6426. Reviewed by John Darrington.
+
+ * format.c (min_width): Renamed fmt_min_width and made public.
+ Updated all references.
+ (max_width): Renamed fmt_max_width and made public. Updated all
+ references.
+ (max_decimals): Renamed fmt_max_decimals and made public. Updated
+ all references.
+ (var_create): Use the new functions for default variable
+ attributes below.
+ (var_default_formats): New function.
+ (var_default_measure): New function.
+ (var_default_alignment): New function.
+
+ * format.h (macro FMT_MAX_NUMERIC_WIDTH): New macro.
+
+2008-02-09 Ben Pfaff <blp@gnu.org>
+
+ Add a couple of extensions to GET DATA TYPE=TXT. Patch #6412.
+ Thanks to John Darrington for review.
+
+ * data-in.c (data_in): Add new argument to designate the last
+ column of the data field being parsed, for use in error messages.
+ Update all callers.
+
+ * data-parser (struct data_parser): New member `quote_escape'.
+ (data_parser_create): Initialize quote_escape.
+ (data_parser_set_quotes): New function.
+ (cut_field): Support escaped quotes.
+ (parse_delimited_span): Ditto.
+ (parse_delimited_no_span): Ditto.
+
+ * get-data.c (parse_get_txt): Support ESCAPE extension subcommand
+ in enhanced mode. Only support multiple quote characters in
+ enhanced mode.
+
+2008-02-06 John Darrington <john@darrington.wattle.id.au>
+
+ psql-reader.c psql-reader.h: Read more than one tuple at
+ once. Fix bug reading a query which returns no data. Fix bug
+ when transformation followed a reader.
+ Ask the server for the number of records in the query, for the
+ benefit of the gui.
+
+2008-02-05 John Darrington <john@darrington.wattle.id.au>
+
+ psql-reader.c: So yesterday they release postgresql 8.3.0
+ which has money represented with 64 bits. They must get
+ paid more than me.
+
+2008-02-02 John Darrington <john@darrington.wattle.id.au>
+
+ psql-reader.c psql-reader.h: New files. Thanks to Ben Pfaff
+ for reviewing this code.
+
+2008-02-02 Ben Pfaff <blp@gnu.org>
+
+ Patch #6347.
+
+ * sys-file-reader.c (read_variable_record): Allow missing values
+ to be specified on long string variables, but warn about them
+ because PSPP does not yet support them.
+ (read_extension_record): Ignore extension records 20 and 21, which
+ PSPP does not yet support.
+ (read_header): Fix error message when floating-point format cannot
+ be identified.
+
+2008-02-01 Ben Pfaff <blp@gnu.org>
+
+ Patch #6386. Thanks to John Darrington for review and for the
+ updates to gnumeric-reader.c.
+
+ * dictionary.c (make_hinted_name): New function.
+ (make_numeric_name): New function.
+ (dict_make_unique_var_name): New function.
+
+ * gnumeric-reader.c (devise_name): Removed.
+ (munge_name): Removed.
+ (gnumeric_open_reader): Use new function
+ dict_make_unique_var_name.
+
+ * short-names.c (set_var_short_name_suffix): Use new function
+ str_format_26adic.
+
+2008-01-19 John Darrington <john@darrington.wattle.id.au>
+
+ * settings.c settings.h: Moved static variables into a
+ single struct. Renamed functions to have a settings_ prefix.
+
+2008-01-14 John Darrington <john@darrington.wattle.id.au>
+
+ * data-out.c (output_AHEX): Corrected number of bytes in
+ call to output_hex. Closes bug #22011
+
+2008-01-02 John Darrington <john@darrington.wattle.id.au>
+
+ * variable.c variable.h: Replaced var_get_value_name with
+ var_append_value_name which doesn't use any static data.
+ Thanks to Ben for review.
+
+2007-12-07 Ben Pfaff <blp@gnu.org>
+
+ Patch #6302.
+
+ * casegrouper.c (casegrouper_get_next_group): Cause a casegrouper
+ made from an empty casereader produce a casegrouper with no
+ groups, instead of one with one group that has no cases.
+
+ * casereader.c (casereader_is_empty): New function.
+
+2007-12-06 Ben Pfaff <blp@gnu.org>
+
+ Patch #6303.
+
+ * sys-file-reader.c (read_display_parameters): Handle variable
+ display parameters record with only 2 data items per variable.
+ Reported by Guido Gay <gay@irer.it>.
+
+2007-12-04 Ben Pfaff <blp@gnu.org>
+
+ * identifier.c (lex_id_match_n): New function.
+ (lex_id_match): Reimplement in terms of lex_id_match_n.
+
+2007-11-24 Ben Pfaff <blp@gnu.org>
+
+ * automake.mk (src_data_libdata_a_SOURCES): Add val-type.h, to fix
+ make distcheck.
+
+2007-11-24 Ben Pfaff <blp@gnu.org>
+
+ Fix warning reported when reading back system files that include
+ very long string variables. Thanks to Guido Gay <gay@irer.it> for
+ reporting this bug.
+
+ * short-names.c (short_names_assign): Fix dumb typo, in which `i'
+ was written where `j' was meant.
+
+2007-11-18 Ben Pfaff <blp@gnu.org>
+
+ Properly write variables that include a range of missing values to
+ system files. Thanks to Guido Gay <gay@irer.it> for reporting
+ this bug.
+
+ * sys-file-writer.c (write_variable): Correctly calculate
+ n_missing_values field when writing variables that include a range
+ of missing values.
+
+2007-11-10 Ben Pfaff <blp@gnu.org>
+
+ Cleanups and bug fixes devised while writing up documentation.
+ Patch #6262.
+
+ * automake.mk (src_data_libdata_a_SOURCES): Add new files.
+
+ * dict-class.c: New file.
+ (dict_class_from_id): Move here.
+ (dict_class_to_name): Move here.
+
+ * dict-class.h: New file.
+ (enum dict_class): Move here. Change from consecutive integers to
+ consecutive bits, to make testing for any of multiple values
+ easier. Add new DC_ALL constant.
+
+ * dictionary.c (struct dictionary): Change `case_limit' from
+ size_t to casenumber.
+ (dict_get_vars): Make final argument an enum dict_class.
+ (dict_get_vars_mutable): Ditto.
+ (dict_get_case_limit): Change return value to casenumber.
+ (dict_set_case_limit): Change final argument to a casenumber.
+ (dict_unset_split_var): Add assertion. Rephrase slightly.
+ (dict_set_label): Use xstrndup to simplify.
+
+ * format.c (fmt_step_width): AHEX format also needs 2-byte
+ stepping.
+ (fmt_set_style): Simplify assertions.
+
+ * missing-values.c (mv_add_num_range): Rename mv_add_range.
+ Simplify implementation.
+ (mv_has_value): Simplify implementation.
+ (mv_pop_value): Remove the first value, not the last, to avoid
+ having GET followed by SAVE reverse the order of missing values.
+ (mv_peek_value): Rename mv_get_value. Simplify assertion.
+ (mv_has_range): Simplify implementation.
+ (mv_peek_range): Rename mv_get_range.
+ (can_resize_string): Removed.
+ (mv_is_resizable): Use value_is_resizable.
+ (mv_resize): Use value_resize.
+
+ * short-names.h (SHORT_NAME_LEN): Move here.
+
+ * val-type.h: New file, for definitions related to type and width
+ of abstract values. Before, these definitions were mixed among
+ those related to "union value" and those related to variables.
+ (macro SYSMIS): Move here.
+ (macro LOWEST): Move here.
+ (macro HIGHEST): Move here.
+ (macro MAX_STRING); Move here.
+ (enum val_type): New enum with values VAL_NUMERIC and VAL_STRING.
+ Replaces enum var_type that had values VAR_NUMERIC and VAR_STRING.
+ All references updated.
+ (val_type_is_valid): New function. Replaces var_type_is_valid.
+ All references updated.
+ (val_type_from_width): New function. Replaces
+ var_type_from_width. All references updated.
+
+ * value-labels.c (val_labs_copy): Renamed val_labs_clone. All
+ references updated.
+ (val_labs_can_set_width): Use value_is_resizable.
+ (val_labs_add): Simply return false if the value labels set is too
+ wide, instead of having undefined behavior.
+ (val_labs_replace): Ditto.
+ (val_labs_replace): Ditto.
+ (val_labs_first): Set iterator to null if iteration is complete.
+ (val_labs_first_sorted): Ditto.
+ (val_labs_done): Become a no-op if the iterator is null.
+
+ * value.c (value_is_resizable): New function.
+ (value_resize): New function.
+
+ * variable.c (var_get_dict_class): New function.
+
+ * variable.h (macro LONG_NAME_LEN): Rename VAR_NAME_LEN. Update
+ all references.
+
+2007-11-08 Ben Pfaff <blp@gnu.org>
+
+ * data-in.c: Make formatted data parsing locale-independent.
+ (parse_number): Use c_strtod instead of strtod, to avoid
+ locale-specific behavior.
+ (parse_Z): Ditto.
+
+2007-11-06 Ben Pfaff <blp@gnu.org>
+
+ Patch #6256: add support for binary, 360 file formats. Reviewed
+ by John Darrington.
+
+ * data-in.c (struct data_in): Add `encoding' member.
+ (data_in): Add `encoding' parameter, and re-encode the data passed
+ in where appropriate. Update all callers to pass it in.
+ (parse_A): Implement EBCDIC recoding wart described in manual.
+ (parse_AHEX): Implement EBCDIC recoding.
+
+ * data-out.c (data_out_legacy): New function.
+ (data_out): Make into a wrapper around data_out_legacy.
+
+ * file-handle-def.c (struct file_handle): New member `encoding'.
+ (fh_create_file): Set encoding.
+ (fh_default_properties): Set default encoding.
+ (fh_get_legacy_encoding): New function.
+
+ * file-handle-def.h (enum fh_mode): New modes FH_MODE_FIXED
+ (that replaces FH_MODE_BINARY), FH_MODE_VARIABLE,
+ FH_MODE_360_VARIABLE, FH_MODE_360_SPANNED.
+ (struct fh_properties): New member `encoding'.
+
+2007-11-05 Ben Pfaff <blp@gnu.org>
+
+ Patch #6258. Reviewed by John Darrington.
+
+ * file-handle-def.c (fh_lock): Add comment that TYPE should be
+ marked with N_() in the caller. Added these markings to each
+ caller too. Should make i18n easier.
+ Suggested by Chusslove Illich <caslav.ilic@gmx.net>.
+
+2007-11-03 Ben Pfaff <blp@gnu.org>
+
+ Allow output files to overwrite input files (bug #21280).
+
+ * file-handle-def.c: Separate locking of files for input and for
+ output, to allow both to take place at once. Also, distinguish a
+ file handle from the identity of the underlying file, because the
+ identity of a file changes over time and the file handle can't
+ represent two different identities.
+ (struct file_handle): Remove `next', `open_cnt', `deleted',
+ `type', `open_mode', `aux', `identity' members. Change `id' from
+ char array to char *. Add `ref_cnt' member.
+ (file_handle_from_ll) New function.
+ (file_handles) Removed.
+ (named_handles) New variable.
+ (fh_init) Initialize named_handles.
+ (fh_done) Remove name from all named_handles.
+ (free_handle) Updated for modified struct file_handle.
+ (unname_handle) New function.
+ (fh_ref) New function.
+ (fh_from_file_name) Removed.
+ (fh_unref) New function.
+ (fh_unname) New function.
+ (fh_from_id) Rewritten.
+ (create_handle) Updated for modified struct file_handle.
+ (fh_inline_file) Increment the handle's ref count.
+ (fh_create_file) Updated for modified struct file_handle.
+ (fh_create_scratch) Ditto.
+ (fh_free) Removed.
+ (mode_name) Removed.
+ (fh_open) Removed.
+ (fh_close) Removed.
+ (fh_is_open) Removed.
+ (fh_get_id) Updated for modified struct file_handle.
+ (fh_get_default_handle) Increment the handle's ref count.
+ (fh_set_default_handle) Handle ref counts.
+ (struct fh_lock) New structure.
+ (locks) New static var.
+ (fh_lock) New function.
+ (fh_unlock) New function.
+ (fh_lock_get_aux) New function.
+ (fh_lock_set_aux) New function.
+ (fh_is_locked) New function.
+ (make_key) New function.
+ (free_key) New function.
+ (compare_fh_locks) New function.
+ (hash_fh_lock) New function.
+
+ * file-handle-def.h (enum fh_access) New enum.
+
+ * file-name.c: Made file_identity the same in all supported
+ environments.
+ (struct file_identity): New `name' member.
+ (fn_get_identity): For a file that doesn't exist, get the
+ dev/inode of its directory plus its name. If even the directory
+ doesn't exist, just use its name. Merge the Windows
+ implementation into the Unix one.
+ (fn_compare_file_identities): Rewritten. Merge the Windows
+ implementation into the Unix one.
+ (fn_hash_identity): New function.
+
+ * make-file.c (struct replace_file): New structure.
+ (all_files): New static var.
+ (replace_file_start): New function.
+ (replace_file_commit): New function.
+ (replace_file_abort): New function.
+ (free_replace_file): New function.
+ (unlink_replace_files): New function.
+
+ * por-file-reader.c (struct pfm_reader): Add `lock' member.
+ (close_reader): Unlock file.
+ (pfm_open_reader): Lock file.
+
+ * por-file-writer.c (struct pfm_writer): Add fh_lock, replace_file
+ members.
+ (pfm_open_writer): Lock file and prepare for its replacement.
+ (close_writer): Unlock file.
+
+ * scratch-handle.h (struct scratch_handle): Add unique_id so that
+ different generations of a scratch file can be distinguished.
+
+ * scratch-reader.c (scratch_reader_open): Verify that the file is
+ a scratch file.
+
+ * scratch-writer.c (struct scratch_writer): Add `lock' and `dict',
+ remove scratch_handle member.
+ (scratch_writer_open): Lock handle. Prepare to replace handle
+ data, instead of doing it immediately.
+ (scratch_writer_casewriter_destroy): Replace handle data and
+ unlock handle.
+
+ * sys-file-reader.c (struct sfm_reader): Add `lock' member.
+ (sfm_open_reader): Lock file.
+ (close_reader): Unlock file.
+
+ * sys-file-writer.c (struct sfm_writer): Add fh_lock, replace_file
+ members.
+ (sfm_open_writer): Lock file and prepare for its replacement.
+ (close_writer): Unlock file.
+
+2007-11-02 Ben Pfaff <blp@gnu.org>
+
+ * data-out.c (output_number): Use isfinite (from C99) instead of
+ gsl_isfinite, since we now have universal support for it in
+ gnulib.
+ (output_infinite): Ditto.
+ [!HAVE_ROUND] (round): Remove definition, since we now have a
+ replacement in gnulib.
+
+2007-10-27 John Darrington <john@darrington.wattle.id.au>
+
+ * dictionary.c: Fixed bug in dict_clone, where the vardict.dict member
+ wasn't initialised correctly. Closes bugs #21330 and 21397
+
+
+2007-10-08 Ben Pfaff <blp@gnu.org>
+
+ Fix bug #21280. Thanks to John Darrington for review.
+
+ * file-name.c (create_stream): New function.
+
+ * por-file-writer.c (pfm_open_writer): Use fh_open to open the
+ file handle before creating the file, to ensure that we don't
+ truncate a file that we're reading. Make code easier to read by
+ using create_stream.
+
+ * sys-file-write.c (sfm_open_writer): Ditto.
+
+2007-10-01 Ben Pfaff <blp@gnu.org>
+
+ Fix bug #21192. Thanks to John Darrington for review.
+
+ * casereader.c (casereader_read): Decrement case_cnt before
+ calling the casereader's "read" member function, so that we
+ interact properly with lazy_casereader.
+
+ * datasheet.c: Add regression test for above bug fix.
+ (clone_datasheet): New function.
+ (lazy_callback): New function.
+ (check_datasheet_casereader): New function.
+ (check_datasheet): Check datasheet contents are reported correctly
+ through an ordinary casereader and a lazy casereader.
+ (clone_model): Use clone_datasheet.
+
+2007-09-24 Ben Pfaff <blp@gnu.org>
+
+ Patch #6210. Reviewed by John Darrington.
+ * settings.c: Drop "long view", which is not needed any longer.
+ (static var long_view): Removed.
+ (force_long_view): Removed.
+ (get_viewwidth): Removed.
+ (init_viewport): Removed long_view reference.
+
+2007-09-19 John Darrington <john@darrington.wattle.id.au>
+
+ * settings.c settings.h: Changed viewport's length and width to be
+ owned by the user interface which uses the data library. This allows
+ better abstraction, and makes dynamically adjustable dimensions easier.
+
+2007-09-18 Ben Pfaff <blp@gnu.org>
+
+ * procedure.c (proc_extract_active_file_data): New function.
+
+ * lazy-casereader.h: New file.
+
+ * lazy-casereader.c: New file.
+
+ * casereader.c (casereader_dynamic_cast): New function.
+
+2007-09-14 Ben Pfaff <blp@gnu.org>
+
+ * dictionary.c (dict_clone): Copy case indexes from cloned
+ dictionary. Fixes bug #21061. Reviewed by John Darrington.
+
+2007-09-13 John Darrington <john@darrington.wattle.id.au>
+
+ * value.c value.h (create_value): New function.
+
+2007-09-12 Ben Pfaff <blp@gnu.org>
+
+ Make it clear that translator casereader and casewriter translate
+ functions are supposed to destroy their input case.
+
+ * casereader-translator.c (struct casereader_translator): Change
+ input case parameter of translate member function from const
+ struct ccase * to struct ccase *.
+ (casereader_create_translator): Ditto, for translate parameter.
+
+ * casewriter-translator.c (struct casewriter_translator): Ditto.
+ (casewriter_create_translator): Ditto.
+
+2007-08-27 John Darrington <john@darrington.wattle.id.au>
+
+ * sys-file-reader.c (read_display_parameters): Force display width
+ to 8, if the sys file says 0 (like SPSS does).
+
+2007-08-12 Ben Pfaff <blp@gnu.org>
+
+ * dictionary.c (dict_dump): New function.
+
+2007-08-12 Ben Pfaff <blp@gnu.org>
+
+ Drop dict_compactor in favor of using the new struct case_map.
+
+ * dictionary.c (struct copy_map): Removed.
+ (struct dict_compactor): Removed.
+ (dict_make_compactor): Removed.
+ (dict_compactor_compact): Removed.
+ (dict_compactor_destroy): Removed.
+
+ * procedure.c (struct data_set): Change `compactor' member to be a
+ struct case_map *.
+ (proc_open): Use case_map_to_compact_dict instead of
+ dict_make_compactor.
+ (proc_casereader_read): Use case_map_execute instead of
+ dict_compactor_compact.
+ (proc_commit): Use case_map_destroy instead of
+ dict_compactor_destroy.
+
+ * scratch-writer.c (struct scratch_writer): Change `compactor'
+ member to be a struct case_map *.
+ (scratch_writer_open): Use case_map_to_compact_dict instead of
+ dict_make_compactor.
+ (scratch_writer_casewriter_write): Use case_map_execute instead of
+ dict_compactor_compact.
+
+2007-08-12 Ben Pfaff <blp@gnu.org>
+
+ * automake.mk: Add case-map.c, case-map.h.
+
+ * case-map.c: New file.
+
+ * case-map.h: New file.
+
+2007-08-12 Ben Pfaff <blp@gnu.org>
+
+ * dictionary.c (dict_compact_values): Don't delete scratch
+ variables as well as compacting case indexes. Update all callers.
+ (dict_get_compacted_value_cnt): Rename dict_count_values and
+ change interface. Update all callers.
+ (dict_get_compacted_value_cnt): Remove.
+ (dict_compacting_would_shrink): Remove.
+ (dict_compacting_would_change): Remove.
+ (dict_make_compactor): Add new parameter. Update all callers.
+
+ * procedure.c (proc_casereader_read): Use casewriter_get_value_cnt
+ instead of dict_count_values, changing an O(N) operation into
+ O(1).
+
2007-08-12 Ben Pfaff <blp@gnu.org>
* casereader.c (casereader_read): Don't require cases read by a