1 2007-12-04 Ben Pfaff <blp@gnu.org>
3 * placement-parser.c (parse_column): New function.
4 (parse_column_range): Add `base' argument. Update all callers.
6 2007-12-04 Ben Pfaff <blp@gnu.org>
8 Make GET DATA a separate command, instead of something invoked
11 * automake.mk: Remove get-data.h from sources.
13 * get-data.h: Removed.
15 * get-data.c (parse_get_data_command): Rename cmd_get_data.
17 * get.c (parse_read_command): No longer any need to check for DATA
20 2007-12-04 Ben Pfaff <blp@gnu.org>
22 * src/language/data-io/data-reader.c (struct dfm_reader): New
23 `file_size' member to support dfm_get_percent_read.
24 (dfm_open_reader): Initialize file_size.
25 (dfm_get_percent_read): New function.
27 2007-11-08 Ben Pfaff <blp@gnu.org>
29 Patch #6256: add support for binary, 360 file formats. Reviewed
32 * data-reader.c (struct dfm_reader): New member `block_left'.
33 (dfm_open_reader): Initialize block_left. For FH_MODE_TEXT, open
34 the file in text mode.
35 (read_error): New function.
36 (partial_record): New function.
37 (try_to_read_fully): New function.
38 (enum descriptor_type): New enum.
39 (read_descriptor_word): New function.
40 (corrupt_size): New function.
41 (read_size): New function.
42 (read_file_record): Implement new modes.
43 (read_record): Now take care of tracking line numbers here.
44 (dfm_reader_get_legacy_encoding): New function.
46 * data-writer.c (dfm_put_record): Implement new modes.
47 (dfm_writer_get_legacy_encoding): New function.
49 * file-handle.q: Parse new formats.
50 (cmd_file_handle): Set up new formats.
52 * print.c (struct print_trns): New member `encoding'.
53 (internal_cmd_print): Set encoding.
54 (print_trns_proc): Recode output data if necessary.
55 (flush_records): Recode leader byte.
57 2007-11-03 Ben Pfaff <blp@gnu.org>
59 Allow output files to overwrite input files (bug #21280).
61 * data-list.c (cmd_data_list): Manage file handle reference
64 * data-reader.c (struct dfm_reader): Add `lock' member.
65 (dfm_close_reader): Simplify, as reference counting is now
66 separate from locking.
67 (dfm_open_reader): Lock file.
69 * data-writer.c (struct dfm_writer): Add fh_lock, replace_file
71 (dfm_open_writer): Lock file and prepare for replacement.
72 (dfm_close_writer): Unlock file and replace it.
74 * file-handle.q (cmd_close_file_handle): Use fh_unname.
75 (fh_parse): Don't distinguish existing handles for a given file
76 name from new ones. Manage file handle reference counts.
78 * get.c (parse_read_command): Manage file handle reference counts.
79 (parse_write_command): Ditto.
80 (mtf_close_all_files): Ditto.
82 * inpt-pgm.c (cmd_reread): Manage file handle reference counts.
84 * print-space.c (cmd_print_space): Manage file handle reference
87 * print.c (internal_cmd_print): Manage file handle reference
90 2007-11-03 John Darrington <john@darrington.wattle.id.au>
92 * get.c: Add GET DATA command variant.
94 * get-data.c get-data.h (new files): Added support for
95 GET DATA /TYPE='gnm' command.
97 2007-09-23 Ben Pfaff <blp@gnu.org>
99 Bug #21111. Reviewed by John Darrington.
101 * data-list.c (data_list_trns_proc): Properly set retval when END
102 subcommand is in use.
103 (cmd_data_list): Don't allow END subcommand to be used with DATA
106 2007-09-12 Ben Pfaff <blp@gnu.org>
108 * get.c (get_translate_case): Change input case parameter from
109 const struct ccase * to struct ccase *, to match change in
110 casereader and casewriter translators. Destroy input case, to fix
113 2007-08-12 Ben Pfaff <blp@gnu.org>
115 * get.c (parse_read_command): Compact the values in the target
116 dictionary, to save space.
118 2007-08-12 Ben Pfaff <blp@gnu.org>
120 * get.c (struct case_map): Move into new file src/data/case-map.c.
121 (start_case_map): Ditto, and rename case_map_prepare_dict.
122 (finish_case_map): Ditto, and rename case_map_from_dict.
123 (map_case): Ditto, and rename case_map_execute.
124 (destroy_case_map): Ditto, and rename case_map_destroy.
125 (case_map_get_value_cnt): Ditto.
127 2007-08-12 Ben Pfaff <blp@gnu.org>
129 * get.c (case_map_get_value_cnt): New function.
131 2007-07-25 Ben Pfaff <blp@gnu.org>
134 * data-list.c (read_from_data_list_fixed): Handle multi-record
137 2007-07-11 Ben Pfaff <blp@gnu.org>
139 * get.c (map_case): Create destination case instead of leaving it
140 undefined. Fixes bug #20285.
141 Reviewed by John Darrington.
143 2007-06-06 Ben Pfaff <blp@gnu.org>
145 * get.c: Essentially rewrite MATCH FILES to support FIRST and
148 2007-06-06 Ben Pfaff <blp@gnu.org>
150 Adapt case sources, sinks, and clients of procedure code to the
153 * data-list.c: Make DATA LIST into a casereader.
155 * get.c: Change GET, IMPORT, SAVE, EXPORT to use casereaders,
158 * inpt-pgm.c: Use caseinit code. Turn INPUT PROGRAM into a
161 * list.q: Adapt to new procedure code.
163 2007-05-06 Ben Pfaff <blp@gnu.org>
165 Abstract the documents within a dictionary a little better.
166 Thanks to John Darrington for suggestion, initial version, and
169 * get.c (mtf_merge_dictionary): Simplify creating merged document.
171 * sys-file-info.c (display_documents): Use new
172 dict_get_document_line_cnt and dict_get_document_line functions.
174 Thu Feb 1 16:56:02 2007 Ben Pfaff <blp@gnu.org>
176 * file-handle.q (fh_parse): Update to new fh_create_file
179 Sat Dec 16 22:16:18 2006 Ben Pfaff <blp@gnu.org>
181 Make it possible to pull cases from the active file with a
182 function call, instead of requiring indirection through a callback
185 * automake.mk: Removed matrix-data.c.
187 * matrix-data.c: Removed.
189 * data-list.c (data_list_source_read): Conform with new
190 case_source_class interface.
191 (data_list_source_destroy): Ditto.
193 * get.c (case_reader_source_class): Ditto.
194 (case_reader_source_destroy): Ditto.
195 (parse_output_proc): Take advantage of new procedure interface.
196 (output_proc): Removed.
197 (struct mtf_file): Add "struct ccase *" member to allow use of new
199 (cmd_match_files): Take advantage of new procedure interface.
200 (mtf_processing_finish): Removed.
201 (mtf_read_nonactive_records): Renamed mtf_read_records. Now reads
202 from every file, without any exception for the active file.
203 (mtf_compare_BY_values): Simplify for new interface.
204 (mtf_processing): Simplify for new interface.
206 * inpt-pgm.c (is_valid_state): New function.
207 (input_program_source_read): Conform with new case_source_class
209 (input_program_source_destroy): Ditto.
210 (end_case_trns_proc): Now just needs to return TRNS_END_CASE.
212 Sat Dec 9 18:43:34 2006 Ben Pfaff <blp@gnu.org>
214 * list.q (cmd_list): Use new var_create, var_destroy functions.
216 Thu Nov 30 21:51:58 2006 Ben Pfaff <blp@gnu.org>
218 * inpt-pgm.c (cmd_reread): Always return error code upon detecting
219 syntax error. Fixes bug #18419. Thanks to John Darrington for
222 Sun Nov 19 09:17:45 2006 Ben Pfaff <blp@gnu.org>
224 * data-list.c (parse_free): Follow documented (but odd) rule that
225 N format is treated as F format for free-field input.
227 * data-reader.c (read_file_record): Drop new-line character from
228 input text lines. This is symmetrical with the recently changed
229 dfm_put_record semantics.
231 Thu Nov 2 20:56:03 2006 Ben Pfaff <blp@gnu.org>
233 Implement SKIP keyword on DATA LIST. Fixes bug #17099.
235 * data-list.c: (struct data_list_pgm) Add `skip_records' members.
236 (cmd_data_list) Set skip_records based on user input.
237 (data_list_source_read) Skip records requested by user.
239 Tue Oct 31 20:04:06 2006 Ben Pfaff <blp@gnu.org>
241 * placement-parser.c: (PRS_TYPE_T) Now that struct fmt_spec uses
242 an enum fmt_type for its type member, we can't depend on the
243 ability to put negative values into that member as out-of-band
244 values, because enum fmt_type might be an unsigned type. So use
245 values around SCHAR_MAX instead, because we know that SCHAR_MAX
246 will fit into any type, signed or unsigned, and there aren't
247 nearly that many format types.
248 (parse_var_placements) Add for_input parameter to specify whether
249 we're parsing input or output formats. Update all callers.
250 (fixed_parse_columns) Ditto.
251 (fixed_parse_fortran) Ditto.
253 Tue Oct 31 18:21:48 2006 Ben Pfaff <blp@gnu.org>
255 * print-space.c (print_space_trns_proc): Let dfm_put_record add
256 the new-line character, to match dfm_put_record change below.
258 Sat Oct 28 11:57:19 2006 Ben Pfaff <blp@gnu.org>
260 * data-writer.c (struct dfm_writer): Removed `bounce' member, and
261 all references to it.
262 (dfm_put_record) Change semantics so that it adds formatting
263 itself, such as new-line characters, instead of putting that
264 responsibility on the caller. Also, pad binary records with
265 spaces instead of zeros, for compatibility.
267 * print.c (struct prt_out_spec) New member `sysmis_as_spaces'.
268 (struct print_trns) Remove `omit_new_lines' and all references,
269 since dfm_put_record() is taking care of that. Add
271 (internal_cmd_print) Allow an empty set of data to print. Set
273 (parse_specs) Allow an empty set of data to print.
274 (parse_variable_argument) Only add space with PRINT command. Set
276 (print_trns_proc) Indent records if include_prefix is set, for
277 compatibility. Output SYSMIS as spaces if sysmis_as_spaces is
278 set. Put "1" in first column if PRINT EJECT is used with an
279 external output file.
280 (flush_records) Ditto.
282 Sat Oct 28 16:19:57 WST 2006 John Darrington <john@darrington.wattle.id.au>
284 * data-reader.c: Eliminated references to extern variable getl_buf
286 Sat Aug 5 08:25:07 2006 Ben Pfaff <blp@gnu.org>
288 Fix bug #17329 in REREAD parsing, reported by John Darrington.
290 * inpt-pgm.c (cmd_reread): Fix file handle parsing.
292 Mon Jul 31 10:32:31 2006 Ben Pfaff <blp@gnu.org>
294 * print.c (parse_specs): Allow a comma between specifications.
296 Sun Jul 16 19:57:10 2006 Ben Pfaff <blp@gnu.org>
298 * automake.mk: (src_language_data_io_libdata_io_a_SOURCE) Add
299 print-space.c, placement-parser.c, placement-parser.h.
301 * data-list.c: Basically rewrote the whole thing. Broke out a lot
302 of code into placement-parser.c. Code is much cleaner now.
304 * placement-parser.c: New file.
306 * placement-parser.h: New file.
308 * print.c: Basically rewrote the whole thing. Broke out PRINT
309 SPACE into print-space.c. Code is much cleaner now.
311 * print-space.c: New file.
313 Sat Jul 1 17:39:40 2006 Ben Pfaff <blp@gnu.org>
315 Fix bug #11612, "q2c documentation does not agree with code".
317 * list.q: Audit use of q2c "+" prefixes that indicate that a
318 command may appear multiple times.
320 Sat Jul 1 20:44:22 2006 Ben Pfaff <blp@gnu.org>
322 Fix bug #15786: System File Creation crashes if directoy is
325 * get.c (parse_write_command): Check that the any_writer open
328 Tue Jun 27 22:44:28 2006 Ben Pfaff <blp@gnu.org>
330 Fix regression in command name completion reported by John
331 Darrington. Now completion is again state-dependent and occurs
332 only on the first line of a command.
334 * inpt-pgm.c: (cmd_input_program) Reading of first token in
335 command moved into cmd_parse.
337 Fri Jun 9 13:56:00 2006 Ben Pfaff <blp@gnu.org>
339 Reform string library.
341 * matrix-data.c (context): Use dynamic string.
342 (another_token) Deal with changed dfm_get_record() interface.
346 * data-list.c (struct data_list_pgm) Delete delims, delim_cnt
347 members, replacing them by struct string delims. Update all
348 references to use struct string functions.
349 (cut_field) Change interface to avoid needing "end_blank", by
350 getting the data-reader to remember that state for us. Change
351 internals to use substring. Update both callers.
353 * data-reader.c (read_file_record): Use ds_read_stream().
354 (dfm_get_record) Change interface to return substring. Updated
356 (dfm_expand_tabs) Use ds_find_char(). Now maintain position
357 relative to end-of-line. Use ds_swap().
358 (dfm_reread_record) Don't limit position by line length.
359 (dfm_column_start) Make parameter const.
360 (dfm_columns_past_end) New function.
361 (dfm_get_column) New function.
363 Thu May 25 18:26:26 WST 2006 John Darrington <john@darrington.wattle.id.au>
365 * print.c (print_trns_free): Made the code agree with the comment, by
366 not freeing PRT. Has the side effect that the command no longer
367 crashes on invalid syntax.
369 Tue May 9 20:55:46 2006 Ben Pfaff <blp@gnu.org>
371 * get.c (cmd_match_files): Fix memory leak replacing default_dict.
373 Sat May 6 22:25:09 2006 Ben Pfaff <blp@gnu.org>
377 * list.q (write_fallback_headers): (write_fallback_headers)
378 Properly record width of leader and pass it to write_varname().
380 Sat May 6 19:03:13 2006 Ben Pfaff <blp@gnu.org>
382 * get.c: (mtf_merge_dictionary) Fix value label memory leak.
384 Sat May 6 13:51:16 2006 Ben Pfaff <blp@gnu.org>
386 Use a casefile, instead of a case sink, for MATCH FILES output.
387 It's more straightforward, although it has the same effect.
389 * get.c: (struct mtf_proc) Replace `sink' case sink member by
390 `output' casefile member.
391 (cmd_match_files) Work with casefile instead of sink.
392 (mtf_processing) Add case to casefile instead of sink.
394 Sat May 6 10:43:07 2006 Ben Pfaff <blp@gnu.org>
396 Continue reforming procedure execution. In this phase, get rid of
397 the output code for SPLIT FILE groups in procedure.c, which really
398 shouldn't be doing any output. Move it into the individual
399 procedures instead. This also adds some flexibility.
401 * list.q (write_all_headers): Call output_split_file_values().
403 Wed May 3 23:00:17 2006 Ben Pfaff <blp@gnu.org>
405 Continue reforming procedure execution. In this phase, get rid of
406 many global variables, consolidating procedure execution in
407 procedure.c. Encapsulate transformations in new "struct
408 trns_chain". Also, change implementation of N OF CASES, FILTER,
409 and PROCESS IF from special cases to transformations.
411 * data-list.c: (data_list_trns_proc) Return TRNS_END_FILE at end
412 of file. (Why didn't we do this before?)
413 (cmd_match_files) Direct procedure output to null sink.
414 Use discard_variables() instead of indirect version.
416 * inpt-pgm.c: Use transformation chain.
417 (struct input_program_pgm) Add trns_chain member.
418 (cmd_input_program) Initialize trns_chain member and capture
419 transformations with proc_capture_transformations().
420 (input_program_source_read) Use trns_chain_execute().
421 (destroy_input_program) Destroy input chain.
423 Tue May 2 10:39:56 WST 2006 John Darrington <john@darrington.wattle.id.au>
425 * list.q Changed from using fixed length char buffers to struct
426 string so that any length variables can be used.
428 Mon May 1 18:21:19 2006 Ben Pfaff <blp@gnu.org>
430 Further clean up the CMD_* command result codes.
432 * (enum cmd_result_extensions) New. Add CMD_END_INPUT_PROGRAM and
433 CMD_END_CASE result codes.
434 (struct input_program_pgm) Added case_nr, write_case, wc_data
435 members for use by END CASE transformation.
436 (emit_END_CASE) New function.
437 (cmd_input_program) Interpret CMD_END_CASE by outputting an END
438 CASE transformation. If none is output by the input program
439 itself, add one automatically at the end. Change lack of
440 variables from warning to error.
441 (cmd_end_input_program) Return CMD_END_INPUT_PROGRAM instead of
443 (input_program_source_read) No longer any need to special-case END
444 CASE. Handle TRNS_DROP_CASE properly. Initialize new members in
445 inp for use by END CASE transformation.
446 (destroy_input_program) New function.
447 (input_program_source_destroy) Just call destroy_input_program().
448 (cmd_end_case) Just return CMD_END_CASE.
449 (end_case_trns_proc) No longer a stub handled by
450 input_program_source_read(). Actually output the case and
451 increment the case number.
453 Mon May 1 16:06:30 2006 Ben Pfaff <blp@gnu.org>
455 Remove vestiges of REPEATING DATA support.
457 * data-list.c: (struct rpd_num_or_var) Removed.
458 (struct repeating_data_trns) Removed.
459 (cmd_repeating_data) Removed.
460 (find_variable_input_spec) Removed.
461 (parse_num_or_var) Removed.
462 (parse_repeating_data) Removed.
463 (realize_value) Removed.
464 (struct rpd_parse_info) Removed.
465 (rpd_parse_record) Removed.
466 (repeating_data_trns_proc) Removed.
467 (repeating_data_trns_free) Removed.
468 (repeating_data_set_write_case) Removed.
471 * inpt-pgm.c: (input_program_source_read) Don't deal with
474 * data-list.h: Removed.
476 * automake.mk (src_language_data_io_libdata_io_a_SOURCES): Removed
479 Mon May 1 15:58:28 2006 Ben Pfaff <blp@gnu.org>
481 Remove vestiges of FILE TYPE support.
483 * data-list.c: (cmd_data_list) Don't check for FILE TYPE.
484 (cmd_repeating_data) Ditto.
486 * automake.mk (src_language_data_io_libdata_io_a_SOURCES): Remove
487 file-type.c, file-type.h.
489 * file-type.c: Removed.
491 * file-type.h: Removed.
493 Wed Apr 26 13:16:28 2006 Ben Pfaff <blp@gnu.org>
495 Improve the way we handle the various parsing "states". Until now
496 we've hard-coded the state transitions in the command definition
497 file, but that's error-prone and, worse, it's redundant--we can
498 figure out what state we're in anyhow. We can cleanly handle
499 INPUT PROGRAM and FILE TYPE with a nested command-processing loop.
501 * data-list.c: (cmd_data_list) Use in_file_type() or
502 in_input_program() in place of case_source_is_class() or
503 case_source_is_complex().
505 * file-type.c: NB: Not really fixed except minimally to compile,
506 because it doesn't work anyway.
507 (in_file_type) New function.
508 (cmd_record_type) No need to check that we're in FILE TYPE.
509 (cmd_end_file_type) Ditto.
510 (var file_type_source_class) Make static.
512 * get.c: (cmd_match_files) Check vfm_source instead of pgm_state.
514 * inpt-pgm.c: (in_input_program) New function.
515 (cmd_input_program) Rewrite to include nested command processing
517 (cmd_end_input_program) Just return CMD_END_SUBLOOP.
518 (var input_program_source_class) Make static.
519 (cmd_end_case) No need to check that we're in INPUT PROGRAM.
520 (cmd_end_file) Ditto.
522 * automake.mk (src_language_data_io_libdata_io_a_SOURCES): Add
523 file-type.h, inpt-pgm.h.
525 * file-type.h: New file.
527 * inpt-pgm.h: New file.
529 Tue Apr 25 13:11:55 2006 Ben Pfaff <blp@gnu.org>
531 * print.c: Don't special-case MS-DOS line terminators.
532 (macro LINE_END_WIDTH) Removed.
533 (alloc_line) Line ends are 1 byte.
534 (print_trns_proc) Just output \n for line end.
536 Sun Apr 23 22:05:58 2006 Ben Pfaff <blp@gnu.org>
538 Continue reforming error message support. In this phase, get rid
539 of message "titles" and put the message text in `struct error'.
540 Now `struct error' encapsulates a message more properly.
542 * data-list.c: (macro RPD_ERR) Removed.
543 (rpd_msg) New function. Updated all references to tmsg() to call
544 this function instead.
546 Sat Apr 15 19:38:13 2006 Ben Pfaff <blp@gnu.org>
548 Remove last users of struct file_ext so we can get rid of it
551 * data-reader.c: (struct dfm_reader) Change file member from
552 struct file_ext to FILE *. Updated all references.
553 (dfm_close_reader) Close file with fn_close() instead of
554 fn_close_ext(). Also, make a copy of the file name from the file
555 handle before closing it, because we can't extract it after we
557 (dfm_open_reader) Open file with fn_open() instead of
560 * data-writer.c: (struct dfm_writer) Change file member
561 struct file_ext to FILE *. Updated all references.
562 (dfm_close_writer) Close file with fn_close() instead of
563 fn_close_ext(). Also, make a copy of the file name from the file
564 handle before closing it, because we can't extract it after we
566 (dfm_open_writer) Open file with fn_open() instead of
569 Sat Apr 15 18:00:32 2006 Ben Pfaff <blp@gnu.org>
571 * data-list.c: Add prototype to suppress warning for
572 cmd_repeating_data().
574 Thu Mar 2 08:40:33 WST 2006 John Darrington <john@darrington.wattle.id.au>
576 * Moved files from src directory