Ben Pfaff [Tue, 1 Jun 2004 04:57:11 +0000 (04:57 +0000)]
Fix memory leaks.
Generalize casefiles to the extent that we can use them for sorting
and other kinds of data transformations. Change cases to be
copy-on-write to improve memory efficiency in common cases. Every
access to a member of a `struct ccase' was changed to be a call to a
case_*() function, especially case_data(), case_num(), case_str(), or
case_data_rw(). Many instances of a local variable named "case_num"
were changed to "case_idx" as a consequence. Many `struct ccase *'
were changed to actual `struct ccase' because of copying semantics of
cases. In several places there was a choice between updating debug
code to work with the new ADTs or just deleting it because it was
useless; I chose to delete it.
Ben Pfaff [Mon, 31 May 2004 05:50:10 +0000 (05:50 +0000)]
Fully implement arbitrary delimiters on DATA LIST, extending the half
implementation that was already there.
Update our string ADTs, struct string and struct len_string. Get rid
of pool support, which was largely unused. Rename lots of functions
to have more obvious or consistent names.
pjk [Thu, 13 May 2004 18:31:13 +0000 (18:31 +0000)]
Adjusted code for DATA LIST (free/list) to be more consistent with SPSS in
the instance where delimiters are explicitly set. In particular, if a delim.
is explicitly set, we treat quotes / apostrophes as normal chars:
From SPSS Syntax Base manual:
For data with explicitly specified value delimiters (for example, DATA LIST FREE (","):
· Multiple delimiters without any intervening space can be used to specify missing data.
· The specified delimiters cannot occur within a data value, even if you enclose the value
in quotation marks or apostrophes.
configure.ac: check for presence / add libgnugetopt to support
getopt_long on FreeBSD
algorithm.c, alloc.h: Wrap includes of alloca.h with #ifdef
HAVE_ALLOCA_H. Some platforms (FreeBSD for example) have alloca in
stdlib.h... Autoconf detects this in AC_FUNC_ALLOCA
- Fixed bug in pool (free_all_gizmos) where gizmos pointer wasn't being
NULLed after we freed all of the gizmos
- Fixed bug in vfm (cancel_transformations) where we didn't NULL t_trns after
freeing it
- Updated routines around variable labels to allow 255 char. variable labels
(this is supported by SPSS 11.0 at least...) This doesn't appear to break
backward compat. Let me know if it does.
Ben Pfaff [Tue, 27 Apr 2004 05:54:31 +0000 (05:54 +0000)]
We're abusing the current ASCII driver by telling it to allocate a
9999-line, 9999-character page in the tests. This causes some systems
to curl up and die because it allocates 20 MB of contiguous RAM. This
change alleviates at least part of the problem. It is mostly a
stop-gap until the new output system is ready.
Fix bug where a copy dest spec would cause a failed assertion in strdup
Fix bug where a copy dest spec would cause the wrong member of the union
to be copied
Ben Pfaff [Sat, 17 Apr 2004 07:02:10 +0000 (07:02 +0000)]
Contrary to what I'd always understood, there is an efficient
algorithm for deletion from a hash table populated via linear probing.
This change implements it.
Ben Pfaff [Sun, 11 Apr 2004 21:32:00 +0000 (21:32 +0000)]
Rework moments routines for improved numerical stability based on
Michael Kiefte's advice. Any bugs or remaining numerical problems are
still mine though.
There is now a struct moments1 for use with one-pass moments. It uses
a provisional means algorithm as an attempt to improve accuracy of
higher moments. The older struct moments now only handles two-pass
moments.
Michael Kiefte [Fri, 9 Apr 2004 09:58:47 +0000 (09:58 +0000)]
* descript.c:
fixed problem with parsing in match_statistic() causing "DESCRIPTIVE
STAT=MEAN." to barf.
"MEAN" is now default if "SORT" given without specification.
fixed infinite loop with "DESCRIPT GIBBERISH=ALL."
parsing is generally less forgiving of syntax errors: better to have
it do nothing and type it in again then to not know what it actually did instead
(problem also related to infinite loop problem above).
z-score transformation now check score for user-missing values and ch
ecks std_dev for SYSMIS.
Michael Kiefte [Tue, 6 Apr 2004 18:14:36 +0000 (18:14 +0000)]
* Changed dict_get_case_weight() to accept an additional int * flag to complain about system-missing, user-missing, zero, or negative weights.
* Updated existing functions to pass int * to dict_get_case_weight().
Ben Pfaff [Tue, 30 Mar 2004 00:52:36 +0000 (00:52 +0000)]
Add multipass procedures. Add two-pass moments calculation. Rewrite
DESCRIPTIVES as a multipass procedure to use two-pass moments. Fix
some memory leaks.
Ben Pfaff [Tue, 30 Mar 2004 00:42:46 +0000 (00:42 +0000)]
Add multipass procedures. Add two-pass moments calculation. Rewrite
DESCRIPTIVES as a multipass procedure to use two-pass moments. Fix
some memory leaks.
Ben Pfaff [Sun, 21 Mar 2004 03:06:11 +0000 (03:06 +0000)]
Changed DFM from open-at-first-access to explicit-open. Before,
calling dfm_get_record() or dfm_put_record() would automatically
open the file. Now, you have to call dfm_open_for_reading() or
dfm_open_for_writing() explicitly. This makes it possible to
check permissions, file existence, etc. earlier.
Also made struct file_handle more opaque, and clean up in general.
Fixed cmd_parse() so that it always skips past a full command
name. A few special commands for which this would be bad get
special treatment. This lets us drop code for skipping past the
end of a command name in most cmd_*() functions. It's not worth
listing all the commands affected.
Ben Pfaff [Mon, 15 Mar 2004 07:09:38 +0000 (07:09 +0000)]
Redo VFM interface. Get rid of compaction_necessary, compaction_nval,
compaction_case. Replace disk_sink and memory_sink by storage_sink,
disk_source and memory_source by storage_source.
Ben Pfaff [Tue, 17 Feb 2004 06:19:35 +0000 (06:19 +0000)]
Add an Emacs header line to output files that makes generated .c files
read-only by default, to make it difficult to accidentally change
generated files.
FREQUENCIES: Compare by frequency, not bogus a->v.c <=> b->v.c pointer
compare.
Changed data_out() to store string data directly into a `union
value''s s member, not indirectly into c.
Renamed some confusingly named member objects in crosstabs.q.