Jason H Stover [Wed, 15 Oct 2008 16:12:35 +0000 (12:12 -0400)]
glm.q: Removed code to us QR decomposition, which requires the entire data
set to be stored in a matrix.
glm.q: fit_model: New function.
design-matrix.[ch]: Added array of size_t's to store the number of
data for each variable in struct design_matrix. Added accessor
functions design_matrix_increment_case_count,
design_matrix_set_case_count and design_matrix_get_case_count.
Jason H Stover [Wed, 15 Oct 2008 14:06:50 +0000 (10:06 -0400)]
covariance_accumulate: New function for one-pass computation of the
covariance matrix.
Added hash table to temporarily store the elements that will be in the
covariance matrix. The hash consists of the new data structure
covariance_accumulator. New functions hash_numeric_alpha,
covariance_accumulator_compare, match_nodes,
covariance_accumulator_free, covariance_hsh_create,
covariance_accumulator_hash, to assist with the hashing.
update_product: New function.
covariance_accumulator_to_matrix: New function to put hash elements
into the covariance matrix.
get_center: New function to center a value prior to adding it to
the covariance matrix.
covariance_matrix_insert: New function to add a value to the
covariance matrix.
Ben Pfaff [Thu, 9 Oct 2008 05:25:36 +0000 (22:25 -0700)]
Make [ and ] valid tokens in the lexer.
The VARIABLE ATTRIBUTE and DATAFILE ATTRIBUTE commands use [ and ]
to designate array elements. This is the first use for these tokens
in the language implemented by PSPP, so until now these characters
were rejected with an error whenever they appeared in a syntax file
outside a comment or a quoted string.
Ben Pfaff [Sun, 5 Oct 2008 18:17:10 +0000 (11:17 -0700)]
Use Gnulib module "regex", to fix linking on Mingw.
The GUI code uses POSIX regular expressions, which Mingw has in a separate
library (-lregex). We could just search for that library, but by including
our own implementation (via Gnulib) we also work around known bugs in
system regular expression implementations.
Ben Pfaff [Sat, 4 Oct 2008 04:01:50 +0000 (21:01 -0700)]
Also warn about missing prerequisites as we encounter them (bug #24445).
It can happen that configure bails out in the middle due to some issue
that PSPP cannot directly control; for example, AC_CHECK_SIZEOF exits
unconditionally if it fails. If this happens then, before, we would fail
to warn about missing required prerequisites that could well have been
the actual cause of the failure, because we only warn about these at
the end of the run.
Now, we warn about missing prerequisites both at time of detection and
in a summary at the end, to head off this problem.
Thanks to Jim Orr for allowing me to notice the problem.
Ben Pfaff [Sat, 4 Oct 2008 04:00:55 +0000 (21:00 -0700)]
Add note to INSTALL explaining how to use libraries from /usr/local.
Several users have had trouble building PSPP with self-built GSL, etc.
that were installed to /usr/local. Explain how to do it here to be
more friendly to Unix novices.
Thanks to Jim Orr for making me realize that this was important.
Ben Pfaff [Sat, 27 Sep 2008 03:42:01 +0000 (20:42 -0700)]
New hmap and hmapx hash table implementations.
These new hash table implementations should yield better performance
than the older one, for at least two reasons. First, they are based
on "separate chaining" rather than "open addressing", and thus do not
suffer from clustering, which is likely to be an issue with the hash
functions that we have been using. Second, they are carefully written
to generate simple code that should inline well.
Also, move the existing hash functions into a new pair of files,
src/lib/hash-functions.[ch]. This allows users of the new hash tables
to use them without including the older hash table header.
John Darrington [Tue, 30 Sep 2008 02:30:13 +0000 (10:30 +0800)]
Updated the description of the --with-gui-tools flag.
Some users were misunderstanding the purpose of this flag,
and using it in normal builds, and suffering the consequences.
This description, hopefully, makes it clear that they don't
need it.
John Darrington [Tue, 23 Sep 2008 01:14:51 +0000 (09:14 +0800)]
Fixed minor build problems.
Fixed two build problems which crept in when moving to
libtool libraries. One manifested itslef when building
on a system with ncurses in a non-standard place. The
other when building after configured with --enable-debug
John Darrington [Sat, 20 Sep 2008 05:29:41 +0000 (13:29 +0800)]
Moved all static libraries to libtool libraries.
By default, this means that both pspp and psppire
will use two shared libraries. If static libraries
are desired, then this can be achieved by configuring
with LDFLAGS=-static-libtool-libs
Jason H Stover [Wed, 17 Sep 2008 15:36:41 +0000 (11:36 -0400)]
coeff_init: Do not use coeff[0] as the intercept.
rearrange_covariance_matrix: Fix permutation of rows/columns.
run_glm: Account for categorical values.
cmd_glm: Drop double-allocation of model
linreg.h: Declare pspp_linreg_with_cov().
Jason H Stover [Mon, 15 Sep 2008 16:35:11 +0000 (12:35 -0400)]
pspp_coeff_var_to_coeff: Guard against a null pointer in coefs[i]->v_info.
regression.q: Pass variable lists to pspp_linreg_cache_alloc.
pspp_linreg_cache_alloc: Take variable lists as arguments to allow easier access
later and computation of number of coefficients during allocation.
rearrange_covariance_matrix: Use the new stored list of variables in the linreg_cache
instead of calling pspp_linreg_get_vars().
cache_init: Move computation of n_coeffs to pspp_linreg_cache_alloc().
John Darrington [Sat, 13 Sep 2008 00:47:28 +0000 (08:47 +0800)]
Added new casetranslator for appending ranks.
New function casereader_create_append_rank, which
returns a translating casereader which appends the
rank of the variable upon which the casereader is
sorted.
John Darrington [Sat, 13 Sep 2008 00:37:08 +0000 (08:37 +0800)]
New function: var_create_internal.
Added a new function for creating "internal" variables.
Ie, those which don't appear in any dictionary, and
are for the purposes of internal calculations.
John Darrington [Sun, 31 Aug 2008 06:41:25 +0000 (14:41 +0800)]
Add n_missing argument to casereader_create_filter_missing
Add a new argument to casereader_create_filter_missing, which,
if non-null, receives the number of cases filtered out by the
new reader. Update all callers of this function.
John Darrington [Sun, 24 Aug 2008 05:22:02 +0000 (13:22 +0800)]
Fixed some issues calculating percentiles when missing values are involved.
Added an extra argument to order_stats_accumulate, to indicate which classes
of missing values should be excluded. Added an internal cumulative weight
counter, to ensure that the declared total weight agrees with that which is
encountered.
John Darrington [Sat, 23 Aug 2008 06:08:28 +0000 (14:08 +0800)]
Rewrote the EXAMINE command.
Made use of the new casegrouper construct instead of relying on hashes.
Rewrote the percentiles calculations so as not to require dynamically
allocating memory for all the data. Encapsulated many of the other statistics'
calculations, so as to have similar interfaces.
John Darrington [Fri, 5 Sep 2008 02:04:40 +0000 (10:04 +0800)]
Oops. Neglected to add new files.
Actually I'm *sure* that I did do "git add" on all these,
but for whatever reason git didn't check them in.
Hopefully it's right this time. One day I'll work out
how to use git effectively. One day ...
John Darrington [Thu, 4 Sep 2008 11:57:06 +0000 (19:57 +0800)]
Complete re-implementation of the EXAMINE command.
The implementation now avoids storing large amounts of data on the
heap. Instead it uses the casereader functionality and
the casegrouper structure instead of hashes to store group data.
Added documentation for the /ID subcommand which was missing.
Better encapsulated the calculation of boxplot data, extreme
values, histograms, NP plots and percentiles and others. Made
the start of a common base interface for statistic calculations.
Also updated the frequencies code to fit new interfaces.
John Darrington [Sat, 23 Aug 2008 00:44:32 +0000 (08:44 +0800)]
Work around bug #24033 (appending to output in cygwin).
Cygwin (and perhaps other w32 platforms) cannot write to a file
whilst it is open for reading. This kludge therefore closes the
output after pasting into the output viewer and re-opens when new
data is appended. It'll be slow if the output file becomes large.
Ben Pfaff [Sun, 27 Jul 2008 18:12:45 +0000 (11:12 -0700)]
Use standard C99 isfinite, isnan, isinf in place of GSL substitutes.
In change a9afcdd22, "Use gsl_isnan instead of isnan, ...," isfinite,
isnan, and isinf were changed to use the GSL substitutes because of
lack of portability of the C99 versions. Now, gnulib has portable
versions of all of these, so change them back.
This commit changes calls to gsl_finite() to call isfinite() instead
of the equivalent finite(). isfinite() is correct here: both gsl_finite()
and finite() return true for NaNs, which is a surprising result given
that a NaN is definitely not finite, but isfinite() returns false and,
more importantly, behaves as actually wanted in each place it is used
in PSPP code.
This commit requires upgrading gnulib to at least change 5183a0e3c,
"Add missing dependencies on new m4/exponent[fdl].m4 files," or later,
and re-running "make -f Smake".