pspp
2 years agoA few basic tests pass.
Ben Pfaff [Mon, 17 Jan 2022 06:28:32 +0000 (22:28 -0800)]
A few basic tests pass.

2 years agofix one bug with clabels
Ben Pfaff [Mon, 17 Jan 2022 04:22:55 +0000 (20:22 -0800)]
fix one bug with clabels

2 years agofix parsing
Ben Pfaff [Mon, 17 Jan 2022 03:21:56 +0000 (19:21 -0800)]
fix parsing

2 years agobasics work now
Ben Pfaff [Sun, 16 Jan 2022 05:21:02 +0000 (21:21 -0800)]
basics work now

2 years agonumbers back in tables
Ben Pfaff [Sun, 16 Jan 2022 04:50:36 +0000 (20:50 -0800)]
numbers back in tables

2 years agowe don't really need two forks of the output code
Ben Pfaff [Sun, 16 Jan 2022 04:14:49 +0000 (20:14 -0800)]
we don't really need two forks of the output code

2 years agolabeling, with clabels and slabels, all seems to work right
Ben Pfaff [Sun, 16 Jan 2022 03:48:11 +0000 (19:48 -0800)]
labeling, with clabels and slabels, all seems to work right

2 years agobegin refactoring how to do grouping
Ben Pfaff [Sun, 16 Jan 2022 03:07:07 +0000 (19:07 -0800)]
begin refactoring how to do grouping

2 years agobetter comment
Ben Pfaff [Sun, 16 Jan 2022 00:18:18 +0000 (16:18 -0800)]
better comment

2 years agomake rowlabels and collabels more regular
Ben Pfaff [Sun, 16 Jan 2022 00:06:06 +0000 (16:06 -0800)]
make rowlabels and collabels more regular

2 years agocheck label positions for =OPPOSITE or =LAYER
Ben Pfaff [Sat, 15 Jan 2022 21:35:13 +0000 (13:35 -0800)]
check label positions for =OPPOSITE or =LAYER

2 years agomove code around
Ben Pfaff [Sat, 15 Jan 2022 20:11:23 +0000 (12:11 -0800)]
move code around

2 years agominor improvements and cleanups
Ben Pfaff [Sat, 15 Jan 2022 19:52:12 +0000 (11:52 -0800)]
minor improvements and cleanups

2 years agoStart work on slabels != summary_axis.
Ben Pfaff [Sat, 15 Jan 2022 19:03:21 +0000 (11:03 -0800)]
Start work on slabels != summary_axis.

2 years agomore debug prints
Ben Pfaff [Sat, 15 Jan 2022 18:51:55 +0000 (10:51 -0800)]
more debug prints

2 years agomerging works
Ben Pfaff [Sat, 15 Jan 2022 07:24:53 +0000 (23:24 -0800)]
merging works

2 years agoFix parsing totals.
Ben Pfaff [Sat, 15 Jan 2022 06:08:52 +0000 (22:08 -0800)]
Fix parsing totals.

2 years agoMake all the spec_sets distinct.
Ben Pfaff [Sat, 15 Jan 2022 05:56:35 +0000 (21:56 -0800)]
Make all the spec_sets distinct.

2 years agoparallel members to arrays
Ben Pfaff [Sat, 15 Jan 2022 05:33:57 +0000 (21:33 -0800)]
parallel members to arrays

2 years agoimprove naming
Ben Pfaff [Sat, 15 Jan 2022 05:06:17 +0000 (21:06 -0800)]
improve naming

2 years agoBreak ctables_cell_insert__() from ctables_cell_add__()
Ben Pfaff [Sat, 15 Jan 2022 05:03:11 +0000 (21:03 -0800)]
Break ctables_cell_insert__() from ctables_cell_add__()

2 years agoRename var_array to nest.
Ben Pfaff [Sat, 15 Jan 2022 04:51:12 +0000 (20:51 -0800)]
Rename var_array to nest.

2 years agoRename var_array2 to stack.
Ben Pfaff [Sat, 15 Jan 2022 04:44:36 +0000 (20:44 -0800)]
Rename var_array2 to stack.

2 years agoAllow totals to have different statistics
Ben Pfaff [Sat, 15 Jan 2022 02:44:01 +0000 (18:44 -0800)]
Allow totals to have different statistics

2 years agowork on configurable summaries for totals
Ben Pfaff [Thu, 13 Jan 2022 05:51:10 +0000 (21:51 -0800)]
work on configurable summaries for totals

2 years agosubtotal and hsubtotal work
Ben Pfaff [Thu, 6 Jan 2022 04:55:45 +0000 (20:55 -0800)]
subtotal and hsubtotal work

2 years agototals can go first as well as last
Ben Pfaff [Tue, 4 Jan 2022 07:04:00 +0000 (23:04 -0800)]
totals can go first as well as last

2 years agoFix output to spv.
Ben Pfaff [Tue, 4 Jan 2022 06:57:39 +0000 (22:57 -0800)]
Fix output to spv.

2 years agonested totals
Ben Pfaff [Tue, 4 Jan 2022 06:55:15 +0000 (22:55 -0800)]
nested totals

2 years agosingle totals work
Ben Pfaff [Tue, 4 Jan 2022 06:52:57 +0000 (22:52 -0800)]
single totals work

2 years agoUnify categories as the "explicit" categories.
Ben Pfaff [Tue, 4 Jan 2022 06:02:08 +0000 (22:02 -0800)]
Unify categories as the "explicit" categories.

2 years agoRename ctables_freq to ctables_cell.
Ben Pfaff [Mon, 3 Jan 2022 01:38:32 +0000 (17:38 -0800)]
Rename ctables_freq to ctables_cell.

2 years agocategory sort implicitly
Ben Pfaff [Sun, 2 Jan 2022 23:07:02 +0000 (15:07 -0800)]
category sort implicitly

2 years agoSorting categories by explicit values.
Ben Pfaff [Sun, 2 Jan 2022 22:40:49 +0000 (14:40 -0800)]
Sorting categories by explicit values.

2 years agocategory filtering works
Ben Pfaff [Sun, 2 Jan 2022 22:31:34 +0000 (14:31 -0800)]
category filtering works

2 years agotable sections exist
Ben Pfaff [Sun, 2 Jan 2022 21:37:44 +0000 (13:37 -0800)]
table sections exist

2 years agodomains work
Ben Pfaff [Sun, 2 Jan 2022 20:49:26 +0000 (12:49 -0800)]
domains work

2 years agosubtables work
Ben Pfaff [Sun, 2 Jan 2022 19:57:59 +0000 (11:57 -0800)]
subtables work

2 years agoCTABLES: Support setting the title, caption, ...
Ben Pfaff [Sun, 2 Jan 2022 03:27:42 +0000 (19:27 -0800)]
CTABLES: Support setting the title, caption, ...

2 years agopivot-table: New functions for setting captions, etc.
Ben Pfaff [Sun, 2 Jan 2022 03:27:31 +0000 (19:27 -0800)]
pivot-table: New functions for setting captions, etc.

2 years agoscale variables work more sanely
Ben Pfaff [Sun, 2 Jan 2022 03:04:45 +0000 (19:04 -0800)]
scale variables work more sanely

2 years agosummaries work
Ben Pfaff [Sun, 2 Jan 2022 01:41:50 +0000 (17:41 -0800)]
summaries work

2 years agodata filled in
Ben Pfaff [Sun, 2 Jan 2022 01:23:38 +0000 (17:23 -0800)]
data filled in

2 years agoaxes marked properly
Ben Pfaff [Sun, 2 Jan 2022 00:47:29 +0000 (16:47 -0800)]
axes marked properly

2 years agotoward better summaries
Ben Pfaff [Sat, 1 Jan 2022 23:47:23 +0000 (15:47 -0800)]
toward better summaries

2 years agodimensions work, except for sumamries
Ben Pfaff [Sat, 1 Jan 2022 22:47:52 +0000 (14:47 -0800)]
dimensions work, except for sumamries

2 years agooutput-item: Fix memory leak in output_item_dump().
Ben Pfaff [Sat, 1 Jan 2022 21:31:39 +0000 (13:31 -0800)]
output-item: Fix memory leak in output_item_dump().

2 years agowork
Ben Pfaff [Sat, 1 Jan 2022 18:09:12 +0000 (10:09 -0800)]
work

2 years agotrack axes
Ben Pfaff [Fri, 31 Dec 2021 20:09:27 +0000 (12:09 -0800)]
track axes

2 years agosome summary functions work
Ben Pfaff [Fri, 31 Dec 2021 05:34:47 +0000 (21:34 -0800)]
some summary functions work

2 years agogive CTABLES its own freq structure
Ben Pfaff [Fri, 31 Dec 2021 00:43:29 +0000 (16:43 -0800)]
give CTABLES its own freq structure

2 years agovlabels works a little
Ben Pfaff [Thu, 30 Dec 2021 23:58:42 +0000 (15:58 -0800)]
vlabels works a little

2 years agoall but last category
Ben Pfaff [Thu, 30 Dec 2021 23:14:15 +0000 (15:14 -0800)]
all but last category

2 years agoSkipping missing values.
Ben Pfaff [Thu, 30 Dec 2021 22:18:45 +0000 (14:18 -0800)]
Skipping missing values.

2 years agoCTABLES basics work
Ben Pfaff [Thu, 30 Dec 2021 21:58:21 +0000 (13:58 -0800)]
CTABLES basics work

2 years agofreqtabs are per-table
Ben Pfaff [Thu, 30 Dec 2021 19:47:17 +0000 (11:47 -0800)]
freqtabs are per-table

2 years agosort the frequency tables
Ben Pfaff [Thu, 30 Dec 2021 19:37:09 +0000 (11:37 -0800)]
sort the frequency tables

2 years agonow builds frequency tables
Ben Pfaff [Wed, 29 Dec 2021 22:46:47 +0000 (14:46 -0800)]
now builds frequency tables

2 years agofigure out frequency table structure
Ben Pfaff [Wed, 29 Dec 2021 19:57:06 +0000 (11:57 -0800)]
figure out frequency table structure

2 years agomore work
Ben Pfaff [Tue, 28 Dec 2021 05:03:13 +0000 (21:03 -0800)]
more work

2 years agomore validation
Ben Pfaff [Tue, 28 Dec 2021 02:43:03 +0000 (18:43 -0800)]
more validation

2 years agostart validating
Ben Pfaff [Tue, 28 Dec 2021 01:51:33 +0000 (17:51 -0800)]
start validating

2 years agowork on CTABLES
Ben Pfaff [Mon, 27 Dec 2021 00:01:47 +0000 (16:01 -0800)]
work on CTABLES

2 years agoMakefile: Fix running "make dist" before "make all".
Ben Pfaff [Sun, 13 Mar 2022 20:33:51 +0000 (13:33 -0700)]
Makefile: Fix running "make dist" before "make all".

Otherwise, there's a race between generating and using config.h, which
causes build failures.  This happens particularly with the PSPP autobuilder.

2 years agoSwitch from recursive to non-recursive use of Gnulib.
Ben Pfaff [Sun, 13 Mar 2022 19:34:36 +0000 (12:34 -0700)]
Switch from recursive to non-recursive use of Gnulib.

This allows for better use of dependencies in Makefiles.  It causes a new
warning from Automake, which I hope to fix through a change to Gnulib
sometime soon.  I'm consulting the other Gnulib maintainers on that:
https://lists.gnu.org/archive/html/bug-gnulib/2022-03/msg00044.html

2 years agoGnulib needs Automake 1.14 or later.
Ben Pfaff [Sun, 13 Mar 2022 19:12:53 +0000 (12:12 -0700)]
Gnulib needs Automake 1.14 or later.

2 years agotests: Update testsuite results to accomodate new Gnulib.
Ben Pfaff [Sun, 13 Mar 2022 18:41:50 +0000 (11:41 -0700)]
tests: Update testsuite results to accomodate new Gnulib.

It seems that libunistring now does line wrapping slightly differently,
perhaps due to Unicode updates.

2 years agoUpdate to newest gnulib.
Ben Pfaff [Sun, 13 Mar 2022 18:32:58 +0000 (11:32 -0700)]
Update to newest gnulib.

This required a dependency on Automake 1.11 or later.  It also required
dropping C2x compatibility warnings because Gnulib now triggers them.  The
Gnulib tempname module requires linking against $(LIB_GETRANDOM), so this
adds that in each case where the build links against libgl.

2 years agospv-light-decoder: Avoid GCC 11.x warning in decode_spvlb_halign().
Ben Pfaff [Sun, 13 Mar 2022 18:30:49 +0000 (11:30 -0700)]
spv-light-decoder: Avoid GCC 11.x warning in decode_spvlb_halign().

It seems to be wrong, but GCC can't tell and it's hard to blame it.

2 years agoexpressions: Fix value of $TIME variable and add a test.
Ben Pfaff [Sun, 13 Mar 2022 17:55:30 +0000 (10:55 -0700)]
expressions: Fix value of $TIME variable and add a test.

Thanks to Frans Houweling for reporting this bug.
Bug #62003.

2 years agopsppire-dialog-action-crosstabs: Do not add PIVOT or NOPIVOT to command.
Ben Pfaff [Thu, 13 Jan 2022 05:50:54 +0000 (21:50 -0800)]
psppire-dialog-action-crosstabs: Do not add PIVOT or NOPIVOT to command.

The CROSSTABS command does not support /FORMAT=PIVOT or /FORMAT=NOPIVOT.
I can't find any evidence that it ever did.  Perhaps q2c ignored keywords
it didn't recognize?  SPSS does not document support for any such format
keywords, so it's better to remove them from the emitted syntax than to
add support to CROSSTABS.

Thanks to Alexandra Younger for reporting the issue.

2 years agolexer: New functions for parsing real numbers in specified ranges.
Ben Pfaff [Sat, 1 Jan 2022 19:10:05 +0000 (11:10 -0800)]
lexer: New functions for parsing real numbers in specified ranges.

These will be more useful in the future.

2 years agoformat-parser: Improve comment.
Ben Pfaff [Sat, 1 Jan 2022 19:09:49 +0000 (11:09 -0800)]
format-parser: Improve comment.

2 years agoFREQUENCIES: Fix percentiles and median calculation for multiple variables.
Ben Pfaff [Sat, 1 Jan 2022 19:01:00 +0000 (11:01 -0800)]
FREQUENCIES: Fix percentiles and median calculation for multiple variables.

The calculations here were using the same space each time for
calculating percentiles and median, but different areas are needed.  This
commit fixes the problem.

Bug #60728.
Thanks to elias tsolis for reporting the problem.

2 years agoFREQUENCIES: Remove unused pool.
Ben Pfaff [Sat, 1 Jan 2022 18:35:17 +0000 (10:35 -0800)]
FREQUENCIES: Remove unused pool.

2 years agoFREQUENCIES: Remove unused struct members.
Ben Pfaff [Sat, 1 Jan 2022 18:34:56 +0000 (10:34 -0800)]
FREQUENCIES: Remove unused struct members.

2 years agoFixed misleading results in the Kruskal-Wallis test
John Darrington [Sat, 1 Jan 2022 08:56:21 +0000 (09:56 +0100)]
Fixed misleading results in the Kruskal-Wallis test

Fixed misleading results if the lower bound was in fact higher
than the upper bound.

2 years agoReplace more uses of 'cnt' by 'n'.
Ben Pfaff [Fri, 31 Dec 2021 19:06:48 +0000 (11:06 -0800)]
Replace more uses of 'cnt' by 'n'.

This continues the work begun in commit 339f1956cc72 ("treewide: Replace
<name>_cnt by n_<name>s and <name>_cap by allocated_<name>.") by replacing
more uses of 'cnt' by 'n', which I think is a better choice these days.

2 years agowindows cross build: Add example build instructions in documentation
Friedrich Beckmann [Fri, 31 Dec 2021 17:40:43 +0000 (18:40 +0100)]
windows cross build: Add example build instructions in documentation

I added a description of the required steps to do a windows cross
build.

2 years agomoments: Spell out "variance" ("var" is easily confused with "variable").
Ben Pfaff [Fri, 31 Dec 2021 05:35:13 +0000 (21:35 -0800)]
moments: Spell out "variance" ("var" is easily confused with "variable").

2 years agoChange how checking for missing values works.
Ben Pfaff [Thu, 30 Dec 2021 21:58:12 +0000 (13:58 -0800)]
Change how checking for missing values works.

It's more flexible to have the functions that check for missing values
return what kind of missing value was found (or 0 if none), than to have
to pass in the kind of missing values to check for.

2 years agoexpressions: Major work to improve error messages.
Ben Pfaff [Sun, 19 Dec 2021 23:41:50 +0000 (15:41 -0800)]
expressions: Major work to improve error messages.

This makes the error messages related to expressions much better, by being
more precise about the location of the problem.  It also fixes a number of
documentation and compatibility issues encountered along the way, as well
as redoing the testsuite for expressions to make it easier to understand
and to work on.

2 years agoexpressions: Fix built sources.
Ben Pfaff [Sat, 25 Dec 2021 22:12:39 +0000 (14:12 -0800)]
expressions: Fix built sources.

The dependencies were incomplete for the built sources, which meant that
build errors often came up if one built particular targets, e.g. for
"make src/ui/terminal/pspp".  This fixes the problem.

2 years agoexpressions: Simplify type declarations for nodes.
Ben Pfaff [Fri, 17 Dec 2021 02:09:14 +0000 (18:09 -0800)]
expressions: Simplify type declarations for nodes.

The widespread availability of C99 anonymous structs and unions helps to
simplify the node structure.

This will make it a lot easier to add a new member to the expression nodes
for source locators in an upcoming commit.

2 years agoConvert all Perl build tools to Python and remove Perl build dependency.
Ben Pfaff [Thu, 16 Dec 2021 05:00:32 +0000 (21:00 -0800)]
Convert all Perl build tools to Python and remove Perl build dependency.

Python is much more commonly used these days and easier for beginners
to pick up.

2 years agonum-out-cmp.pl: Remove unused script.
Ben Pfaff [Thu, 16 Dec 2021 22:08:06 +0000 (14:08 -0800)]
num-out-cmp.pl: Remove unused script.

2 years agoMATRIX: Update EVAL test to avoid failure on i386.
Ben Pfaff [Thu, 16 Dec 2021 04:20:48 +0000 (20:20 -0800)]
MATRIX: Update EVAL test to avoid failure on i386.

The i386 buildbot produced different precision for the eigenvalues
(integers rather than almost-integers), which isn't a significant
difference.  This commit should avoid the problem.

2 years agomatrix: Avoid gsl_linalg_LU_invx() because it was new in GSL 2.6.
Ben Pfaff [Thu, 16 Dec 2021 03:22:46 +0000 (19:22 -0800)]
matrix: Avoid gsl_linalg_LU_invx() because it was new in GSL 2.6.

Debian only added GSL 2.6 in the current stable (bullseye) but one of the
CI systems is using oldstable (buster), thus failures.

Thanks to Friedrich Beckmann for reporting the problem.

2 years agoperl-module: Skip memory test if Address Sanitizer is in use.
Ben Pfaff [Thu, 16 Dec 2021 02:56:45 +0000 (18:56 -0800)]
perl-module: Skip memory test if Address Sanitizer is in use.

Otherwise, Address Sanitizer produces unexpected memory behavior and
causes this test to fail even though there is no real leak.

2 years agoperl-module: Fix build errors and ignore warnings.
Ben Pfaff [Thu, 16 Dec 2021 02:55:44 +0000 (18:55 -0800)]
perl-module: Fix build errors and ignore warnings.

This allows the module to build again, for me.

2 years agoconfigure: Check for Perl modules needed to build PSPP's Perl module.
Ben Pfaff [Thu, 16 Dec 2021 02:54:47 +0000 (18:54 -0800)]
configure: Check for Perl modules needed to build PSPP's Perl module.

Fedora doesn't have Config::Perl::V by default and its absence caused
mysterious build failures.

2 years agoexpressions: Rewrite code generator in Python.
Ben Pfaff [Tue, 14 Dec 2021 05:06:23 +0000 (21:06 -0800)]
expressions: Rewrite code generator in Python.

This doesn't change the code that's generated at all.  It's preparation
for doing that later.

2 years agoFix build requirements by correctly noting that Python is required.
Ben Pfaff [Thu, 16 Dec 2021 01:11:32 +0000 (17:11 -0800)]
Fix build requirements by correctly noting that Python is required.

Building PSPP has required Python since support for the SPV format was
added, because the SPV parser-generators use Python.

Only Python 3+ can reasonably be supported, because support for Python 2.x
ended in April 2020.

2 years agoClean up how transformations work.
Ben Pfaff [Fri, 10 Dec 2021 06:23:59 +0000 (22:23 -0800)]
Clean up how transformations work.

Transformations had a messy system that included "finalizers" that needed
to be called before a chain could be executed and the need to keep track
of relative offsets for control flow.  This commit reworks and simplifies
the whole thing.

Also adds some INPUT PROGRAM, LOOP, and DO IF tests.

2 years agotreewide: Replace <name>_cnt by n_<name>s and <name>_cap by allocated_<name>.
Ben Pfaff [Thu, 9 Dec 2021 23:03:45 +0000 (15:03 -0800)]
treewide: Replace <name>_cnt by n_<name>s and <name>_cap by allocated_<name>.

I used to think that _cnt was a good suffix for a count representing the
number of items in an array, but I haven't used that convention in a long
time and it looks wrong now.  This changes these to an n_ prefix instead.

Similarly, but in far fewer cases, replace <name>_cap for the allocated
capacity of an array by allocated_<name>.

This doesn't change the name of ref_cnt members used as reference counts.
Those are pure counts, not the length of anything.

This doesn't change the Perl library because that's an external API.

2 years agoImplement the MATRIX command.
Ben Pfaff [Mon, 27 Sep 2021 05:35:44 +0000 (22:35 -0700)]
Implement the MATRIX command.

2 years agomessage: Introduce underlining for error message regions.
Ben Pfaff [Mon, 6 Dec 2021 06:03:31 +0000 (22:03 -0800)]
message: Introduce underlining for error message regions.

This will allow for the upcoming matrix language implementation
to give better error messages that clearly show the erroneous
part of the source.

2 years agomessage: Intern file names in msg_location to make them cheaper to copy.
Ben Pfaff [Thu, 25 Nov 2021 01:00:57 +0000 (17:00 -0800)]
message: Intern file names in msg_location to make them cheaper to copy.

2 years agolexer: Allow for negative lookahead within a command.
Ben Pfaff [Tue, 23 Nov 2021 03:16:48 +0000 (19:16 -0800)]
lexer: Allow for negative lookahead within a command.

This allows to getting the location of prior tokens within a command,
which makes it easier to find the end of a group of tokens that was just
parsed by referring to the token at offset -1.

2 years agolexer: Drop 'line_pos' from lex_token and lex_source.
Ben Pfaff [Mon, 22 Nov 2021 20:46:19 +0000 (12:46 -0800)]
lexer: Drop 'line_pos' from lex_token and lex_source.

It can now be calculated by reading backward in the source buffer.